Video coding method and device

Abstract

The invention discloses a video coding method and video coding equipment. The method comprises the following steps: carrying out video coding according to an initially set key frame interval; obtaining the sizes of the first two non-key frames of which the ith precoding is the key frame, wherein i is greater than 1; calculating to obtain the size variation of the two non-key frames; and according to the size variation of the two non-key frames, determining whether the ith video frame pre-coded as the key frame is coded as the non-key frame or the key frame. According to the method, whether the video frame code of which the ith precoding is the key frame is the key frame or the non-key frame is dynamically determined, so that the interval of the key frames is dynamically changed. The adjustment of the key frame interval mainly considers the non-key frame size variation and / or the illegal or abnormal behavior recognition condition in the coded video, so that the key frame interval is increased, the number of key frames is reduced and the video size is reduced under the condition of ensuring the picture quality requirement.

Description

Technical Field

[0001] The present invention relates to the technical field of video coding transmission, and in particular to a video coding method and device. Background Art

[0002] In most video acquisition terminals, such as recorders and surveillance cameras, the key frame interval for video encoding is generally fixed, for example, set to 1s-2s. The key frame interval affects the size of the video file. The smaller the interval, the larger the video file size. When the video is transmitted over the network, the network bandwidth occupied is large, which can easily cause network congestion and slow transmission speed. In order to reduce the size of the video file, blindly setting the key frame interval to a larger value will cause the quality of the image frames far away from the I frame to deteriorate a lot, resulting in blurred images, and failing to balance the relationship between image quality and reducing video size. Therefore, there is an urgent need for a video encoding method that can both ensure image quality and minimize video size. Summary of the invention

[0003] The purpose of the present invention is to provide a video encoding method and device, which can balance the relationship between image quality and video size, and maximize the reduction of video size while ensuring image quality, so as to reduce the bandwidth occupied by transmitting video.

[0004] In one aspect, the present invention provides a video encoding method, comprising the following steps:

[0005] Video encoding is performed according to the initially set key frame interval;

[0006] Obtain the sizes of the first two non-key frames pre-coded as the i-th key frame, where i is greater than 1;

[0007] Calculate and obtain the size change of the two non-key frames;

[0008] According to the size variation of the two non-key frames, it is determined whether the i-th video frame pre-encoded as a key frame is encoded as a non-key frame or a key frame.

[0009] In some possible embodiments, determining whether to encode the i-th video frame pre-encoded as a key frame as a non-key frame or a key frame according to the size change of the two non-key frames specifically includes:

[0010] According to the encoding parameter, obtaining a first preset threshold;

[0011] Compare the relationship between the size change and a first preset threshold value, if the size change is lower than the first preset threshold value, encode the i-th video frame pre-encoded as a key frame as a non-key frame; otherwise, encode the i-th video frame pre-encoded as a key frame as a key frame.

[0012] In some possible embodiments, obtaining the first preset threshold according to the encoding parameter specifically includes:

[0013] The first preset threshold is obtained according to a mapping table of correspondence between encoding parameters and first preset thresholds, wherein the mapping table is a pre-established data table of relationships between encoding parameters and the first preset thresholds, and the encoding parameters include resolution, bit rate, and frame rate.

[0014] In some possible embodiments, the method further includes:

[0015] Obtain the video frame size variance of the first n non-key frames pre-encoded as key frames of the i-th frame, where the number of video frames in the GOP group is initially set to m, and n is less than m-1;

[0016] If the variance is less than the second preset threshold, the frame rate is adjusted: the variance is divided into several continuous intervals, and a frame rate adjustment coefficient is assigned to each interval. According to the frame rate adjustment coefficient, the frame rate used in the current encoding is reduced, and encoding is continued according to the adjusted frame rate.

[0017] In some possible embodiments, the method further includes:

[0018] Perform image recognition analysis on the encoded video frames to determine whether there are any violations or abnormal behaviors. Based on the violation or abnormal behavior identification and the size change, determine whether to encode the i-th video frame pre-encoded as a key frame as a non-key frame or a key frame.

[0019] In some possible embodiments, determining whether to encode the i-th video frame pre-encoded as a key frame as a non-key frame or a key frame according to the violation or abnormal behavior identification situation and the size change specifically includes:

[0020] According to the identification of illegal or abnormal behavior, the first adjustment value y1 is determined. When illegal or abnormal behavior is identified, the first adjustment value y1 is 1; if the behavior is compliant, the first adjustment value y1 is 0;

[0021] The second adjustment value y2 is determined according to the size change. The size change can be divided into multiple continuous intervals, and a second adjustment value y2 is assigned to each interval. The larger the size change, the larger the second adjustment value y2. The first adjustment value and the second adjustment value are weighted summed to obtain the key frame adjustment coefficient: y=w1×y1+w2×y2, where w1 and w2 are weight coefficients, and the weight setting is optimized through training data;

[0022] If the key frame adjustment coefficient y is greater than a third preset threshold, the i-th video frame pre-encoded as a key frame is encoded as a key frame; otherwise, the i-th video frame pre-encoded as a key frame is encoded as a non-key frame.

[0023] In some possible embodiments, the present invention further includes:

[0024] Obtain the network bandwidth change rate and the network bandwidth of the current time period. According to the network bandwidth and the network bandwidth change rate of the current time period, obtain the network bandwidth evaluation value of the time period after the current time period. The moment when the previous non-key frame pre-coded as the key frame is encoded is the current moment. The second before the current moment is the current time period. i is greater than 1. The duration of each time period is 1s.

[0025] The ratio of the bit rate in the current time period to the network bandwidth evaluation value is calculated. If the ratio is higher than a fourth threshold, the compression ratio of the video frame is adjusted, and the i-th video frame pre-encoded as a key frame is encoded according to the adjusted compression ratio.

[0026] In some possible embodiments, obtaining the network bandwidth change rate in the current time period specifically includes:

[0027] A preset first number of historical time periods before the current time period is used as partial historical time periods of the current time period, and among the partial historical time periods of the current time period, a preset second number of historical time periods closest to the current time period is used as adjacent historical time periods of the current time period, where the second number is smaller than the first number;

[0028] The average value A of the network bandwidth of all adjacent historical time periods of the current time period and the average value B of the network bandwidth of all partial historical time periods of the current time period are obtained, and the ratio of A to B is used as the network bandwidth change rate of the current time period.

[0029] In some possible embodiments, the compression ratio of the video frame is adjusted to: In the formula, Z is the bit rate of the current time period, M is the network bandwidth evaluation value of the time period after the current time period, g is the video compression ratio of the current time period, is the estimated value of the compression ratio when the bandwidth occupancy rate is 100%, and k is the fourth threshold.

[0030] On the other hand, the present invention also provides a video encoding device, including a processor and a memory, wherein the processor and the memory are connected, wherein the memory is used to store program code, and the processor is used to call the program code to execute the above-mentioned video encoding method.

[0031] The beneficial effects of the present invention are as follows: by dynamically determining whether the i-th video frame pre-encoded as a key frame is encoded as a key frame or a non-key frame based on the initially set GOP group size, the key frame interval is dynamically changed. The adjustment of the key frame interval mainly considers the change in the size of the non-key frame and / or the identification of illegal or abnormal behavior in the encoded video, so as to increase the key frame interval, reduce the number of key frames, and reduce the video size while ensuring the picture quality requirements;

[0032] Dynamically reduce the frame rate used for encoding and reduce the video size based on the variance of the video frame size of non-key frames;

[0033] Adjust the video encoding compression level based on bandwidth changes to reduce the video file size, thereby ensuring stable and smooth video transmission even when the network bandwidth environment is poor. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A flowchart of a video encoding method provided by an embodiment of the present invention;

[0036] Figure 2 A functional block diagram of a video stream encoding device provided by an embodiment of the present invention; DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0038] like Figure 1 As shown, a video encoding method comprises the following steps:

[0039] S1, video encoding according to the initially set key frame interval;

[0040] For video encoding devices, encoding parameters are initially set before encoding, and then encoding is started.

[0041] S2. Obtain the sizes of the first two non-key frames pre-coded as the i-th key frame, where i is greater than 1;

[0042] In general encoding devices, after the video encoding device starts encoding, the encoding of the first key frame is completed first, and then the non-key frame is encoded. After the encoding of a set GOP is completed, the encoding of the next GOP is performed. The interval between key frames is fixed, and a GOP contains a key frame and several non-key frames. However, after the first GOP set initially is completed, before encoding the i-th pre-encoded key frame, the present invention needs to obtain the size of the first two non-key frames of the i-th pre-encoded key frame, where i is greater than 1, and the size of the frame refers to the amount of bit data contained.

[0043] For example, a GOP contains 5 video frames, so the interval between key frames is 5. After completing the encoding of the first GOP, before encoding the second pre-encoded key frame, the sizes of the first two key frames of the second pre-encoded key frame will be obtained. Assuming that the combination in the first GOP is I, P, P, B, P, the sizes of the last two B frames and P frame will be obtained.

[0044] S3, calculating and obtaining the size change of the two non-key frames;

[0045] According to the above example, the size changes of the next two B frames and P frame are calculated.

[0046] S4. Determine whether to encode the i-th video frame pre-encoded as a key frame as a non-key frame or a key frame according to the size change of the two non-key frames.

[0047] Before encoding the i-th pre-encoded key frame, the embodiment of the present invention will determine whether to encode the video frame as a key frame according to the initial key frame interval, and one of the reference factors in the judgment is the size change of the two non-key frames.

[0048] The above encoding method is very suitable for scenes where the content of the picture changes little in some time periods or all the time. According to the initially determined key frame interval, before pre-encoding the key frame, the size change of the two non-key frames before the pre-encoded key frame is obtained, and according to the size change, the key frame interval is increased to reduce the video size without affecting the picture quality.

[0049] In some possible embodiments, determining whether to encode the i-th video frame pre-encoded as a key frame as a non-key frame or a key frame according to the size change of the two non-key frames specifically includes:

[0050] According to the encoding parameter, obtaining a first preset threshold;

[0051] Compare the relationship between the size change and a first preset threshold value, if the size change is lower than the first preset threshold value, encode the i-th video frame pre-encoded as a key frame as a non-key frame; otherwise, encode the i-th video frame pre-encoded as a key frame as a key frame.

[0052] In some possible embodiments, obtaining the first preset threshold according to the encoding parameter specifically includes:

[0053] The first preset threshold is obtained according to a mapping table of correspondence between encoding parameters and first preset thresholds, wherein the mapping table is a pre-established data table of relationships between encoding parameters and the first preset thresholds, and the encoding parameters include resolution, bit rate, and frame rate.

[0054] The first preset threshold value under different coding parameter combinations is different. Assuming that the number of values ​​of bit rate, frame rate and resolution are i, j and k respectively, the first preset threshold value under i×j×k coding parameters is determined in advance by debugging.

[0055] In some possible embodiments, the present invention further includes:

[0056] S5, obtaining the video frame size variance of the first n non-key frames pre-encoded as the key frame of the ith video frame, the number of video frames in the GOP group is initially set to be m, and n is less than m-1;

[0057] S6. If the variance is less than the second preset threshold, the frame rate is adjusted: the variance is divided into several continuous intervals, and a frame rate adjustment coefficient is assigned to each interval. According to the frame rate adjustment coefficient, the frame rate used in the current encoding is reduced, and encoding is continued according to the adjusted frame rate.

[0058] Frame rate is another key factor affecting video size. The variance of non-key frame size represents the size fluctuation between frames and can reflect the rate of change of the screen content over a period of time. For scenes with very small rates of change of screen content, reducing the frame rate appropriately can reduce the video size without affecting the effect of the supervision screen.

[0059] In some possible embodiments, the method of the present invention further comprises:

[0060] Perform image recognition analysis on the encoded video frames to determine whether there are any violations or abnormal behaviors. Based on the violation or abnormal behavior identification and the size change, determine whether to encode the i-th video frame pre-encoded as a key frame as a non-key frame or a key frame.

[0061] Image recognition analysis can be specifically performed by detecting the target motion area from the video frame image through the target detection algorithm, and then extracting the behavior features to obtain the spatial features and time features, sending the spatial features and time features to the feature fusion module for feature fusion to obtain the fused features, and using the designed support vector machine classification model to perform behavior classification and recognition to obtain the analysis results of whether there are any violations or abnormal behaviors. You can select the appropriate target detection algorithm and classification algorithm according to the applicable scenario to complete the recognition and analysis of the corresponding behavior.

[0062] The embodiment of the present invention comprehensively considers the size change (obtained in step S3) and whether there are any violations or abnormal behaviors in the currently monitored video to dynamically adjust the interval of key frames.

[0063] In some possible embodiments, determining whether to encode the i-th video frame pre-encoded as a key frame as a non-key frame or a key frame according to the violation or abnormal behavior identification situation and the size change specifically includes:

[0064] According to the identification of illegal or abnormal behavior, the first adjustment value y1 is determined. When illegal or abnormal behavior is identified, the first adjustment value y1 is 1; if the behavior is compliant, the first adjustment value y1 is 0;

[0065] The second adjustment value y2 is determined according to the size change. The size change can be divided into multiple continuous intervals, and a second adjustment value y2 is assigned to each interval. The larger the size change, the larger the second adjustment value y2. The first adjustment value and the second adjustment value are weighted summed to obtain the key frame adjustment coefficient: y=w1×y1+w2×y2, where w1 and w2 are weight coefficients, and the weight setting is optimized through training data;

[0066] If the key frame adjustment coefficient y is greater than a third preset threshold, the i-th video frame pre-encoded as a key frame is encoded as a key frame; otherwise, the i-th video frame pre-encoded as a key frame is encoded as a non-key frame.

[0067] The embodiment of the present invention comprehensively considers the change in non-key frame size and the identification of violations or abnormal behaviors in the encoded video, so as to increase the key frame interval, reduce the number of key frames, and reduce the video size when there are no violations or abnormal behaviors and the picture changes are small.

[0068] In some possible embodiments, the present invention further includes:

[0069] Obtain the network bandwidth change rate and the network bandwidth of the current time period, and obtain the network bandwidth evaluation value of the time period after the current time period according to the network bandwidth and the network bandwidth change rate of the current time period;

[0070] The moment when the previous non-key frame pre-coded as the i-th key frame is encoded is the current moment, and the second before the current moment is the current time period. i is greater than 1, and the length of each time period is 1 second.

[0071] Get the network bandwidth change rate in the current time period. The specific method is:

[0072] A preset first number of historical time periods before the current time period is used as partial historical time periods of the current time period, and among the partial historical time periods of the current time period, a preset second number of historical time periods closest to the current time period is used as adjacent historical time periods of the current time period, where the second number is smaller than the first number;

[0073] The average value A of the network bandwidth of all adjacent historical time periods of the current time period and the average value B of the network bandwidth of all partial historical time periods of the current time period are obtained, and the ratio of A to B is used as the network bandwidth change rate of the current time period.

[0074] The larger the ratio of A to B is, the closer it is to the current time period, the better the network bandwidth is, and it is positively correlated with the network bandwidth.

[0075] The product of the network bandwidth of the current time period and the network bandwidth change rate is used as the network bandwidth evaluation value of the time period after the current period;

[0076] The ratio of the bit rate in the current time period to the network bandwidth evaluation value is calculated. If the ratio is higher than a fourth threshold, the compression ratio of the video frame is adjusted, and the i-th video frame pre-encoded as a key frame is encoded according to the adjusted compression ratio.

[0077] The ratio of the bit rate in the current time period to the network bandwidth evaluation value can be used to estimate the network bandwidth occupancy rate in the time period after the current period. When the occupancy rate is higher than a certain value, such as 80%, it is easy to cause network congestion.

[0078] Adjust the compression ratio of the video frame to: In the formula, Z is the bit rate of the current time period, M is the network bandwidth evaluation value of the time period after the current time period, g is the video compression ratio of the current time period, is the estimated value of the compression ratio when the bandwidth occupancy rate is 100%, and k is the fourth threshold.

[0079] Network bandwidth fluctuates over time. In order to ensure that videos have a certain degree of smoothness during network transmission, it is necessary to adjust the video encoding compression level based on bandwidth changes to reduce the video file size, thereby ensuring that the video can be transmitted stably and smoothly even when the network bandwidth environment is poor.

[0080] In an embodiment of the present invention, after completing the first GOP that is initially set, before encoding the i-th pre-encoded key frame (i is greater than 1), the network bandwidth evaluation value of the time period after the current time period, the bit rate of the current time period, the video compression ratio of the current time period and the fourth threshold are calculated and evaluated, and the video encoding compression ratio is adjusted. According to the adjusted compression ratio, the i-th video frame pre-encoded as the key frame is encoded to reduce the video size and avoid video file transmission congestion caused by deterioration of network bandwidth.

[0081] like Figure 2 As shown, the present invention also discloses a video encoding device, including a processor and a memory, wherein the processor and the memory are connected, wherein the memory is used to store program code, and the processor is used to call the program code to execute the video encoding method as described above.

[0082] The above is a detailed introduction to a video encoding method and device provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method and core ideas of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0083] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A video encoding method, characterized in that: The steps include: Video encoding is performed according to the initially set key frame interval; Obtain the sizes of the first two non-key frames pre-coded as the i-th key frame, where i is greater than 1; Calculate and obtain the size change of the two non-key frames; According to the size variation of the two non-key frames, it is determined whether the i-th video frame pre-encoded as a key frame is encoded as a non-key frame or a key frame.

2. The video encoding method according to claim 1, characterized in that The step of determining whether to encode the i-th video frame pre-encoded as a key frame as a non-key frame or a key frame according to the size change of the two non-key frames specifically includes: According to the encoding parameter, obtaining a first preset threshold; Compare the relationship between the size change and a first preset threshold value, if the size change is lower than the first preset threshold value, encode the i-th video frame pre-encoded as a key frame as a non-key frame; otherwise, encode the i-th video frame pre-encoded as a key frame as a key frame.

3. The video encoding method according to claim 2, characterized in that: The obtaining, according to the encoding parameter, a first preset threshold value specifically includes: The first preset threshold is obtained according to a mapping table of correspondence between encoding parameters and first preset thresholds, wherein the mapping table is a pre-established data table of relationships between encoding parameters and the first preset thresholds, and the encoding parameters include resolution, bit rate, and frame rate.

4. The video encoding method according to claim 1, characterized in that: The method also includes: Obtain the video frame size variance of the first n non-key frames pre-encoded as key frames of the i-th frame, where the number of video frames in the GOP group is initially set to m, and n is less than m-1; If the variance is less than the second preset threshold, the frame rate is adjusted: the variance is divided into several continuous intervals, and a frame rate adjustment coefficient is assigned to each interval. According to the frame rate adjustment coefficient, the frame rate used in the current encoding is reduced, and encoding is continued according to the adjusted frame rate.

5. The video encoding method according to claim 1, characterized in that: The method also includes: Perform image recognition analysis on the encoded video frames to determine whether there are any violations or abnormal behaviors. Based on the violation or abnormal behavior identification and the size change, determine whether to encode the i-th video frame pre-encoded as a key frame as a non-key frame or a key frame.

6. The video encoding method according to claim 5, characterized in that: The step of determining whether to encode the i-th video frame pre-encoded as a key frame as a non-key frame or a key frame according to the violation or abnormal behavior identification situation and the size change specifically includes: According to the identification of illegal or abnormal behavior, the first adjustment value y1 is determined. When illegal or abnormal behavior is identified, the first adjustment value y1 is 1; if the behavior is compliant, the first adjustment value y1 is 0; The second adjustment value y2 is determined according to the size change. The size change can be divided into multiple continuous intervals, and a second adjustment value y2 is assigned to each interval. The larger the size change, the larger the second adjustment value y2. The first adjustment value and the second adjustment value are weighted summed to obtain the key frame adjustment coefficient: y = w1 × y1 + w2 × y2, where w1 and w2 are weight coefficients, and the weight setting is optimized through training data; If the key frame adjustment coefficient y is greater than a third preset threshold, the i-th video frame pre-encoded as a key frame is encoded as a key frame; otherwise, the i-th video frame pre-encoded as a key frame is encoded as a non-key frame.

7. The video encoding method according to claim 1, characterized in that: Also includes: Obtain the network bandwidth change rate and the network bandwidth of the current time period. According to the network bandwidth and the network bandwidth change rate of the current time period, obtain the network bandwidth evaluation value of the time period after the current time period. The moment when the previous non-key frame pre-coded as the key frame is encoded is the current moment. The second before the current moment is the current time period. i is greater than 1. The duration of each time period is 1s. The ratio of the bit rate in the current time period to the network bandwidth evaluation value is calculated. If the ratio is higher than a fourth threshold, the compression ratio of the video frame is adjusted, and the i-th video frame pre-encoded as a key frame is encoded according to the adjusted compression ratio.

8. The video encoding method according to claim 7, characterized in that: Get the network bandwidth change rate for the current time period, including: A preset first number of historical time periods before the current time period is used as partial historical time periods of the current time period, and among the partial historical time periods of the current time period, a preset second number of historical time periods closest to the current time period is used as adjacent historical time periods of the current time period, where the second number is smaller than the first number; The average value A of the network bandwidth of all adjacent historical time periods of the current time period and the average value B of the network bandwidth of all partial historical time periods of the current time period are obtained, and the ratio of A to B is used as the network bandwidth change rate of the current time period.

9. The video encoding method according to claim 7, characterized in that: Adjust the compression ratio of the video frame to: In the formula, Z is the bit rate of the current time period, M is the network bandwidth evaluation value of the time period after the current time period, g is the video compression ratio of the current time period, is the estimated value of the compression ratio when the bandwidth occupancy rate is 100%, and k is the fourth threshold.

10. A video encoding device, characterized in that: It comprises a processor and a memory, wherein the processor and the memory are connected, wherein the memory is used to store program codes, and the processor is used to call the program codes to execute the video encoding method according to any one of claims 1 to 9.

Images