Video encoding method, device and electronic device

Abstract

The present invention provides a video encoding method, device and electronic device, which obtain a video scene to which a first frame of video image in a video to be encoded belongs; obtain a first bit rate matching the video scene, and encode the first frame of video image based on the first bit rate; for each frame of video image except the first frame of video image, based on the acquired video complexity of the current video image, the packet loss rate at the current moment and the first bit rate, determine a second bit rate matching the current video image, and encode the current video image based on the second bit rate. After obtaining the first bit rate corresponding to the first frame of video image, for each frame of video image except the first frame of video image, the method can make a real-time decision and update the second bit rate corresponding to the current video image based on the first bit rate, the video complexity of the current video image, and the packet loss rate at the current moment, and encode each frame of video image using its corresponding bit rate, thereby improving bit rate utilization and encoding efficiency.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a video encoding method, device and electronic equipment. Background Art

[0002] With the large-scale application of RTC (Real-Time Communication) in scenarios such as live streaming and live teaching, video bit rate and video quality closely related to RTC are becoming more and more important. In related technologies, a target bit rate is usually pre-set for a video based on user experience, and each frame of video in the video is uniformly encoded using the target bit rate. Since the quality and content complexity of each frame of video in the video may be different, and video images of different quality and content complexity are usually suitable for different video bit rates. Some video images require a higher bit rate to ensure video quality, while some video images only require a lower bit rate to ensure video quality. Therefore, this method of using a fixed bit rate to encode video images of different quality and content complexity will lead to low bit rate utilization and low coding efficiency. Summary of the invention

[0003] The object of the present invention is to provide a video encoding method, device and electronic equipment to improve bit rate utilization and encoding efficiency.

[0004] The present invention provides a video encoding method, which includes: obtaining a video image in a video to be encoded, and a video scene to which a first frame of video image belongs; obtaining a first bit rate matching the video scene, and encoding the first frame of video image based on the first bit rate; for each frame of video image except the first frame of video image, obtaining the video complexity of the current video image and the packet loss rate at the current moment; determining a second bit rate matching the current video image based on the video complexity of the current video image, the packet loss rate at the current moment and the first bit rate, and encoding the current video image based on the second bit rate.

[0005] Furthermore, based on the video complexity of the current video image, the packet loss rate at the current moment and the first bit rate, the step of determining a second bit rate that matches the current video image includes: determining a first parameter according to the video complexity of the current video image and the packet loss rate at the current moment; wherein the first parameter decreases as the packet loss rate at the current moment increases; the higher the video complexity, the slower the rate at which the first parameter decreases; and multiplying the first bit rate by the first parameter to obtain a second bit rate that matches the current video image.

[0006] Furthermore, the linear curve model is:

[0007] Second code rate = first code rate * f(p, r, x);

[0008] Wherein, p is a preset parameter coefficient; r represents the video complexity of the current video image; and x represents the packet loss rate at the current moment.

[0009] Furthermore, the step of obtaining a video image in the video to be encoded and a video scene to which the first frame of the video image belongs includes: obtaining a video image in the video to be encoded; inputting the first frame of the video image in the video image into a pre-trained neural network model to determine the video scene to which the first frame of the video image belongs through the neural network model.

[0010] Furthermore, the first bit rate matching the video scene is determined in advance in the following manner: obtaining a plurality of frames of video image samples belonging to the video scene; determining at least one preset bit rate within the preset bit rate range at a preset interval within a preset bit rate range; encoding each frame of the video image sample at at least one preset bit rate, and determining at least one image quality of each frame of the encoded video image sample; performing curve fitting processing on at least one image quality and the preset bit rate corresponding to each image quality, and determining a fitting curve matching the video scene; and determining the first bit rate matching the video scene based on the fitting curve.

[0011] Furthermore, the image quality is represented by a quality score; based on the fitting curve, the step of determining a first bit rate matching the video scene comprises: selecting a target position point on the fitting curve; wherein, taking the bit rate corresponding to the target position point as a starting point, as the bit rate gradually increases, the absolute value of the difference between the quality score corresponding to the increased bit rate and the quality score corresponding to the target position point is within a preset range; and determining the bit rate corresponding to the target position point as the first bit rate matching the video scene.

[0012] Furthermore, for each frame of video image except the first frame of video image, the step of obtaining the video complexity of the current video image includes: for each frame of video image except the first frame of video image, calculating the absolute value of the difference between the pixel value at each pixel position in the current video image and the pixel value at the same pixel position in the previous frame of video image; based on multiple absolute values ​​of the difference, determining the first number of pixels whose absolute value of the difference is greater than a preset threshold; calculating the ratio between the first number of pixels and the total number of pixels in the current video image, and determining the ratio as the video complexity of the current video image.

[0013] The present invention provides a video encoding device, which includes: a first acquisition module, used to acquire a video image in a video to be encoded, and a video scene to which a first frame of video image belongs; a first encoding module, used to acquire a first bit rate matching the video scene, and encode the first frame of video image based on the first bit rate; a second acquisition module, used to acquire, for each frame of video image except the first frame of video image, the video complexity of the current video image and the packet loss rate at a current moment; a second encoding module, used to determine a second bit rate matching the current video image based on the video complexity of the current video image, the packet loss rate at a current moment and the first bit rate, and encode the current video image based on the second bit rate.

[0014] An electronic device provided by the present invention includes a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the above-mentioned video encoding method.

[0015] The present invention provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the above-mentioned video encoding method.

[0016] The video encoding method, device and electronic device provided by the present invention first obtain the video image in the video to be encoded, and the video scene to which the first frame of the video image belongs; obtain the first bit rate matching the video scene, and encode the first frame of the video image based on the first bit rate; then obtain the video complexity of the current video image and the packet loss rate at the current moment for each frame of the video image except the first frame of the video image; finally, determine the second bit rate matching the current video image based on the video complexity of the current video image, the packet loss rate at the current moment and the first bit rate, and encode the current video image based on the second bit rate. After obtaining the first bit rate corresponding to the first frame of the video image, for each frame of the video image except the first frame of the video image, the second bit rate corresponding to the current video image can be determined and updated in real time based on the first bit rate, the video complexity of the current video image, and the packet loss rate at the current moment, and each frame of the video image is encoded using the corresponding bit rate, thereby improving the bit rate utilization and encoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flowchart of a video encoding method provided by an embodiment of the present invention;

[0019] Figure 2 A flowchart of another video encoding method provided by an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a change curve of a linear curve model provided by an embodiment of the present invention;

[0021] Figure 4 A flowchart of another video encoding method provided by an embodiment of the present invention;

[0022] Figure 5 A flowchart of another video encoding method provided by an embodiment of the present invention;

[0023] Figure 6 A schematic diagram of the structure of a video encoding device provided by an embodiment of the present invention;

[0024] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] With the large-scale application of RTC real-time communication in scenarios such as live streaming and live teaching, user experience in complex network environments is particularly critical, especially when the network status is poor or there is network packet loss. How to achieve high-quality transmission under limited bandwidth requires accurate decision-making on the video encoding bit rate.

[0027] The related technology mainly determines the RTC bit rate based on the target bit rate and packet loss rate. Specifically, a target bit rate is pre-set for a certain video based on user experience, and each frame of the video image in the video is uniformly encoded using the target bit rate. Since the quality and content complexity of each frame of the video image in the video may be different, the fixed bit rate encoding method for video images of different quality and content complexity will lead to low bit rate utilization and low encoding efficiency. In addition, a certain target bit rate can be pre-set, and the bit rate can be appropriately increased or decreased according to the packet loss rate, such as appropriately reducing the bit rate when the packet loss rate increases, and appropriately increasing the bit rate when the packet loss rate decreases or there is no packet loss. This method is mainly based on a step or linear model to adjust the packet loss rate, and there are problems such as slow bit rate adjustment and untimely updates. Based on this, the embodiment of the present invention provides a video encoding method, device and electronic device, which can be applied to applications that require video encoding.

[0028] To facilitate understanding of this embodiment, a video encoding method disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, the method comprises the following steps:

[0029] Step S102, obtaining a video image in the video to be encoded, and a video scene to which the first frame of the video image belongs.

[0030] The above-mentioned video to be encoded usually includes multiple frames of video images, and the video scenes to which each frame of video image belongs may be different. For example, some video images belong to game scenes, some video images belong to educational and teaching scenes, etc. The video images can also be called video data; in actual implementation, when it is necessary to encode the video to be encoded, it is usually necessary to first obtain the video scene to which the first frame of video image in the video to be encoded belongs.

[0031] Step S104: acquiring a first bit rate matching the video scene, and encoding the first frame of video image based on the first bit rate.

[0032] The above-mentioned first bit rate can be understood as using the first bit rate to encode the first frame of video image belonging to the video scene, which can ensure that the quality of the encoded first frame of video image is high and will not bring great pressure on the network bandwidth; if a bit rate lower than the first bit rate is used to encode the first frame of video image, the image quality of the encoded first frame of video image will usually be significantly reduced; if a bit rate higher than the first bit rate is used to encode the first frame of video image, the image quality of the encoded first frame of video image will usually not be significantly improved, and will bring great pressure on the network bandwidth; therefore, after obtaining the video scene to which the first frame of video image belongs, the predetermined first bit rate matching the video scene can be obtained. For example, the first bit rate can be determined based on the pre-acquired fitting curve of the video scene matching, and the first bit rate can be used to encode the first frame of video image.

[0033] Step S106, for each video frame except the first video frame, obtain the video complexity of the current video image and the packet loss rate at the current moment.

[0034] The above-mentioned video complexity can be used to indicate the degree of similarity between the current video image and the previous frame of video image. Generally, the higher the video complexity, the smaller the degree of similarity between the current video image and the previous frame of video image, and the faster the video image changes; the lower the video complexity, the greater the degree of similarity between the current video image and the previous frame of video image, and the slower the video image changes; the video complexity can be calculated by frame difference method, optical flow method, deep learning and other methods. For details, please refer to the calculation method in related technologies, which is not limited here; the above-mentioned packet loss rate can be understood as the ratio of the number of lost data packets to the sent data group; in actual implementation, each frame of video image except the first frame of video image is used as the current video image, and the video complexity of the current video image can be calculated by frame difference method, optical flow method, deep learning and other methods. In addition, since the packet loss rate usually changes in real time, this embodiment also needs to obtain the packet loss rate corresponding to each frame of video image at the current moment.

[0035] Step S108, determining a second bit rate matching the current video image based on the video complexity of the current video image, the packet loss rate at the current moment, and the first bit rate, and encoding the current video image based on the second bit rate.

[0036] For each frame of video image except the first frame of video image, after calculating the video complexity of the current video image and the packet loss rate at the current moment, the second bit rate matching the current video image can be calculated in combination with the first bit rate matching the video scene to which the first frame of video image belongs. For example, the second bit rate can be calculated based on a linear curve model, and the current video image can be encoded using the second bit rate.

[0037] The above-mentioned video encoding method first obtains the video image in the video to be encoded, and the video scene to which the first frame of the video image belongs; obtains the first bit rate matching the video scene, and encodes the first frame of the video image based on the first bit rate; then, for each frame of the video image except the first frame of the video image, obtains the video complexity of the current video image and the packet loss rate at the current moment; finally, based on the video complexity of the current video image, the packet loss rate at the current moment and the first bit rate, determines the second bit rate matching the current video image, and encodes the current video image based on the second bit rate. After obtaining the first bit rate corresponding to the first frame of the video image, this method can make a real-time decision and update the second bit rate corresponding to the current video image based on the first bit rate, the video complexity of the current video image, and the packet loss rate at the current moment for each frame of the video image except the first frame of the video image, and encode each frame of the video image using its corresponding bit rate, thereby improving bit rate utilization and encoding efficiency.

[0038] The embodiment of the present invention further provides another video encoding method, which is implemented on the basis of the above-mentioned embodiment method; the method focuses on describing the specific process of determining the second bit rate matching the current video image based on the video complexity of the current video image, the packet loss rate at the current moment and the first bit rate, which specifically corresponds to the following steps S208 to S210; Figure 2 As shown, the method comprises the following steps:

[0039] Step S202, obtaining a video image in the video to be encoded, and a video scene to which the first frame of the video image belongs.

[0040] Step S204: Obtain a first bit rate that matches the video scene, and encode the first frame of video image based on the first bit rate.

[0041] Step S206: for each video frame except the first video frame, obtain the video complexity of the current video image and the packet loss rate at the current moment.

[0042] Step S208, determining a first parameter according to the video complexity of the current video image and the packet loss rate at the current moment, wherein the first parameter decreases as the packet loss rate at the current moment increases; the higher the video complexity, the slower the rate at which the first parameter decreases.

[0043] In actual implementation, the video complexity of the current video image and the packet loss rate at the current moment may be input into a preset linear curve model to obtain the first parameter.

[0044] The above linear curve model can be: Wherein, p is a preset parameter coefficient; r represents the video complexity of the current video image; and x represents the packet loss rate at the current moment.

[0045] It should be noted that the above p value can be set according to actual needs, for example, p can be set to 10, etc.; in order to reduce the complexity of the above linear curve model, the above 1-r can be expressed by k, that is, k=1-r, then the above linear curve model can be expressed as:

[0046] For the current video image, the acquired video complexity of the current video image and the packet loss rate at the current moment can be input into the above-mentioned linear curve model to obtain a calculation result, which is the above-mentioned first parameter. Combined with the above-mentioned linear curve model, it can be seen that the higher the packet loss rate at the current moment, the smaller the first parameter; the higher the video complexity, the slower the rate at which the first parameter decreases.

[0047] Step S210: multiply the first bit rate by the first parameter to obtain a second bit rate matching the current video image.

[0048] In actual implementation, the first bit rate determined above can be set as the maximum bit rate for encoding the video, and the following formula can be used to adjust, update and determine the specific bit rate for the current video image: current_bitrate=target_bitrate*f(p, k, x); wherein current_bitrate corresponds to the second bit rate; target_bitrate corresponds to the first bit rate; k=1-r; therefore, the formula can also be expressed as: second bit rate=first bit rate*f(p, r, x); the first parameter is the calculation result of the linear curve model f(p, r, x); the calculation formula can be used for each frame of video image except the first frame of video image to obtain the second bit rate matching each frame of video image.

[0049] When the video complexity is the same, the higher the packet loss rate at the current moment, the smaller the first parameter, so the second bit rate obtained according to the first parameter and the first bit rate is also smaller; the higher the image complexity of the current video image, the slower the rate at which the first parameter is reduced, so the rate at which the second bit rate is reduced according to the first parameter and the first bit rate is also slower. For example, see Figure 3 A schematic diagram of a variation curve of a linear curve model shown; Figure 3The solid line curve in is the mapping relationship between the packet loss rate x and f(p, r, x) when k=0.5, that is, the video complexity r=1-k=0.5, and the dotted line curve is the mapping relationship between the packet loss rate x and f(p, r, x) when k=0.2, that is, the video complexity r=1-k=0.8; for the same video image, the video complexity is a certain value. If the packet loss rate x at the current moment is higher, the calculated result of the first parameter, i.e., f(p, r, x), is smaller. Since the first bit rate corresponding to the first frame of the video image is fixed, the calculated second bit rate of the video image is smaller. On the contrary, if the packet loss rate x at the current moment is lower, the calculated result of the first parameter, i.e., f(p, r, x), is larger. Since the first bit rate corresponding to the first frame of the video image is fixed, the calculated second bit rate of the video image is larger.

[0050] Depend on Figure 3 It can also be seen that when the k value is smaller, that is, the video complexity is higher, as the packet loss rate x increases, the rate at which the calculation result of the first parameter, i.e., f(p, r, x), decreases is slower, and the corresponding rate at which the second bit rate decreases is slower; when the k value is larger, that is, the smaller the video complexity, as the packet loss rate x increases, the rate at which the calculation result of the first parameter, i.e., f(p, r, x), decreases is faster, and the corresponding rate at which the second bit rate decreases is faster.

[0051] Step S212: encode the current video image based on the second bit rate.

[0052] The above-mentioned video encoding method obtains the video image in the video to be encoded, and the video scene to which the first frame of the video image belongs. A first bit rate matching the video scene is obtained, and the first frame of the video image is encoded based on the first bit rate. For each frame of the video image except the first frame of the video image, the video complexity of the current video image and the packet loss rate at the current moment are obtained. A first parameter is determined according to the video complexity of the current video image and the packet loss rate at the current moment. The first bit rate is multiplied by the first parameter to obtain a second bit rate matching the current video image. The current video image is encoded based on the second bit rate. This method determines the first bit rate according to the video scene to which the first frame of the video image belongs, and makes a real-time decision and updates the second bit rate matching the current video image according to the video complexity, packet loss rate and first bit rate of the current video image, and the bit rate decision accuracy is high and the update speed is fast.

[0053] The embodiment of the present invention further provides another video encoding method, which is implemented on the basis of the above-mentioned embodiment method; the method focuses on the specific process of obtaining the video image in the video to be encoded and the video scene to which the first frame of the video image belongs, which specifically corresponds to the following steps S402 to S404; Figure 4 As shown, the method comprises the following steps:

[0054] Step S402: obtaining a video image in the video to be encoded.

[0055] Step S404: input the first frame of the video image into a pre-trained neural network model to determine the video scene to which the first frame of the video image belongs through the neural network model.

[0056] The above-mentioned neural network model can be implemented by a variety of convolutional neural networks, such as residual networks, VGG networks, etc. The neural network model can be a convolutional neural network model of any size, for example, it can be resnet34_05x, etc.; the neural network model can also be called a video scene classification model; after obtaining the first frame of video image, it is usually necessary to confirm the video scene to which the first frame of video image belongs. Specifically, the first frame of video image can be input into a pre-trained neural network model to output the video scene to which the first frame of video image belongs through the neural network model. For example, the video scene to which the first frame of video image belongs is a show, game, education and teaching scene, etc.; of course, other methods can also be used to determine the video scene to which the first frame of video image belongs.

[0057] Step S406: Obtain a first bit rate that matches the video scene, and encode the first frame of video image based on the first bit rate.

[0058] Specifically, the first bit rate matching the video scene is determined in advance through the following steps A to E:

[0059] Step A, obtaining multiple frames of video image samples belonging to a video scene.

[0060] Among the multiple frames of video image samples obtained, the video scene to which each frame of video image sample belongs is the same as the video scene to which the above-mentioned first frame of video image belongs; and, the more the number of video image samples, the more accurate the first bit rate determined to match the video scene; when it is necessary to determine the first bit rate matching the video scene, it is usually necessary to first obtain multiple frames of video image samples belonging to the video scene.

[0061] Step B: within a preset bit rate range, determining at least one preset bit rate within the preset bit rate range at preset intervals.

[0062] The above preset bit rate range can be set according to actual needs. For example, for video images, the image quality of the encoded video images obtained by encoding at a bit rate of more than 2000kbps is almost the same, while the image quality of the encoded video images obtained by encoding at a bit rate lower than 100kbps is usually poor. Therefore, the upper and lower limits of the bit rate of the video image can be set to 2000kbps and 100kbps respectively, that is, the preset bit rate range can be set to 100kbps-2000kbps; the above preset interval can also be set according to actual needs. For example, the preset interval can be set to 10kbps, 15kbps, etc.; in actual implementation, the above-mentioned preset bit rate range is usually obtained in advance, and then at least one preset bit rate is selected from the preset bit rate range according to the preset interval; for example, if the preset bit rate range is 100kbps-2000kbps and the preset interval is 10kbps, then a bit rate can be selected within the range of 100kbps-2000kbps at an interval of 10kbps, that is, the selected bit rate can be 100kbps, 110kbps, 120kbps, etc.

[0063] Step C: encode each frame of video image samples according to at least one preset bit rate, and determine at least one image quality of each frame of video image samples after encoding.

[0064] After determining at least one preset bit rate within the preset bit rate range, each frame of video image sample can be encoded using the determined at least one preset bit rate, and then at least one image quality of each frame of video image sample after encoding is calculated; for example, for any frame of video image sample, if 10 different bit rates are used to encode the frame of video image sample, 10 image qualities corresponding to the frame of video image sample can be obtained, and these 10 image qualities may be different or partially the same.

[0065] Step D: performing curve fitting processing on at least one image quality and a preset bit rate corresponding to each image quality to determine a fitting curve matching the video scene.

[0066] In actual implementation, for each frame of video image sample after encoding, multiple pairs of mapping data of preset bit rate and image quality can be obtained. By performing curve fitting processing on these mapping data, the relationship Q = f(B) can be obtained, where Q represents image quality, B is bit rate, and f is the function mapping from B to Q. This relationship can also be called a quality-bit rate model. Based on the above relationship, the fitting curve of the video scene matching can be obtained.

[0067] Different video scenes can be matched to the fitting curves matching the video scenes. For each video scene, multiple frames of video image samples under the video scene can be collected. The more the number of video image samples is, the higher the model accuracy of the fitting curve is.

[0068] Step E: determining a first bit rate matching the video scene based on the fitting curve.

[0069] If the image quality is represented by a quality score, step E can be specifically implemented by the following steps 1 and 2:

[0070] Step 1: Select a target position point on the fitting curve; wherein, taking the bit rate corresponding to the target position point as the starting point, as the bit rate gradually increases, the absolute value of the difference between the quality score corresponding to the increased bit rate and the quality score corresponding to the target position point is within a preset range.

[0071] Depending on the video scene, the target position point may be located at any position of the fitting curve that matches the video scene; the absolute value of the difference may be used to indicate the gap between the quality score corresponding to the increased bit rate and the quality score corresponding to the target position point; the preset range is usually a smaller range; after determining the fitting curve that matches the video scene to which the first frame of video image belongs, the target position point may be selected on the fitting curve, and starting from the target position point, when the bit rate increases, the quality score corresponding to the increased bit rate changes little or remains unchanged, and from the starting position of the fitting curve to the target position point, as the bit rate gradually increases, the quality score corresponding to the increased bit rate usually changes greatly, that is, the target position point is usually an inflection point on the fitting curve.

[0072] Step 2: determine the bit rate corresponding to the target position point as the first bit rate matching the video scene.

[0073] Since the quality score does not change much as the bit rate increases from the target position point as the starting point, the bit rate corresponding to the target position point can be determined as the target bit rate matching the video scene. This method first classifies the scene of the first frame of the video image, and then obtains the target bit rate value of the scene, that is, the first bit rate, based on the scene category information and the quality-bit rate model under the scene.

[0074] Step S408: for each video frame except the first video frame, obtain the video complexity of the current video image and the packet loss rate at the current moment.

[0075] Step S410: Determine a second bit rate matching the current video image based on the video complexity of the current video image, the packet loss rate at the current moment, and the first bit rate, and encode the current video image based on the second bit rate.

[0076] The above-mentioned video encoding method obtains a video image in a video to be encoded. The first frame of the video image in the video image is input into a pre-trained neural network model to determine the video scene to which the first frame of the video image belongs through the neural network model. A first bit rate matching the video scene is obtained, and the first frame of the video image is encoded based on the first bit rate; for each frame of the video image except the first frame of the video image, the video complexity of the current video image and the packet loss rate at the current moment are obtained. Based on the video complexity of the current video image, the packet loss rate at the current moment and the first bit rate, a second bit rate matching the current video image is determined, and the current video image is encoded based on the second bit rate. This method determines the first bit rate matching the video scene to which the first frame of the video image belongs based on the fitting curve, and the determined bit rate has a high accuracy; then, according to the speed of the motion change of the video frame, the second bit rate matching each frame of the video image except the first frame of the video image is determined in real time in combination with the packet loss rate, thereby improving the accuracy of the video bit rate control and the timeliness of the bit rate update speed.

[0077] The embodiment of the present invention further provides another video encoding method, which is implemented on the basis of the above embodiment method; the method focuses on describing the specific process of obtaining the video complexity of the current video image for each frame of video image except the first frame of video image, which specifically corresponds to the following steps S506 to S512; Figure 5 As shown, the method comprises the following steps:

[0078] Step S502: Obtain video images in the video to be encoded and the video scene to which the first frame of the video image belongs.

[0079] Step S504: Obtain a first bit rate that matches the video scene, and encode the first frame of video image based on the first bit rate.

[0080] Step S506, for each video frame except the first video frame, calculate the absolute value of the difference between the pixel value at each pixel position in the current video image and the pixel value at the same pixel position in the previous video frame.

[0081] Since the number of pixels in each frame of the video to be encoded is the same, for each frame of the video except the first frame, the difference between the pixel value at each pixel position in the current video image and the pixel value at the same pixel position in the previous frame of the video image is calculated and output, and the absolute value is taken to obtain the absolute value of the difference corresponding to each pixel position of the current video image.

[0082] Step S508: determining the number of first pixels whose absolute difference values ​​are greater than a preset threshold based on the plurality of absolute difference values.

[0083] The above-mentioned preset threshold can be pre-set according to actual needs; after obtaining the absolute value of the difference corresponding to each pixel position of the current video image, each absolute value of the difference can be compared with the preset threshold to determine the first number of pixels whose absolute value of the difference is greater than the preset threshold.

[0084] Step S510: Calculate the ratio between the first pixel number and the total pixel number of the current video image, and determine the ratio as the video complexity of the current video image.

[0085] For the current video image, the proportion of the first pixel number in the total pixel number is calculated, which is the video complexity of the current video image. The larger the value of the video complexity, the smaller the similarity between the previous and next frames, the faster the current video image changes, and the more complex the video image. Conversely, the smaller the value of the video complexity, the higher the similarity between the previous and next frames, and the slower the current video image changes.

[0086] Step S512, obtaining the current packet loss rate.

[0087] Step S514: determine a second bit rate matching the current video image based on the video complexity of the current video image, the packet loss rate at the current moment, and the first bit rate, and encode the current video image based on the second bit rate.

[0088] The video encoding method obtains a video image in a video to be encoded and a video scene to which a first frame of the video image belongs. A first bit rate matching the video scene is determined, and the first frame of the video image is encoded based on the first bit rate. For each frame of the video image except the first frame of the video image, the absolute value of the difference between the pixel value at each pixel position in the current video image and the pixel value at the same pixel position in the previous frame of the video image is calculated. Based on multiple absolute values ​​of the difference, the number of first pixels whose absolute value of the difference is greater than a preset threshold is determined. The ratio between the first number of pixels and the total number of pixels in the current video image is calculated, and the ratio is determined as the video complexity of the current video image. The packet loss rate at the current moment is obtained. Based on the video complexity of the current video image, the packet loss rate at the current moment and the first bit rate, a second bit rate matching the current video image is determined, and the current video image is encoded based on the second bit rate. This method determines the video complexity according to the difference between the previous and next frames of the video image, and then determines the speed of change of the video image, which can improve the accuracy of determining the video complexity of the video image.

[0089] The embodiment of the present invention also provides a video encoding device, such as Figure 6As shown, the device includes: a first acquisition module 60, used to acquire a video image in a video to be encoded, and a video scene to which a first frame of video image belongs; a first encoding module 61, used to acquire a first bit rate matching the video scene, and encode the first frame of video image based on the first bit rate; a second acquisition module 62, used to acquire, for each frame of video image except the first frame of video image, the video complexity of the current video image, and the packet loss rate at a current moment; a second encoding module 63, used to determine a second bit rate matching the current video image based on the video complexity of the current video image, the packet loss rate at a current moment and the first bit rate, and encode the current video image based on the second bit rate.

[0090] The above-mentioned video encoding device first obtains the video image in the video to be encoded, and the video scene to which the first frame of the video image belongs; obtains the first bit rate matching the video scene, and encodes the first frame of the video image based on the first bit rate; then, for each frame of the video image except the first frame of the video image, obtains the video complexity of the current video image and the packet loss rate at the current moment; finally, based on the video complexity of the current video image, the packet loss rate at the current moment and the first bit rate, determines the second bit rate matching the current video image, and encodes the current video image based on the second bit rate. After obtaining the first bit rate corresponding to the first frame of the video image, the device can make a real-time decision and update the second bit rate corresponding to the current video image based on the first bit rate, the video complexity of the current video image, and the packet loss rate at the current moment for each frame of the video image except the first frame of the video image, and encodes each frame of the video image using the corresponding bit rate, thereby improving the bit rate utilization and encoding efficiency.

[0091] Furthermore, the second encoding module is also used to: determine the first parameter according to the video complexity of the current video image and the packet loss rate at the current moment, wherein the first parameter decreases with the increase of the packet loss rate at the current moment; the higher the video complexity, the slower the rate at which the first parameter decreases; and multiply the first bit rate by the first parameter to obtain a second bit rate that matches the current video image.

[0092] Furthermore, the first parameter is determined by the following formula:

[0093] Second code rate = first code rate * f(p, r, x);

[0094] Wherein, p is a preset parameter coefficient; r represents the video complexity of the current video image; and x represents the packet loss rate at the current moment.

[0095] Furthermore, the first acquisition module is also used to: acquire a video image in the video to be encoded; input a first frame of the video image into a pre-trained neural network model to determine the video scene to which the first frame of the video image belongs through the neural network model.

[0096] Furthermore, the device also includes: a first bit rate determination module, which is used to: obtain multiple frames of video image samples belonging to a video scene; determine at least one preset bit rate within the preset bit rate range at a preset interval within a preset bit rate range; encode each frame of the video image sample at at least one preset bit rate, and determine at least one image quality of each frame of the encoded video image sample; perform curve fitting processing on at least one image quality and the preset bit rate corresponding to each image quality to determine a fitting curve matching the video scene; and determine a first bit rate matching the video scene based on the fitting curve.

[0097] Furthermore, the image quality is represented by a quality score; the first bit rate determination module is also used to: select a target position point on the fitting curve; wherein, taking the bit rate corresponding to the target position point as the starting point, as the bit rate gradually increases, the absolute value of the difference between the quality score corresponding to the increased bit rate and the quality score corresponding to the target position point is within a preset range; the bit rate corresponding to the target position point is determined as the first bit rate matching the video scene.

[0098] Furthermore, the second acquisition module is also used to: for each frame of video image except the first frame of video image, calculate the absolute value of the difference between the pixel value at each pixel position in the current video image and the pixel value at the same pixel position in the previous frame of video image; based on multiple absolute values ​​of the difference, determine the first number of pixels whose absolute value of the difference is greater than a preset threshold; calculate the ratio between the first number of pixels and the total number of pixels of the current video image, and determine the ratio as the video complexity of the current video image.

[0099] The video encoding device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned video encoding method embodiment. For the sake of brief description, for matters not mentioned in the video encoding device embodiment, reference may be made to the corresponding contents in the aforementioned video encoding method embodiment.

[0100] The embodiment of the present invention further provides an electronic device, see Figure 7 As shown, the electronic device includes a processor 130 and a memory 131 , wherein the memory 131 stores machine executable instructions that can be executed by the processor 130 , and the processor 130 executes the machine executable instructions to implement the above-mentioned XX method.

[0101] Further, Figure 7 The electronic device shown further includes a bus 132 and a communication interface 133 , and the processor 130 , the communication interface 133 and the memory 131 are connected via the bus 132 .

[0102] The memory 131 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 133 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 132 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0103] The processor 130 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 130. The above processor 130 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 131, and the processor 130 reads the information in the memory 131 and completes the steps of the method of the above embodiment in combination with its hardware.

[0104] An embodiment of the present invention also provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the above-mentioned video encoding method. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0105] The computer program products of the video encoding method, apparatus, and electronic device provided in the embodiments of the present invention include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0106] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video encoding method, characterized in that: The method comprises: Obtaining a video image in the video to be encoded and a video scene to which the first frame of the video image belongs; Acquire a first bit rate matching the video scene, and encode the first frame of video image based on the first bit rate; For each video frame except the first video frame, obtaining the video complexity of the current video image and the packet loss rate at the current moment; Determine, based on the video complexity of the current video image, the packet loss rate at the current moment, and the first bit rate, a second bit rate matching the current video image, and encode the current video image based on the second bit rate; The step of determining a second bit rate matching the current video image based on the video complexity of the current video image, the packet loss rate at the current moment, and the first bit rate comprises: Determining a first parameter according to the video complexity of the current video image and the packet loss rate at the current moment, wherein the first parameter decreases as the packet loss rate at the current moment increases; the higher the video complexity, the slower the rate at which the first parameter decreases; The first bit rate is multiplied by the first parameter to obtain a second bit rate matching the current video image.

2. The method according to claim 1, characterized in that The first parameter is determined by the following formula: Second code rate = first code rate * f(p, r, x); Among them, p is a preset parameter coefficient; r represents the video complexity of the current video image; and x represents the packet loss rate at the current moment.

3. The method according to claim 1, characterized in that The step of obtaining the video image in the video to be encoded and the video scene to which the first frame of the video image belongs comprises: Acquire a video image in the video to be encoded; A first frame of video images in the video images is input into a pre-trained neural network model to determine the video scene to which the first frame of video images belongs through the neural network model.

4. The method according to claim 1, characterized in that: The first bit rate matching the video scene is determined in advance in the following manner: Acquire multiple frames of video image samples belonging to the video scene; Within a preset bit rate range, determining at least one preset bit rate within the preset bit rate range at preset intervals; Encode each frame of the video image sample according to the at least one preset bit rate, and determine at least one image quality of each frame of the video image sample after encoding; Performing curve fitting processing on the at least one image quality and the preset bit rate corresponding to each image quality to determine a fitting curve matching the video scene; Based on the fitting curve, a first bit rate matching the video scene is determined.

5. The method according to claim 4, characterized in that The image quality is represented by a quality score; and the step of determining a first bit rate matching the video scene based on the fitting curve comprises: A target position point is selected on the fitting curve; wherein, taking the bit rate corresponding to the target position point as a starting point, as the bit rate gradually increases, the absolute value of the difference between the quality score corresponding to the increased bit rate and the quality score corresponding to the target position point is within a preset range; The bit rate corresponding to the target position point is determined as a first bit rate matching the video scene.

6. The method according to claim 1, characterized in that The step of obtaining the video complexity of the current video image for each frame of video image except the first frame of video image comprises: For each frame of video image except the first frame of video image, calculating the absolute value of the difference between the pixel value at each pixel position in the current video image and the pixel value at the same pixel position in the previous frame of video image; Based on the plurality of absolute values ​​of the difference, determining a first number of pixels whose absolute values ​​of the difference are greater than a preset threshold; The ratio between the first number of pixels and the total number of pixels of the current video image is calculated, and the ratio is determined as the video complexity of the current video image.

7. A video encoding device, characterized in that: The device comprises: A first acquisition module, used to acquire a video image in a video to be encoded, and a video scene to which a first frame of the video image belongs; A first encoding module, configured to obtain a first bit rate matching the video scene, and encode the first frame of video image based on the first bit rate; A second acquisition module is used to acquire the video complexity of the current video image and the packet loss rate at the current moment for each video image frame except the first video image frame; a second encoding module, configured to determine a second bit rate matching the current video image based on the video complexity of the current video image, the packet loss rate at the current moment, and the first bit rate, and encode the current video image based on the second bit rate; The second encoding module is also used for: Determining a first parameter according to the video complexity of the current video image and the packet loss rate at the current moment, wherein the first parameter decreases as the packet loss rate at the current moment increases; the higher the video complexity, the slower the rate at which the first parameter decreases; The first bit rate is multiplied by the first parameter to obtain a second bit rate matching the current video image.

8. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the video encoding method according to any one of claims 1 to 6.

9. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the video encoding method according to any one of claims 1 to 6.

Images