Code rate adjustment method and device, electronic equipment, storage medium and product

Abstract

The embodiment of the invention provides a code rate adjustment method and device, electronic equipment, a storage medium and a product, and relates to the technical field of video processing. The method comprises the following steps: receiving video data sent by a sending end, wherein the video data comprises a plurality of video frames; for each video frame in the video data, determining a first sending code rate of the video frame; if the first sending code rate of the video frame is greater than a preset code rate, acquiring a starting moment for forwarding the video frame and a second sending code rate for forwarding; determining a forwarding completion estimation moment of the video frame according to the minimum time difference of the video data and the timestamp of the video frame; determining a third sending code rate of the video frame according to the starting moment, the second sending code rate, the forwarding completion estimation moment and the data volume of the video frame; and continuing to send the video frame to the receiving end by adopting the third sending code rate. According to the embodiment of the invention, the video playing quality of the receiving end can be improved.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a bit rate adjustment method, device, electronic device, storage medium and product. Background Art

[0002] When forwarding uploaded video data, the video data bit rate varies greatly, causing the bandwidth occupied by the forwarding process to fluctuate. Therefore, when forwarding multiple channels of video data in parallel, due to the cumulative effect, the overall bandwidth may exceed the system limit, causing network congestion, packet loss, etc. Usually, the prior art uses two methods to control the overall bandwidth fluctuation. The first method is to limit the bit rate of a single-channel video transmission, and the second method is to reduce the quality of key frames. However, the first method will cause video delay, resulting in poor video playback quality. Although the second method can make the bit rate relatively smooth, the video quality will be significantly reduced when playing key frames, and there is also the problem of poor video playback quality. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a bit rate adjustment method, device, electronic device, storage medium and product, aiming to improve the playback quality of the video at the receiving end.

[0004] To achieve the above object, a first aspect of an embodiment of the present application provides a bit rate adjustment method, the method comprising:

[0005] Receiving video data sent by a transmitting end, wherein the video data includes a plurality of video frames;

[0006] For each of the video frames in the video data, determining a first sending bit rate of the video frame, where the first sending bit rate is determined according to a frame rate of the video data and a data amount of the video frame;

[0007] If the first sending bit rate of the video frame is greater than the preset bit rate, obtaining the starting time of forwarding the video frame and the second sending bit rate of forwarding;

[0008] Determining an estimated time when forwarding of the video frame is completed according to a minimum time difference of the video data and a timestamp of the video frame;

[0009] Determining a third sending bit rate for the video frame according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame;

[0010] The video frame continues to be sent to the receiving end using the third sending bit rate.

[0011] In some embodiments, determining a third sending bit rate of the video frame according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame includes:

[0012] Determine a magnification factor corresponding to the video frame according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame;

[0013] A third sending bit rate of the video frame is determined according to the amplification factor, the second sending bit rate, the starting time, and the estimated forwarding completion time.

[0014] In some embodiments, determining the amplification factor corresponding to the video frame according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame includes:

[0015] According to a preset bit rate growth function, construct an integral expression corresponding to the bit rate growth function, wherein the unknown variable of the integral expression includes a magnification coefficient corresponding to the video frame;

[0016] Based on the starting time, the second sending bit rate, the estimated forwarding completion time and the data amount of the video frame, calculation is performed according to the integral expression to obtain the amplification factor corresponding to the video frame.

[0017] In some embodiments, determining a third sending bit rate of the video frame according to the amplification factor, the second sending bit rate, the start time, and the forwarding completion estimated time includes:

[0018] Subtracting the estimated forwarding completion time from the start time to obtain the forwarding duration of the video frame;

[0019] Determining a bit rate increase value of the video frame according to the forwarding duration and the amplification factor;

[0020] The second sending bit rate is added to the bit rate increase value to obtain a third sending bit rate of the video frame.

[0021] In some embodiments, after receiving the video data sent by the transmitting end, before determining the estimated time of completion of forwarding of the video frame according to the minimum time difference of the video data and the timestamp of the video frame, the method further includes:

[0022] Obtaining a timestamp of each of the video frames and a receiving time of each of the video frames;

[0023] Determine a first time difference corresponding to each of the video frames according to a receiving time of each of the video frames and a timestamp of each of the video frames, and obtain a plurality of first time differences corresponding to a plurality of the video frames;

[0024] Taking the minimum value among the plurality of first time differences as the minimum time difference;

[0025] Determining an estimated time of completion of forwarding of the video frame according to the minimum time difference of the video data and the timestamp of the video frame includes:

[0026] The minimum time difference, the timestamp, and the preset delay duration of the video frame are added together to obtain an estimated time when forwarding of the video frame is completed.

[0027] In some embodiments, determining, for each of the video frames in the video data, a first transmission bit rate of the video frame includes:

[0028] For each of the video frames in the video data, determining a single frame sending duration of the video data according to a frame rate of the video data;

[0029] The ratio of the data amount of the video frame to the single frame sending duration is determined as the first sending bit rate.

[0030] To achieve the above object, a second aspect of an embodiment of the present application provides a bit rate adjustment device, the device comprising:

[0031] A receiving module, used for receiving video data sent by a sending end, wherein the video data includes a plurality of video frames;

[0032] A first determining module, configured to determine, for each of the video frames in the video data, a first sending bit rate of the video frame, wherein the first sending bit rate is determined according to a frame rate of the video data and a data amount of the video frame;

[0033] An acquisition module, configured to acquire a start time for forwarding the video frame and a second transmission bit rate for forwarding if the first transmission bit rate of the video frame is greater than a preset bit rate;

[0034] A second determining module, configured to determine an estimated time of completion of forwarding of the video frame according to a minimum time difference of the video data and a timestamp of the video frame;

[0035] A third determining module, configured to determine a third sending bit rate of the video frame according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame;

[0036] The sending module is used to continue sending the video frame to the receiving end by using the third sending bit rate.

[0037] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the bit rate adjustment method described in the first aspect when executing the computer program.

[0038] To achieve the above objectives, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the bit rate adjustment method described in the first aspect is implemented.

[0039] To achieve the above-mentioned purpose, a fifth aspect of an embodiment of the present application proposes a computer program product. When instructions in the computer program product are executed by an electronic device, the electronic device executes the above-mentioned bit rate adjustment method.

[0040] The present application proposes a method, device, electronic device, storage medium and product for adjusting the bit rate, which receives the video data sent by the sending end, and determines the first sending bit rate of the video frame for each video frame in the video data. If the first sending bit rate of the video frame is greater than the preset bit rate, the starting time of forwarding the video frame and the second sending bit rate of forwarding are obtained, and then the estimated time of forwarding completion of the video frame is determined according to the minimum time difference of the video data and the timestamp of the video frame, and then the third sending bit rate of the video frame is determined according to the starting time, the second sending bit rate, the estimated time of forwarding completion and the data volume of the video frame, and finally the third sending bit rate is used to continue to send the video frame to the receiving end. Through the above steps, the sending bit rate of each video frame in the video data can be adjusted during the forwarding process of the video data uploaded by the sending end, so as to reduce the possibility of large fluctuations in the actual occupied bandwidth due to large changes in the bit rate during forwarding, thereby improving the playback quality of the video at the receiving end. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of a bit rate adjustment method provided in an embodiment of the present application;

[0042] Figure 2 is a structural diagram of a bit rate adjustment device provided in an embodiment of the present application;

[0043] Figure 3 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0047] First, some nouns involved in this application are analyzed:

[0048] Video: Video is a visual medium formed by the rapid playback of continuous static images (i.e. video frames) at certain time intervals. It can present dynamic image content. Video usually includes images, sounds and other possible elements (such as subtitles, special effects, etc.), and is recorded, edited, stored and played through certain technologies and equipment.

[0049] Video data: refers to a collection of data representing video content in a digital way, usually including images, audio, and metadata related to the video (such as timestamp, resolution, frame rate, encoding method, etc.). These data can be stored, transmitted, edited, and played, and compressed and optimized through a specific video encoding format. Video data not only contains the image information of each video frame, but also includes audio data synchronized with the video frame.

[0050] Bit rate: refers to the amount of data transmitted per unit time, usually used to measure the rate of data transmission such as audio and video files, streaming media, network connections, etc. In the field of audio and video, bit rate is usually expressed in kbps (kilobits per second) or Mbps (megabits per second). In audio and video compression and transmission, a higher bit rate usually means better audio and video quality, because more data can represent finer details. However, a higher bit rate will also take up more storage space and bandwidth.

[0051] Based on this, the embodiments of the present application provide a bit rate adjustment method, device, electronic device, storage medium and product, aiming to improve the playback quality of the video at the receiving end.

[0052] A bit rate adjustment method, device, electronic device, storage medium and product provided in the embodiments of the present application are specifically described through the following embodiments. First, the bit rate adjustment method in the embodiments of the present application is described.

[0053] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0054] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0055] The bit rate adjustment method provided in the embodiment of the present application relates to the field of video processing technology. The bit rate adjustment method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN) and big data and artificial intelligence platforms; the software can be an application that implements the bit rate adjustment method, etc., but is not limited to the above forms.

[0056] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0057] Figure 1 is a flow chart of the bit rate adjustment method provided in the embodiment of the present application. Figure 1 As shown, the bit rate adjustment method provided in the embodiment of the present application can be applied to a forwarding server, and the forwarding server can be a CDN distribution server, a streaming media forwarding server, etc. Figure 1 The method may include but is not limited to steps 101 to 106.

[0058] Step 101: receiving video data sent by a transmitter, wherein the video data includes a plurality of video frames.

[0059] The sending end may be a device, program or system that communicates with the forwarding server, such as a camera or other device that can shoot video. The sending end may send video data to the forwarding server, so that the forwarding server processes the video data and then forwards the processed video data to the receiving end. The video data includes multiple video frames.

[0060] Step 102: for each of the video frames in the video data, determine a first sending bit rate of the video frame, where the first sending bit rate is determined according to a frame rate of the video data and a data amount of the video frame.

[0061] When the forwarding server forwards the video data, the video data needs to be forwarded frame by frame. And, for each video frame in the video data, the same processing method is adopted. Specifically, for each video frame, a first sending bit rate of the video frame can be determined, and the first sending bit rate is the theoretical sending bit rate when the video frame is forwarded. The first sending bit rate can be determined according to the frame rate of the video data and the data volume of the video frame. Among them, the frame rate can be set by the sending end according to the actual situation, and the video data sent by the sending end includes the frame rate. The data volume of the video frame can be determined by the forwarding server in the process of receiving the video data.

[0062] Step 103: If the first sending bit rate of the video frame is greater than the preset bit rate, the starting time for forwarding the video frame and the second sending bit rate for forwarding are obtained.

[0063] After determining the first transmission bit rate of the video frame, the first transmission bit rate can be compared with the preset bit rate. The preset bit rate is set according to the actual situation and is not limited here. When the first transmission bit rate is less than or equal to the preset bit rate, the fluctuation of the actual occupied bandwidth when the forwarding server forwards the video frame is small, and there is no need to adjust the transmission bit rate of the video frame. When the first transmission bit rate is greater than the preset bit rate, in order to reduce the possibility of large fluctuations in the actual occupied bandwidth due to large changes in the transmission bit rate during forwarding, the transmission bit rate can be adjusted to smooth the actual occupied bandwidth during forwarding.

[0064] Specifically, the starting time of forwarding the video frame can be obtained. When the forwarding server receives the video frame, it puts the video frame into the forwarding queue. Therefore, the reception time of the video frame received by the forwarding server can be regarded as the starting time of forwarding the video frame. In addition, the second forwarding transmission bit rate can also be obtained. The second transmission bit rate is the actual transmission bit rate when the forwarding server forwards the video frame. Exemplarily, the second transmission bit rate can be the transmission bit rate at the starting time. Alternatively, the second transmission bit rate can also be the transmission bit rate at a preset time after the starting time.

[0065] Step 104: Determine an estimated time when forwarding of the video frame is completed according to the minimum time difference of the video data and the timestamp of the video frame.

[0066] In the process of forwarding each video frame, in order to reduce the possibility of transmission delay of the next video frame, the estimated time of forwarding completion of the video frame can be estimated based on the minimum time difference of the video data and the timestamp of the video frame. Specifically, before determining the estimated time of forwarding completion of the video frame, the minimum time difference of the video data can be determined, that is, the minimum time error between the actual reception time of each video frame received by the forwarding server and the theoretical reception time of each video frame. After determining the minimum time difference, the time of forwarding completion of the video frame can be estimated according to the minimum time difference and the timestamp of the video frame (i.e., the mark used to identify the playback time of the video frame) to obtain the estimated time of forwarding completion of the video frame. Exemplarily, the timestamp, the single frame transmission time and the minimum time difference can be added to obtain the estimated time of forwarding completion. Among them, the single frame transmission time refers to the time taken for each video frame to be sent from the beginning to the completion of sending under theoretical circumstances, which can be determined according to the frame rate. Alternatively, the timestamp, the preset delay time and the minimum time difference can be added to obtain the estimated time of forwarding completion.

[0067] Step 105: Determine a third sending bit rate of the video frame according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame.

[0068] In the time period from the start time to the estimated time of forwarding completion, the forwarding of all data of the video frame should be completed. Therefore, according to the start time, the second transmission bit rate, the estimated time of forwarding completion and the data volume of the video frame, the third transmission bit rate of the video frame can be determined. Among them, the third transmission bit rate refers to the transmission bit rate of the target when forwarding the video frame. Exemplarily, a bit rate growth function can be pre-constructed, and the bit rate growth function is related to the start time, the estimated time of forwarding completion, the second transmission bit rate and the magnification factor corresponding to the video frame. Then, the second transmission bit rate can be substituted into the bit rate growth function, and the bit rate growth function is integrated with the start time as the lower limit of the integration and the estimated time of forwarding completion as the upper limit of the integration, and the integration result is equal to the data volume of the video frame. In this way, the magnification factor corresponding to the video frame can be calculated according to the integral expression corresponding to the bit rate growth function. Subsequently, the forwarding completion estimated time can be subtracted from the start time to determine the forwarding duration of the video frame. Substituting the magnification factor, the forwarding duration and the second transmission bit rate into the bit rate growth function, the third transmission bit rate can be obtained. It should be noted that the method for determining the third sending bit rate is not limited to the aforementioned example, and can be adjusted according to actual conditions as long as the requirement for smoothing the actual occupied bandwidth when forwarding video frames is met.

[0069] Step 106: Continue to send the video frame to the receiving end using the third sending bit rate.

[0070] The receiving end may be a mobile device (such as a mobile phone, a tablet computer, etc.), a vehicle-mounted device, a monitoring system, etc. After determining the third transmission bit rate, the video frame may be forwarded to the receiving end according to the third transmission bit rate, and the video frame may be played at the receiving end.

[0071] It should be noted that when the data volume of the video frame is large, the forwarding time of the video frame is long. In order to improve the adjustment accuracy of the sending bit rate, the time period from the start time to the estimated time of forwarding completion can be divided into multiple time slices, and the sending bit rate of the video frame can be adjusted once in each time slice.

[0072] Steps 101 to 106 shown in the embodiment of the present application are to receive the video data sent by the sender, and for each video frame in the video data, determine the first sending bit rate of the video frame. If the first sending bit rate of the video frame is greater than the preset bit rate, obtain the starting time of forwarding the video frame and the second sending bit rate of forwarding, then determine the estimated time of forwarding completion of the video frame based on the minimum time difference of the video data and the timestamp of the video frame, and then determine the third sending bit rate of the video frame based on the starting time, the second sending bit rate, the estimated time of forwarding completion and the data volume of the video frame, and finally use the third sending bit rate to continue sending the video frame to the receiving end. Through the aforementioned steps, the sending bit rate of each video frame in the video data can be adjusted during the forwarding process of the video data uploaded by the sender, reducing the possibility of large fluctuations in the actual occupied bandwidth due to large changes in the bit rate during forwarding, thereby improving the playback quality of the video at the receiving end.

[0073] In some embodiments, after receiving the video data sent by the transmitting end, before determining the estimated time of completion of forwarding of the video frame according to the minimum time difference of the video data and the timestamp of the video frame, the method further includes:

[0074] Obtaining a timestamp of each of the video frames and a receiving time of each of the video frames;

[0075] Determine a first time difference corresponding to each of the video frames according to a receiving time of each of the video frames and a timestamp of each of the video frames, and obtain a plurality of first time differences corresponding to a plurality of the video frames;

[0076] Taking the minimum value among the plurality of first time differences as the minimum time difference;

[0077] Determining an estimated time of completion of forwarding of the video frame according to the minimum time difference of the video data and the timestamp of the video frame includes:

[0078] The minimum time difference, the timestamp, and the preset delay duration of the video frame are added together to obtain an estimated time when forwarding of the video frame is completed.

[0079] In order to avoid causing a transmission delay of the next video frame, the minimum time difference of the video data can be determined, and then the estimated time of completion of forwarding the video frame can be estimated based on the minimum time difference. Specifically, after starting to receive the video data sent by the sender, and before determining the estimated time of completion of forwarding the video frame based on the minimum time difference and the timestamp of the video frame, the timestamp of each video frame and the reception time when each video frame is actually received by the forwarding server can be obtained.

[0080] Subsequently, according to the receiving time of each video frame and the timestamp of each video frame, the first time difference corresponding to each video frame can be determined. Specifically, in order to determine the first time difference between the receiving time of each video frame and the timestamp of each video frame, the receiving time of each video frame or the timestamp of each video frame can be converted. Exemplarily, the timestamp of each video frame can be added to the starting time of receiving the video data to obtain the theoretical receiving time of each video frame, and then the receiving time of each video frame and the theoretical receiving time of each video frame are subtracted to obtain the first time difference corresponding to each video frame. Alternatively, the receiving time of each video frame can be subtracted from the starting time of receiving the video data to determine the second time difference of the receiving time of each video frame relative to the starting time of receiving the video data, and then the timestamp of each video frame is subtracted from the second time difference to obtain the first time difference corresponding to each video frame. For each video frame, the calculation is performed according to the above steps to obtain multiple first time differences corresponding to multiple video frames, and then the minimum value of the multiple first time differences is used as the minimum time difference of the video data.

[0081] It should be noted that when a video frame is split into n data packets for transmission, since the forwarding server receives each data packet at a different time, at this time, the aforementioned method can be used to determine the third time difference corresponding to each data packet according to the timestamp contained in each data packet and the receiving time of each data packet for each data packet in the n data packets, thereby obtaining n third time differences corresponding to the n data packets. Wherein, n is an integer greater than 1, and the timestamps contained in the n data packets are the same, which are all timestamps of the video frame. After completing the aforementioned calculation process for multiple video frames, multiple third time differences corresponding to multiple video frames can be obtained. Furthermore, the multiple third time differences corresponding to the multiple video frames can be compared, and the minimum value among the multiple third time differences corresponding to the multiple video frames can be determined as the minimum time difference.

[0082] After determining the minimum time difference, the minimum time difference, the timestamp, and the preset delay duration of the video frame can be added together to obtain the estimated time when the forwarding of the video frame is completed. The preset delay duration can be determined based on the jitter delay that can be processed by the receiving end, or can also be set according to actual conditions, which is not limited here. In this way, the estimated time when the forwarding of the video frame is completed can be calculated, which is convenient for determining the target transmission bit rate of the video frame based on the estimated time when the forwarding is completed, that is, the third transmission bit rate.

[0083] In some embodiments, determining, for each of the video frames in the video data, a first transmission bit rate of the video frame includes:

[0084] For each of the video frames in the video data, determining a single frame sending duration of the video data according to a frame rate of the video data;

[0085] The ratio of the data amount of the video frame to the single frame sending duration is determined as the first sending bit rate.

[0086] The first transmission bit rate is the theoretical transmission bit rate when forwarding the video frame. According to the first transmission bit rate of the video frame, it can be determined whether the transmission bit rate of the video frame needs to be adjusted. Specifically, the frame rate of the video data can be obtained, and the single frame transmission time of the video data can be determined according to the frame rate, that is, the single frame transmission time t 1 =1000 / F, where t 1 is the single frame transmission time in milliseconds, and F is the frame rate. Then, the ratio between the data volume of the video frame and the single frame transmission time can be calculated, and the ratio can be determined as the first transmission bit rate. The first transmission bit rate can satisfy R 1 =D f / t 1 , where R 1 is the first transmission code rate, D f is the data volume of the video frame, t 1 In this way, the first transmission bit rate can be calculated, so that it can be judged whether the transmission bit rate of the video frame needs to be adjusted based on the first transmission bit rate.

[0087] In some embodiments, determining a third sending bit rate of the video frame according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame includes:

[0088] Determine a magnification factor corresponding to the video frame according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame;

[0089] A third sending bit rate of the video frame is determined according to the amplification factor, the second sending bit rate, the starting time, and the estimated forwarding completion time.

[0090] According to the starting time, the second sending bit rate, the estimated time of forwarding completion and the data volume of the video frame, the third sending bit rate of the video frame can be determined. Specifically, according to the starting time, the estimated time of forwarding completion, the second sending bit rate and the data volume of the video frame, the magnification factor corresponding to the video frame can be determined. Exemplarily, a bit rate growth function can be pre-constructed, and the bit rate growth function is related to the starting time, the estimated time of forwarding completion, the second sending bit rate and the magnification factor corresponding to the video frame. Subsequently, the second sending bit rate is substituted into the bit rate growth function, and the bit rate growth function is integrated with the starting time as the lower limit of the integral and the estimated time of forwarding completion as the upper limit of the integral, and the integral result is equal to the data volume of the video frame. In this way, the magnification factor corresponding to the video frame can be obtained by calculation according to the integral expression corresponding to the bit rate growth function.

[0091] After obtaining the amplification factor, the third transmission bit rate of the video frame can be determined according to the amplification factor, the second transmission bit rate, the starting time, and the estimated forwarding completion time. Exemplarily, the forwarding duration of the video frame can be obtained by subtracting the starting time from the estimated forwarding completion time, and then the amplification factor, the forwarding duration, and the second transmission bit rate are substituted into the pre-constructed bit rate growth function to obtain the third transmission bit rate. According to the third transmission bit rate, the forwarding server can continue to forward the video frame to the receiving end, so that the actual occupied bandwidth of the forwarding process fluctuates less, thereby improving the playback quality of the video at the receiving end.

[0092] In some embodiments, determining the amplification factor corresponding to the video frame according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame includes:

[0093] According to a preset bit rate growth function, construct an integral expression corresponding to the bit rate growth function, wherein the unknown variable of the integral expression includes a magnification coefficient corresponding to the video frame;

[0094] Based on the starting time, the second sending bit rate, the estimated forwarding completion time and the data amount of the video frame, calculation is performed according to the integral expression to obtain the amplification factor corresponding to the video frame.

[0095] According to the starting time, the second transmission bit rate, the estimated forwarding completion time and the data volume of the video frame, the amplification factor corresponding to the video frame can be determined. Specifically, a bit rate growth function can be pre-constructed, for example, the bit rate growth function can satisfy the following formula (1):

[0096] f(x)=R 2 +k*x 1 / 3 (1)

[0097] Where f(x) is the function value of the code rate growth function, that is, the third transmission code rate, R2 is the second sending code rate, k is the amplification factor (unknown), and x is the forwarding duration determined according to the starting time and the estimated time of forwarding completion.

[0098] Based on the bit rate growth function, we can further construct the integral expression corresponding to the bit rate growth function, that is: Among them, t 2 is the starting time, t 3 It is understood that in order to ensure that the forwarding of the video frame can be completed within the forwarding time, the integral result of the integral expression should be equal to the data volume of the video frame. In this way, the integral expression corresponding to the bit rate growth function can be calculated to obtain the magnification factor corresponding to the video frame, which is convenient for the subsequent calculation of the third transmission bit rate based on the magnification factor corresponding to the video frame.

[0099] In some embodiments, determining a third sending bit rate of the video frame according to the amplification factor, the second sending bit rate, the start time, and the forwarding completion estimated time includes:

[0100] Subtracting the estimated forwarding completion time from the start time to obtain the forwarding duration of the video frame;

[0101] Determining a bit rate increase value of the video frame according to the forwarding duration and the amplification factor;

[0102] The second sending bit rate is added to the bit rate increase value to obtain a third sending bit rate of the video frame.

[0103] After determining the amplification factor, the third transmission bit rate of the video frame can be determined according to the amplification factor, the second transmission bit rate, the starting time, and the estimated forwarding completion time. Specifically, the forwarding time of the video frame can be obtained by subtracting the starting time from the estimated forwarding completion time. Subsequently, based on the bit rate growth function, the bit rate growth value of the video frame can be determined according to the amplification factor and the forwarding time. Exemplarily, taking the bit rate growth function using formula (1) as an example, then, according to the amplification factor and the forwarding time, the bit rate growth value that satisfies R a =k*x 1 / 3 , where k is the amplification factor and x is the forwarding duration. Then, the second sending bit rate is added to the bit rate growth value to obtain the third sending bit rate of the video frame. According to the third sending bit rate, the forwarding server can continue to forward the video frame to the receiving end, so that the fluctuation of the actual occupied bandwidth during the forwarding process is small, so that the playback quality of the video at the receiving end can be improved.

[0104] It should be noted that the preset bit rate increase function is not limited to the aforementioned formula (1), and the aforementioned bit rate increase function can also be adjusted according to user preference or current network bandwidth load.

[0105] The following is an example of the bit rate adjustment method provided in the embodiment of the present application.

[0106] Step 1: before starting to forward or distribute the video, the sender reports information such as the preset bit rate and frame rate of the video data to the forwarding server;

[0107] Step 2: When receiving video data, put the data packet of each video frame in the video data into the forwarding queue, and calculate the first time difference between the receiving time and the timestamp of the video frame, where the first time difference = receiving time - timestamp of the video frame. In the process of receiving video data, multiple calculations and comparisons are required to obtain the minimum time difference of the video data;

[0108] Step 3, in the process of receiving the video data sent by the sender, the ideal sending bit rate of each video frame (i.e., the first sending bit rate) is continuously calculated, where the ideal sending bit rate = the data volume of the video frame / the sending time of a single frame, and the sending time of a single frame (milliseconds) = 1000 / frame rate. When the ideal sending bit rate is greater than the preset bit rate, in order to avoid sudden changes in the sending bit rate, the subsequent actual occupied bandwidth smoothing mechanism needs to be enabled;

[0109] Step 4. Generally speaking, the theoretical time when the video frame is sent is = the timestamp of the video frame + the minimum time difference + the sending duration of a single frame. However, the introduction of the actual occupied bandwidth smoothing mechanism will inevitably increase the delay of the video frame in order to achieve a better bandwidth control effect. The receiving end is usually equipped with a jitter buffer function, which can be used to handle network jitter and the delay introduced by the actual occupied bandwidth smoothing mechanism. Moreover, when the delay introduced by the actual occupied bandwidth smoothing mechanism is less than the jitter delay that the jitter buffer can handle, it will not affect the playback experience of the receiving end. Therefore, when estimating the estimated time when forwarding is completed, the sending duration of a single frame can be increased to the delay introduced by the actual occupied bandwidth smoothing mechanism, that is, increased to the jitter delay that the jitter buffer can handle (that is, the preset delay duration). Therefore, the estimated time when the forwarding of the video frame is completed is: the estimated time when the forwarding is completed = the timestamp of the video frame + the minimum time difference + the jitter delay that the jitter buffer can handle;

[0110] Step 5: To control the bit rate of the video frame, a bit rate growth function can be selected as a bit rate increase strategy. Assume that the second transmission rate is R 2 , the amplification factor (an unknown number) is k, then we can get the rate growth function that changes with the forwarding time x, and the rate growth function satisfies formula (1);

[0111] Step 6, based on the bit rate growth function, the integral expression corresponding to the bit rate growth function can be constructed, and then the area covered by the bit rate growth function f(x) and the coordinate x-axis (i.e., the forwarding duration direction) within the forwarding duration can be obtained. The covered area is equal to the amount of data that needs to be sent (i.e., the amount of data in the video frame). In this way, the amplification factor k can be calculated. Substituting the amplification factor k, the second sending bit rate, and the forwarding duration into the bit rate growth function, the third sending bit rate can be calculated;

[0112] Step 7: Continue forwarding the video frames to the receiving end according to the third sending bit rate.

[0113] Furthermore, due to the time precision limitation of the forwarding server, it is impossible to implement real-time adjustment of the sending bit rate of the video frame. Therefore, in order to improve the adjustment precision of the sending bit rate, the forwarding time can be divided into n time slices, and the sending bit rate that can be increased in a single time slice can be determined with reference to the aforementioned third sending bit rate determination method, and the sending bit rate of the video frame can be adjusted once in each time slice. In this way, by dividing the longer forwarding time into multiple time slices and adjusting the sending bit rate of the video frame in each time slice, the adjustment precision of the sending bit rate of the video frame can be improved.

[0114] The bit rate adjustment method provided by the present application can accurately control the transmission bit rate of the video frame. Moreover, under the premise of not changing the video quality and controllable delay, the transmission bit rate growth function during video forwarding and video distribution is controlled, so as to smoothly forward the actual occupied bandwidth when forwarding video data, and ensure the playback quality of the video at the receiving end.

[0115] Figure 2 This is a schematic diagram of the structure of the bit rate adjustment device provided in the embodiment of the present application. Figure 2 The embodiment of the present application further provides a bit rate adjustment device 200, which can implement the above bit rate adjustment method. The device 200 includes:

[0116] A receiving module 201, configured to receive video data sent by a transmitting end, wherein the video data includes a plurality of video frames;

[0117] A first determining module 202 is used to determine, for each of the video frames in the video data, a first sending bit rate of the video frame, where the first sending bit rate is determined according to a frame rate of the video data and a data amount of the video frame;

[0118] The acquisition module 203 is used to acquire the start time of forwarding the video frame and the second transmission bit rate of forwarding if the first transmission bit rate of the video frame is greater than the preset bit rate;

[0119] A second determination module 204, configured to determine an estimated time of completion of forwarding of the video frame according to a minimum time difference of the video data and a timestamp of the video frame;

[0120] A third determining module 205, configured to determine a third sending bit rate of the video frame according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame;

[0121] The sending module 206 is used to continue sending the video frame to the receiving end by using the third sending bit rate.

[0122] In some embodiments, the third determination module 205 includes:

[0123] A first determination submodule, configured to determine a magnification factor corresponding to the video frame according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data volume of the video frame;

[0124] The second determination submodule is used to determine a third sending bit rate of the video frame according to the amplification factor, the second sending bit rate, the start time and the estimated forwarding completion time.

[0125] In some embodiments, the first determining submodule includes:

[0126] A construction unit, configured to construct an integral expression corresponding to the bit rate growth function according to a preset bit rate growth function, wherein the unknown variables of the integral expression include a magnification coefficient corresponding to the video frame;

[0127] The operation unit is used to perform operation according to the integral expression based on the starting time, the second sending bit rate, the estimated forwarding completion time and the data amount of the video frame to obtain the amplification factor corresponding to the video frame.

[0128] In some embodiments, the second determining submodule includes:

[0129] A first calculation unit, configured to subtract the estimated forwarding completion time from the start time to obtain a forwarding duration of the video frame;

[0130] A determination unit, configured to determine a bit rate increase value of the video frame according to the forwarding duration and the amplification factor;

[0131] The second calculation unit is used to add the second sending bit rate and the bit rate increase value to obtain a third sending bit rate of the video frame.

[0132] In some embodiments, the bit rate adjustment device 200 further includes a fourth determination module, and the fourth determination module includes:

[0133] An acquisition submodule, used for acquiring a timestamp of each video frame and a receiving time of each video frame;

[0134] A third determining submodule, configured to determine a first time difference corresponding to each of the video frames according to a receiving time of each of the video frames and a timestamp of each of the video frames, and obtain a plurality of first time differences corresponding to a plurality of the video frames;

[0135] a fourth determining submodule, configured to use a minimum value among the plurality of first time differences as the minimum time difference;

[0136] The second determining module 204 includes:

[0137] The calculation submodule is used to add the minimum time difference, the timestamp and the preset delay duration of the video frame to obtain an estimated time when the forwarding of the video frame is completed.

[0138] In some embodiments, the first determination module 202 includes:

[0139] A fifth determining submodule, configured to determine, for each of the video frames in the video data, a single frame sending duration of the video data according to a frame rate of the video data;

[0140] The sixth determination submodule is used to determine the ratio of the data amount of the video frame to the single frame sending duration as the first sending bit rate.

[0141] The specific implementation of the bit rate adjustment device 200 is substantially the same as the specific implementation of the bit rate adjustment method described above, and will not be described in detail herein.

[0142] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned bit rate adjustment method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0143] Figure 3 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. Figure 3 . Electronic equipment includes:

[0144] The processor 301 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0145] The memory 302 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 302 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 302, and the processor 301 calls and executes the bit rate adjustment method of the embodiment of this application;

[0146] Input / output interface 303, used to implement information input and output;

[0147] The communication interface 304 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0148] A bus 305 that transmits information between the various components of the device (e.g., the processor 301, the memory 302, the input / output interface 303, and the communication interface 304);

[0149] The processor 301 , the memory 302 , the input / output interface 303 and the communication interface 304 are connected to each other in communication within the device via the bus 305 .

[0150] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned bit rate adjustment method is implemented.

[0151] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0152] The embodiment of the present application also provides a computer program product for implementation. When the instructions in the computer program product are executed by an electronic device, the electronic device implements any one of the bit rate adjustment methods in the above embodiments.

[0153] The present application proposes a method, device, electronic device, storage medium and product for adjusting the bit rate, which receives the video data sent by the sending end, and determines the first sending bit rate of the video frame for each video frame in the video data. If the first sending bit rate of the video frame is greater than the preset bit rate, the starting time of forwarding the video frame and the second sending bit rate of forwarding are obtained, and then the estimated time of forwarding completion of the video frame is determined according to the minimum time difference of the video data and the timestamp of the video frame, and then the third sending bit rate of the video frame is determined according to the starting time, the second sending bit rate, the estimated time of forwarding completion and the data volume of the video frame, and finally the third sending bit rate is used to continue to send the video frame to the receiving end. Through the above steps, the sending bit rate of each video frame in the video data can be adjusted during the forwarding process of the video data uploaded by the sending end, so as to reduce the possibility of large fluctuations in the actual occupied bandwidth due to large changes in the bit rate during forwarding, thereby improving the playback quality of the video at the receiving end.

[0154] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0155] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0156] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0157] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0158] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0159] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0160] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0161] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0162] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0163] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0164] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A bit rate adjustment method, characterized in that: The method comprises: Receiving video data sent by a transmitting end, wherein the video data includes a plurality of video frames; For each of the video frames in the video data, determining a first sending bit rate of the video frame, where the first sending bit rate is determined according to a frame rate of the video data and a data amount of the video frame; If the first sending bit rate of the video frame is greater than the preset bit rate, obtaining the starting time of forwarding the video frame and the second sending bit rate of forwarding; Determining an estimated time when forwarding of the video frame is completed according to a minimum time difference of the video data and a timestamp of the video frame; Determining a third sending bit rate for the video frame according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame; The video frame continues to be sent to the receiving end using the third sending bit rate.

2. The method according to claim 1, characterized in that: The determining, according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame, a third sending bit rate of the video frame includes: Determine a magnification factor corresponding to the video frame according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame; A third sending bit rate of the video frame is determined according to the amplification factor, the second sending bit rate, the starting time, and the estimated forwarding completion time.

3. The method according to claim 2, characterized in that The determining, according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame, a magnification factor corresponding to the video frame includes: According to a preset bit rate growth function, construct an integral expression corresponding to the bit rate growth function, wherein the unknown variable of the integral expression includes a magnification coefficient corresponding to the video frame; Based on the starting time, the second sending bit rate, the estimated forwarding completion time and the data amount of the video frame, calculation is performed according to the integral expression to obtain the amplification factor corresponding to the video frame.

4. The method according to claim 2, characterized in that: The determining, according to the amplification factor, the second sending bit rate, the start time, and the forwarding completion estimated time, of a third sending bit rate of the video frame includes: Subtracting the estimated forwarding completion time from the start time to obtain the forwarding duration of the video frame; Determining a bit rate increase value of the video frame according to the forwarding duration and the amplification factor; The second sending bit rate is added to the bit rate increase value to obtain a third sending bit rate of the video frame.

5. The method according to claim 1, characterized in that After receiving the video data sent by the transmitting end, before determining the estimated time of completion of forwarding of the video frame according to the minimum time difference of the video data and the timestamp of the video frame, the method further includes: Obtaining a timestamp of each of the video frames and a receiving time of each of the video frames; Determine a first time difference corresponding to each of the video frames according to a receiving time of each of the video frames and a timestamp of each of the video frames, and obtain a plurality of first time differences corresponding to a plurality of the video frames; Taking the minimum value among the plurality of first time differences as the minimum time difference; Determining an estimated time of completion of forwarding of the video frame according to the minimum time difference of the video data and the timestamp of the video frame includes: The minimum time difference, the timestamp, and the preset delay duration of the video frame are added together to obtain an estimated time when forwarding of the video frame is completed.

6. The method according to claim 1, characterized in that The step of determining, for each of the video frames in the video data, a first sending bit rate of the video frame comprises: For each of the video frames in the video data, determining a single frame sending duration of the video data according to a frame rate of the video data; The ratio of the data amount of the video frame to the single frame sending duration is determined as the first sending bit rate.

7. A bit rate adjustment device, characterized in that: The device comprises: A receiving module, used for receiving video data sent by a sending end, wherein the video data includes a plurality of video frames; A first determining module, configured to determine, for each of the video frames in the video data, a first sending bit rate of the video frame, wherein the first sending bit rate is determined according to a frame rate of the video data and a data amount of the video frame; An acquisition module, configured to acquire a start time for forwarding the video frame and a second transmission bit rate for forwarding if the first transmission bit rate of the video frame is greater than a preset bit rate; A second determining module, configured to determine an estimated time of completion of forwarding of the video frame according to a minimum time difference of the video data and a timestamp of the video frame; A third determining module, configured to determine a third sending bit rate of the video frame according to the starting time, the second sending bit rate, the estimated forwarding completion time, and the data amount of the video frame; The sending module is used to continue sending the video frame to the receiving end by using the third sending bit rate.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the bit rate adjustment method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the bit rate adjustment method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that When the instructions in the computer program product are executed by an electronic device, the electronic device executes the bit rate adjustment method according to any one of claims 1 to 6.

Images