Method and processing terminal for determining the number of buffers for USB video transmission under V4L2 framework

Abstract

The present invention discloses a method for determining the number of buffers for USB video transmission under a V4L2 framework, comprising the following steps: collecting the CPU usage rate of the process where the USB video transmission is located and the current packet loss rate of the USB video transmission, determining the current network state based on the current packet loss rate, and using different schemes to determine the number of buffers for video transmission according to different network states, wherein the different schemes include at least two schemes, one of which determines the number of buffers only according to the CPU usage rate, and the other determines the number of buffers in combination with the CPU usage rate and the current packet loss rate. The present invention can specifically quantify the number of buffers, without the need to collect data information such as the receiving time difference and the timestamp difference between two adjacent frames in real time as in the traditional technology, thereby enabling the present invention to be applicable to USB video transmission under the V4L2 framework.

Description

Technical Field

[0001] The invention relates to the field of computer technology, and in particular to a method for determining the number of buffer zones for USB video transmission under a V4L2 framework and a processing terminal. Background Art

[0002] In order to ensure the transmission speed based on USB video transmission, the setting of the buffer is the key, which directly affects the quality and transmission efficiency of the video stream. For the buffer setting of video transmission, the existing technology mainly has the following two technical paths:

[0003] 1. Calculate the network status and calculate the buffer size based on the reception time difference and timestamp difference between two adjacent video frames. The buffer size refers to the number of video frames that can be stored. This method is to perceive network changes by counting the arrival time of the data packets of the video stream, so as to dynamically adjust the buffer size according to the network status.

[0004] 2. Calculate the packet loss rate, determine the storage capacity of the video buffer area based on the comparison between the target packet loss rate and the first preset packet loss rate threshold and the second preset packet loss rate threshold, and then dynamically adjust the size of the buffer.

[0005] Among the above prior arts, it is not applicable to USB video transmission under the V4L2 framework. The reason is that, before setting the buffer during video acquisition, the USB video transmission under the V4L2 framework needs to query the performance of the USB video device (such as a camera using a USB interface for transmission), including the video format, video resolution and frame rate output by the USB video device, and then set the acquired video format, which is the same as the video format output by the USB video device, and then set the number of buffers. V4L2 is a kernel driver, which allocates a fixed buffer size, that is, each buffer size is fixed and the same. Under such a video transmission principle framework, it is impossible to calculate the size of the buffer based on the receiving time difference and timestamp difference between two adjacent video frames during video transmission, and dynamically adjust the size of the buffer.

[0006] Simply calculating the packet loss rate of the network status to adjust the buffer size has great limitations. It is usually applicable to situations where the network status is relatively stable, which can be regarded as a situation where the network status is very good. However, for video transmission with multi-threaded video acquisition, simply adjusting the buffer size based on the packet loss rate is not ideal.

[0007] In addition, in many existing technologies, only the situations in which the number of buffers needs to be adjusted, such as increasing or decreasing the number of buffers, are given, but the specific number of buffers that need to be adjusted cannot be quantified, resulting in inaccurate adjustment of the buffers and affecting the video transmission effect. Summary of the invention

[0008] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a method and a processing terminal for determining the number of buffers for USB video transmission under a V4L2 framework, which can solve the problems described in the background technology.

[0009] The technical solution for achieving the purpose of the present invention is: a method for determining the number of buffers for USB video transmission under a V4L2 framework, comprising the following steps:

[0010] Collect the CPU usage CPU_U of the process where the USB video transmission is located and the current packet loss rate η of the USB video transmission,

[0011] Based on the current packet loss rate η, the current network state is determined, and different schemes are adopted according to different network states to determine the number of buffers used for video transmission. The different schemes include at least two schemes, one of which determines the number of buffers only according to the CPU usage CPU_U, and the other determines the number of buffers in combination with the CPU usage CPU_U and the current packet loss rate η.

[0012] Furthermore, the process identifier PID of the USB video transmission program is first obtained, and then the CPU usage rate of the CPU occupied by the current process is queried according to the process identifier PID.

[0013] Furthermore, if the current packet loss rate η≤the pre-trial packet loss rate, the network state is regarded as the first network state; otherwise, the network state is regarded as the second network state. If the network state is in the first network state, the number of buffers is determined only according to the CPU usage CPU_U; if the network state is in the second network state, the number of buffers is determined in combination with the CPU usage CPU_U and the current packet loss rate η.

[0014] Further, in the first network state, the first solution is used to determine the number of buffers, and the first solution is to determine the number of buffers only according to the CPU usage CPU_U. In the second network state, the second solution is used to determine the number of buffers, and the second solution is to determine the number of buffers in combination with the CPU usage CPU_U and the current packet loss rate η.

[0015] The first solution: Use formula ① to determine the theoretical number of buffers β that need to be set:

[0016] CPU_U=α*β+β 0 ------①

[0017] In the formula, α is the slope, is a preset constant, and β 0 is the intercept, is a preset constant,

[0018] The second solution: Use formula ② to determine the theoretical number of buffers β that need to be set:

[0019] CPU_U=b 0 +b 1 η+b 2 β+b 3 ηβ+b 4 η 2 +b 5 β 2 +b 6 ηβ 2 +b 7 η 2 β+b 8 η 2 β 2 ------②

[0020] Where b 0 、b 1 、b 2 、b 3 、b 4 、b 5 、b 6 、b 7 、b 8 represents the coefficient, which is a constant,

[0021] The actual buffer quantity a is calculated according to the following formula:

[0022] a=round(β),β∈[3,M]

[0023] In the formula, round() means rounding to an integer.

[0024] Furthermore, in the first solution, the CPU usage CPU_U under different theoretical buffer numbers β is determined through multiple experiments to fit α and β 0 , according to the fitted α and β 0 The resulting formula ① determines the theoretical number of buffers β under the current CPU usage CPU_U.

[0025] In the second scheme, after measuring multiple groups of different packet loss rates η and buffer numbers β through multiple experiments, multiple groups of data are obtained, and the 8 coefficients of formula ② are solved by mathematical methods. Then, based on formula ② formed by the solved coefficients, the current CPU usage CPU_U and the theoretical buffer number β under the current packet loss rate η are determined.

[0026] A processing terminal, comprising:

[0027] A memory for storing program instructions;

[0028] The processor is used to run the program instructions to execute the steps of the method for determining the number of buffers for USB video transmission under the V4L2 framework.

[0029] Beneficial effects of the present invention: The present invention can specifically quantify the number of buffers, without the need to collect data information such as the reception time difference and timestamp difference between two adjacent frames in real time as in traditional technologies, thereby making the present invention applicable to USB video transmission under the V4L2 framework. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the process of the present invention;

[0031] Figure 2 A schematic diagram of the structure of the processing terminal. DETAILED DESCRIPTION

[0032] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments:

[0033] like Figure 1 As shown, a method for determining the number of buffers for USB video transmission under a V4L2 framework includes the following steps:

[0034] Step 1: Collect the CPU usage CPU_U of the process where the USB video transmission is located and the current packet loss rate η of the USB video transmission.

[0035] V4L2 is an API and driver framework for video devices in the Linux operating system. When video transmission is performed, the Linux operating system will assign a process to the video transmission transaction. The process is used for video transmission. The process needs to use the CPU, so it will occupy the CPU, and thus the process will correspond to a CPU usage rate.

[0036] You can first obtain the process identifier PID of the USB video transmission program (taking stream as an example), and then query the CPU usage of the current process based on the process identifier PID. The relevant query instructions are as follows:

[0037] PID = $(pgrep stream)

[0038] Cpu_U=$(top-bn1-p$pid|grep$PID|awk'{print$9}')

[0039] The Linux operating system has many tools that can obtain CPU usage, such as the top tool. The above instruction uses the top tool to query the CPU usage based on the process identifier PID.

[0040] Similarly, the packet loss rate may also be obtained by using existing tools, for example, by using a ping command or other tools.

[0041] Step 2: Based on the current packet loss rate η, determine the current network state, and adopt different schemes according to different network states to determine the number of buffers used for video transmission. The different schemes include at least two schemes, one of which determines the number of buffers only according to the CPU usage CPU_U, and the other determines the number of buffers in combination with the CPU usage CPU_U and the current packet loss rate η.

[0042] Specifically, if the current packet loss rate η ≤ the pre-trial packet loss rate, the network is considered to be stable and does not affect video transmission, and it is classified as the first network state; otherwise, the current packet loss rate η is considered to be too large, the network is considered to be unstable, affecting video transmission, and it is classified as the second network state.

[0043] In the first network state, the first solution is used to determine the number of buffers; in the second network state, the second solution is used to determine the number of buffers.

[0044] The first solution: Use formula ① to determine the theoretical number of buffers β that need to be set:

[0045] CPU_U=α*β+β 0 ------①

[0046] In the formula, α is the slope, is a preset constant, and β 0 is the intercept, is a constant, and is also the preset value.

[0047] In formula ①, only the CPU usage affects the theoretical buffer quantity β. Therefore, the first solution is to determine the buffer quantity only according to the CPU usage CPU_U.

[0048] In an optional implementation, multiple tests can be performed by assigning different specific values ​​of the theoretical buffer quantity β to obtain the CPU usage corresponding to different values ​​of the theoretical buffer quantity β, thereby obtaining multiple sets of (CPU_U, β) parameters, so that the slope α and intercept β can be fitted. 0 . The fitted slope α and intercept β 0 Substitute into formula ① and use it as the preset formula. In subsequent applications, the current CPU usage CPU_U can be obtained in real time to determine the theoretical number of buffers β that need to be set under the current CPU usage CPU_U.

[0049] Since the β calculated by formula ① may contain decimals, and the theoretical number of buffers must be an integer, it is necessary to obtain the actual number of buffers a that represents the actual number of buffers that need to be set:

[0050] a=round(β)

[0051] In the formula, round() means rounding to an integer, that is, rounding β to an integer. For example, β = 4.62, then a = 5, that is, the actual number of buffers that need to be set is 5.

[0052] Generally speaking, it is necessary to limit β≥3, that is, it is also necessary to limit the maximum number of buffers that can be set, M, because the number of buffers is also limited by storage hardware resources and is not infinite. Therefore, β∈[3,M].

[0053] In the first solution, the number of buffers and the CPU usage are in a linear relationship. As the number of buffers increases, the CPU usage of the process will also increase to a certain extent. Therefore, the relationship between the number of buffers and the CPU usage can be fitted through formula ①.

[0054] The second solution: The first solution is for a stable network state, which will not affect video transmission. However, in actual applications, network fluctuations are often encountered. Under network fluctuations, the number of buffers and CPU usage no longer maintain a linear relationship, so the second solution is used.

[0055] Use formula ② to determine the theoretical number of buffers β that need to be set:

[0056] CPU_U=b 0 +b 1 η+b 2 β+b 3 ηβ+b 4 η 2 +b 5 β 2 +b 6 ηβ 2 +b 7 η 2 β+b 8 η 2 β 2 ------②

[0057] Where b 0 、b 1 、b 2 、b 3 、b 4 、b 5 、b 6 、b 7 、b 8Represents the coefficient, which is a constant and also a preset value.

[0058] Formula ② includes the first and second powers of the packet loss rate η. Similarly, it also includes the first and second powers of the theoretical buffer number β. In addition, it also includes the product of the packet loss rate η and the theoretical buffer number β, the second power of the product, and so on. The packet loss rate and the theoretical buffer number β play the same role, that is, the impact on the CPU usage is the same, which is more in line with the actual situation.

[0059] For example, how to solve the 8 coefficients in formula ② can be performed as follows:

[0060] 1) According to the actual test of the device, the CPU occupancy rate of the video transmission thread when the independent variable packet loss rate η and the theoretical buffer number β change during USB video transmission based on the V4L2 framework. A sufficient number of samples need to be recorded. The more samples, the higher the accuracy of subsequent calculations.

[0061] 2) Construct a matrix. Construct a matrix X and a response vector Y based on the sample data. The matrix X should contain the individual terms of the independent variables η and β and their power and interaction terms. The response vector Y should be the observed value of the CPU usage. The i-th CPU usage CPU_U i The packet loss rate is η i , the theoretical number of buffers is β i .

[0062] Matrix X i =[1,η i ,β i ,η i β i ,η i 2 ,β i 2 ,η i 2 β i ,η i β i 2 ,η i 2 β i 2 ]

[0063] Among them, 1 is the intercept term.

[0064] Response vector Y i =[CPU_U 1 ,CPU_U 2 ,CPU_U 3 ,…,CPU_U i ].

[0065] Then the least square method is used to estimate the above 8 coefficients, and the best estimated value b_hat of the coefficient is determined by minimizing the residual sum of squares, b_hat = (X T X) -1 X T Y, b_hat is a vector that protects all b coefficients, that is, b_hat = [b 0 ,b 1 ,b 2 ,b 3 ,b 4 ,b 5 ,b 6 ,b 7 ,b 8 ] T . Thus, formula ② can be determined.

[0066] The above is only exemplary. These eight coefficients can be derived using existing mathematical methods. There are other mathematical methods that can also calculate these eight coefficients, so they will not be elaborated here.

[0067] The above 8 coefficients can be obtained by measuring multiple groups of different packet loss rates η and buffer quantities β through actual experiments, and then these 8 coefficients can be solved by mathematical methods. Substituting these 8 coefficients into formula ② can obtain an applicable expression. Subsequently, it is only necessary to determine the current packet loss rate η and the current CPU usage CPU_U to determine the number of buffers that need to be set.

[0068] Similarly, the theoretical buffer number λ may be a decimal, and the actual buffer number a that needs to be set can be solved by rounding off, that is,

[0069] a=round(β),β∈[3,M]

[0070] Through the above processing, the number of buffers can be adjusted dynamically. For example, if the CPU usage rate of the video transmission site of the current USB video device is allowed to be 12%, and the maximum number of buffers M is 10, the first solution or the second solution can be adopted according to the packet loss rate that characterizes different network states, and the formula is derived to calculate the theoretical number of buffers β, and finally the actual number of buffers a as an integer is determined by rounding. In this way, the number of buffers can be specifically quantified, and unlike the prior art, it does not simply depend on the network state, but also takes into account the CPU usage rate, and does not need to collect data information such as the receiving time difference and timestamp difference between two adjacent frames in real time as in traditional technology, so that the present invention can be applied to USB video transmission under the V4L2 framework.

[0071] like Figure 2 As shown, the present invention also provides a processing terminal 100, which includes:

[0072] Memory 101, used for storing program instructions;

[0073] The processor 102 is used to run the program instructions to execute the steps of the method for determining the number of buffers in USB video transmission under the V4L2 framework.

[0074] The embodiment disclosed in this specification is only an example of a unilateral feature of the present invention, and the protection scope of the present invention is not limited to this embodiment, and any other functionally equivalent embodiments fall within the protection scope of the present invention. For those skilled in the art, various other corresponding changes and deformations can be made according to the technical solutions and concepts described above, and all of these changes and deformations should fall within the protection scope of the claims of the present invention.

Claims

1. A method for determining the number of buffers for USB video transmission under a V4L2 framework, characterized in that: The following steps are involved: Collect the CPU usage of the USB video transmission process and the current packet loss rate of USB video transmission , Based on the current packet loss rate , determine the current network status, If the current packet loss rate ≤ pre-trial packet loss rate, the network status is regarded as the first network status. Otherwise, the network status is regarded as the second network status. In the first network status, only the CPU usage is used to To determine the number of buffers, in the second network state, combine the CPU usage and the current packet loss rate Determine the number of buffers, In the first network state, the first solution is used to determine the number of buffers. The first solution is based only on CPU usage. To determine the number of buffers, in the second network state, the second solution is used to determine the number of buffers. The second solution is to combine the CPU usage and the current packet loss rate Determine the number of buffers, Solution 1: Use formula ① to determine the theoretical number of buffers that need to be set : ------① In the formula, is the slope, is the preset constant, is the intercept, is a preset constant, Solution 2: Use formula ② to determine the theoretical number of buffers that need to be set : ------② In the formula, , , , , , , , , represents the coefficient, which is a constant, The actual number of buffers is calculated as follows: : ， In the formula, round() means rounding to an integer, and M means the maximum number of buffers allowed to be set.

2. The method for determining the number of buffers for USB video transmission under the V4L2 framework according to claim 1, characterized in that: First, the process identifier PID of the USB video transmission program is obtained, and then the CPU usage rate of the CPU occupied by the current process is queried according to the process identifier PID.

3. The method for determining the number of buffers for USB video transmission under the V4L2 framework according to claim 1, characterized in that: In the first approach, multiple experiments were performed to determine the theoretical number of buffer zones. CPU usage under , to fit and , according to the fitting and The resulting formula ① determines the current CPU usage Theoretical number of buffers under , In the second solution, multiple groups of different packet loss rates were measured through multiple tests. and the number of buffers After that, multiple sets of data are obtained, and the 8 coefficients of formula ② are solved by mathematical methods, so as to determine the current CPU usage rate based on the formula ② formed by the solved coefficients. and the current packet loss rate Theoretical number of buffers under .

4. A processing terminal, characterized in that: It includes: A memory for storing program instructions; A processor is used to run the program instructions to execute the steps of the method for determining the number of buffers for USB video transmission under the V4L2 framework as described in any one of claims 1-3.

Images