A video bit rate identification method for encrypted video, a video playback index estimation method and device

Abstract

The present invention relates to a video bit rate identification method, a video playback index estimation method and a device for encrypted video, wherein the video playback index estimation method comprises S1, generating an actual download integral curve by statistics of captured video transmission data packets; S2, aggregating captured video transmission data packets based on three granularities, namely, request granularity, extremum granularity, and time window granularity, and determining the download mode; S3, estimating the video bit rate within the time window, S4, generating a theoretical playback integral curve; S5, estimating the actual playback integral curve; S6, calculating various video playback indicators. The method and device of the present invention can accurately identify whether the encrypted video is playing normally or is stuck, can accurately estimate the real-time bit rate of the video, and can estimate various stuck indicators, including video duration, initial buffering duration, stuck time point, stuck recovery time point, stuck duration, etc.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile communication networks, and in particular relates to a video bit rate recognition method for encrypted videos, and a video playback index estimation method and device. Background Art

[0002] Video playback fluency is a key factor in evaluating the quality of mobile networks. To evaluate video fluency, you first need to obtain the video bit rate, which can usually be obtained from the media description in the video stream header file using deep packet inspection (DPI) technology. However, when the video is transmitted using an encrypted protocol, information such as the bit rate of the video fragments cannot be obtained from the media description. Key indicators for evaluating video service quality need to be calculated based on the video bit rate. In order to calculate video playback indicators when the bit rate is unknown, the industry's commonly used solution is to set the bit rate to a default fixed value. However, video service indicators calculated based on this fixed bit rate are obviously not accurate enough.

[0003] The industry has also conducted some research on encrypted video evaluation. One type uses supervised learning methods to extract features from labeled data to train the model, which requires a large amount of training data. The estimation accuracy is directly related to the amount of training data and the coverage scenario. The second type is a business-based method, but it uses video parameters as prior information, such as video bit rate, video length, etc., which is obviously unreasonable. The third type is a business-based unsupervised, no-priori method, but it only targets a single normal and smooth download mode. This type of video is not the focus of video playback indicator identification. The focus is on whether the freeze can be identified and whether the degree of freeze can be identified and described. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the prior art and provide a method for identifying the video bit rate of an encrypted video and a method and device for estimating video playback indicators.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] Technical solution 1:

[0007] A video bit rate identification method for encrypted video, comprising:

[0008] Aggregate the captured video transmission data packets based on the three granularities of request granularity, extremum granularity, and time window granularity, and determine the download mode;

[0009] For the download mode, estimate the video bit rate within the time window and obtain the benchmark bit rate R of the video B ;

[0010] Furthermore, the method for aggregating captured video transmission data packets based on request granularity, extremum granularity, and time window granularity is as follows:

[0011] S21: Aggregate and count request-level packets

[0012] S22, Extremum granular packet aggregation and statistics

[0013] S23, identification of download mode.

[0014] Furthermore, in S21, the specific method for aggregating and counting the request granularity data packets is as follows:

[0015] Statistics are collected on captured video transmission data packets, two uplink download request data packets and downlink data packets during the two uplink download request data packets are defined as a request statistical granularity, and statistical features of the request granularity are obtained;

[0016] In S22, the Extremum granularity packet aggregation and statistics method is:

[0017] S221, identifying the minimum value of Request;

[0018] S222, based on the Request granularity, perform Extremum granularity aggregation;

[0019] In S23, the method of identifying the download mode is:

[0020] S231, identifying a valid Request;

[0021] S232, take a video data segment that meets the total valid request duration greater than or equal to 3×T0 seconds, form a set R, and calculate the download time duty ratio r in the set R d The proportion of valid requests with a request duration greater than T0 r s ;

[0022] S233, based on the download time duty cycle r d The proportion of valid requests with a request duration greater than T0 r s ,, determine the download mode;

[0023] Among them, T0 is the duration of the video block.

[0024] Furthermore, in S231, the minimum Request value must simultaneously satisfy the following conditions:

[0025] 1) The download data amount of the Request is smaller than the download data amount of the previous Request;

[0026] 2) The download data volume of the Request is smaller than the download data volume of the next Request;

[0027] 3) The download data size of the Request is less than 250kB.

[0028] Furthermore, in S222, the method for performing the Extremum granularity aggregation is: defining all Requests between two Request minimum values ​​as an Extremum granularity, and obtaining the statistical characteristics of the Extremum granularity.

[0029] Furthermore, the statistical characteristics of the Extremum particle size include:

[0030] 1) Initial extreme value Request;

[0031] 2) Terminate the extreme value Request;

[0032] 3) Request sequence list from start to end <request>;

[0033] 4) Total amount of Request download data in Extremum granularity.

[0034] Furthermore, in S231, the valid Request satisfies the following conditions at the same time:

[0035] 1) The amount of data downloaded by the request is greater than 10kB;

[0036] 2) The non-downloading time of Request is less than T0 / 2;

[0037] Among them, T0 is the duration of the video block.

[0038] Furthermore, the statistical characteristics of the request granularity include:

[0039] 1) Start Request timestamp t start ;

[0040] 2) Termination Request timestamp t end ;

[0041] 3) Download completion timestamp t in Request download_over ;

[0042] 4) The amount of downloaded data in the Request.

[0043] Further, in S232, the download time duty cycle r d The calculation formula is: Where R is the set of valid Requests;

[0044] The proportion of valid requests with a request duration greater than T0 is r s The calculation formula is:

[0045]

[0046] in, is the number of valid requests in set R whose request duration is greater than T0;

[0047] The total number of valid Requests in set R.

[0048] Further, in S233, based on the download time duty cycle r d The proportion of valid requests with a request duration greater than T0 r s ,,The method to determine the download mode is:

[0049] First determine the download time duty cycle r d , when r d When the value is less than the threshold value th1, it is judged as the smooth download mode;

[0050] When d ≥ threshold th1, the proportion of Request duration greater than T0 r s To judge, when r s When the value is less than the threshold value th2, it is determined to be in balanced download mode;

[0051] When s When the value is greater than or equal to the threshold value th2, it is determined to be a stuck download mode.

[0052] Furthermore, the method for aggregating captured video transmission data packets based on request granularity, extreme granularity, and time window granularity also includes S24, time window granularity data packet aggregation and statistics.

[0053] Furthermore, in S24, the method of aggregating and counting data packets at the time window granularity is:

[0054] S241, Identify effective Extremum;

[0055] S242, based on the Extremum granularity, aggregate the time window granularity;

[0056] Furthermore, in S241, the effective Extremum must simultaneously meet the following conditions:

[0057] 1) The total amount of Extremum downloaded data is greater than 250kB;

[0058] 2) When the download mode is determined to be the jamming download mode or the balanced download mode, the Extremum download time duty cycle should be greater than 0.8.

[0059] Further, in S242, the length of the aggregation time window is taken as th n The effective Extremum time length and the time length threshold th t The smaller value in .

[0060] Furthermore, for the download mode, the method for estimating the video bit rate within the time window is:

[0061] 1) When the download mode is a periodic download mode, the real-time bit rate is estimated by using a method for estimating the real-time bit rate in the periodic download mode; the periodic download mode includes a smooth download mode and a stuck download mode.

[0062] 2) When the download mode is non-periodic download mode, first, calculate the size variation coefficient c of all valid requests within the time window v ;

[0063]

[0064] Among them, σ is the standard deviation of request size, μ is the mean of request size;

[0065] Then, set the threshold value th cv , when c v >th cv When c v ≤th cv When the real-time bit rate estimation method in the non-periodic download mode is used, the real-time bit rate estimation method is continued to be used to estimate the real-time bit rate;

[0066] The non-periodic downloading mode includes a balanced downloading mode.

[0067] Furthermore, the method for estimating the real-time bit rate in the periodic download mode is: estimating the real-time bit rate based on the Extremum granularity within the time window.

[0068] Furthermore, the method for estimating the real-time bit rate in the periodic download mode is as follows: in the periodic download mode, there are multiple valid extremums in the time window, the data size of all valid extremums in the time window is calculated, and the median of the valid extremum size is taken as the reference value V0 of the video segment size; the reference bit rate R in the time window B =V0 / T0;

[0069] Among them, T0 is the duration of the video block.

[0070] Furthermore, the method for estimating the real-time bit rate in the non-periodic download mode is: estimating the real-time bit rate based on the Request granularity within the time window.

[0071] Furthermore, the data size of all valid requests in the time window is calculated, and the median of the valid request size is taken as the benchmark value V0 of the video segment size; the benchmark bit rate R in the time window B =V0 / T0;

[0072] Among them, T0 is the duration of the video block.

[0073] Furthermore, the method for obtaining the video block duration T0 is: observing the captured video transmission data, determining the size of the audio block and the preset audio sampling rate, and estimating the video block duration. Assuming the size of the audio block is H and the audio sampling rate is M, the video block duration T0 = H / M, where H / M is an integer.

[0074] Technical solution 2:

[0075] A method for estimating encrypted video playback indicators, comprising:

[0076] An actual download integral curve is generated by statistics of captured video transmission data packets;

[0077] Identify the video bit rate using the video bit rate identification method for the encrypted video as described above;

[0078] Generate theoretical playback integral curve;

[0079] Estimate the actual playback integral curve;

[0080] Calculate various video playback metrics.

[0081] Furthermore, the specific method of generating the actual download integral curve by statistics of the captured video transmission data packets is as follows: in chronological order, the sizes of the data packets downloaded before each time point are accumulated, and a graph is drawn with the download time as the horizontal axis and the total size of the data packets downloaded before each time point as the vertical axis to form a download data volume integral curve.

[0082] Furthermore, the method for drawing the theoretical playback integral curve is:

[0083] S41, assuming that the video is played without interruption, within the time window, the video playback time is used as the horizontal axis and the reference bit rate within the time window is used as the slope to form a theoretical playback integral curve; the vertical axis is the total amount of data accumulated up to the current video playback time;

[0084] S42, connecting the theoretical playback integral curve of each time window drawn in S41 in chronological order to form a theoretical playback integral curve of the entire video.

[0085] Furthermore, the actual playback integral curve is estimated based on the theoretical playback integral curve and the download integral curve.

[0086] Furthermore, the method for estimating the actual playback integral curve is:

[0087] S51, set buffering time t buf parameter,

[0088] S52, shift the theoretical playback curve to the right and place it below the actual download integral curve, and ensure that at the initial playback and freeze recovery time points, the download curve is larger than the playback curve by the buffering time t buf The corresponding video data volume can obtain the actual playback integral curve.

[0089] Furthermore, the video playback indicators include video duration, initial buffering duration, freeze time point, freeze recovery time point, and freeze duration.

[0090] Furthermore, the calculation of the video playback index is performed based on an actual playback curve.

[0091] Furthermore, 1) video duration: that is, the end time point of the theoretical playback curve;

[0092] 2) Initial buffering time: the initial download time t on the actual playback curve buf The time point of the video data volume;

[0093] 3) Freeze time point: the time point corresponding to the intersection of the actual playback curve and the actual download curve, that is, the time point when the actual download data volume = the actual playback data volume;

[0094] 4) Jam recovery time point: that is, the download time t after jamming on the actual playback curve buf Video data volume time point;

[0095] 5) Freeze duration: The freeze duration of each freeze is calculated by dividing the freeze recovery time point by the freeze time point.

[0096] Technical solution three:

[0097] A video bit rate identification device for encrypted video, comprising:

[0098] The data packet aggregation and statistics module is used to aggregate the captured video transmission data packets based on the three granularities of request granularity, extremum granularity, and time window granularity, and determine the download mode;

[0099] The video bit rate estimation module is used to estimate the video bit rate within a time window for videos in different download modes.

[0100] Furthermore, the data packet aggregation and statistics module is used for request granularity data packet aggregation and statistics, extreme granularity data packet aggregation and statistics, download mode recognition, and time window granularity data packet aggregation and statistics.

[0101] Furthermore, the video bit rate estimation module is used in the periodic download mode to estimate the real-time bit rate based on the Extremum granularity; and is used in the non-periodic download mode to estimate the real-time bit rate based on the Request granularity.

[0102] Technical solution 4:

[0103] A device for estimating encrypted video playback index, comprising:

[0104] A download integral curve generating module, used for generating an actual download integral curve by statistics from captured video transmission data packets;

[0105] The video bit rate identification device for the encrypted video as described above;

[0106] A theoretical playback integral curve generation module, used to generate a theoretical playback integral curve according to the video bit rate estimated within the time window;

[0107] Video playback index estimation module: used to estimate the actual playback integral curve based on the actual download integral curve and the theoretical playback integral curve, and calculate the video playback index.

[0108] Compared with the prior art, the present invention has the following beneficial effects:

[0109] The present invention captures packets of video transmission data, generates an actual download integral curve by statistics of captured data packets, aggregates captured data packets based on three granularities, namely, request granularity, extremum granularity, and time window granularity, estimates the video bit rate within the time window, generates a theoretical playback integral curve, and finally estimates the actual playback integral curve and various video playback indicators. The method and device of the present invention can accurately identify whether the encrypted video is playing normally or is stuck, can accurately estimate the real-time bit rate of the video, and can estimate various stuck indicators, including video duration, initial buffering duration, stuck time point, stuck recovery time point, stuck duration, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0110] Figure 1 It is a schematic diagram of a flow chart of a method for estimating encrypted video playback index according to an embodiment of the present invention;

[0111] Figure 2 is a schematic diagram of an actual download integral curve in one embodiment of the present invention;

[0112] Figure 3 It is a flow chart of a method for aggregating data packets based on three granularities: request granularity, extreme granularity, and time window granularity in one embodiment of the present invention;

[0113] Figure 4 is a schematic diagram of a normal download mode in one embodiment of the present invention;

[0114] Figure 5 is a schematic diagram of a stuttering download mode in one embodiment of the present invention;

[0115] Figure 6 is a schematic diagram of a balanced download mode in one embodiment of the present invention;

[0116] Figure 7 is a schematic diagram of a download pattern recognition binary tree in one embodiment of the present invention;

[0117] Figure 8 is a schematic diagram of a flow chart of estimating a video bit rate within a time window according to an embodiment of the present invention;

[0118] Fig. 9 A schematic diagram of a flow chart of a method for identifying a video bit rate in one embodiment of the present invention;

[0119] Fig.10 is a schematic diagram of a method for calculating video playback indicators in one embodiment of the present invention;

[0120] Fig.11 is a schematic diagram of an actual playback integral curve in one embodiment of the present invention;

[0121] Fig.12 It is a structural block diagram of an encrypted video playback index estimation device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0122] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0123] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0124] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0125] The present invention is described in further detail below with reference to the accompanying drawings.

[0126] like Figure 1 , 3 , 7, and 8 show an embodiment of a method for estimating a video playback index of the present invention, comprising:

[0127] S1, generating an actual download integral curve by statistics of captured video transmission data packets;

[0128] In chronological order, the sizes of the data packets downloaded before each time point are accumulated, and a graph is drawn with the download time as the horizontal axis and the total size of the data packets downloaded before each time point (unit: bytes) as the vertical axis, such as Figure 2 As shown, a download data amount integral curve is formed. In the figure, the slope of the tangent line at each point represents the real-time rate of data download at the current time.

[0129] S2, based on the three granularities of request granularity, extremum granularity, and time window granularity, the captured video transmission data packets are aggregated and the download mode is determined;

[0130] S21: Aggregate and count request-level packets

[0131] Specifically, the terminal needs to send a download request uplink to download a video clip, and then transmit the video data downlink. After the video transmission is completed, it sends a request uplink to download the next video clip. Two uplink download requests represent a complete data block download; statistics are collected on the captured video transmission data packets, and the two uplink download request data packets and the downlink data packets during the two uplink download request data packets are defined as a request statistical granularity, and the statistical characteristics of the request granularity are obtained;

[0132] The statistical characteristics of the request granularity include:

[0133] 1) Start Request timestamp t start ;

[0134] 2) Termination Request timestamp t end ;

[0135] 3) Download completion timestamp t in Request download_over ;

[0136] 4) The amount of data downloaded in the Request (bytes);

[0137] Among them, the termination Request timestamp t end That is, the start request timestamp t of the next request granularity start ;

[0138] S22, Extremum granular packet aggregation and statistics

[0139] S221, identify the minimum value of Request,

[0140] The minimum Request value must meet the following conditions at the same time:

[0141] 1) The Request is smaller than the previous Request;

[0142] 2) The Request is smaller than the next Request;

[0143] 3) The request is smaller than 250kB (approximately the size of a 144P video of 10s).

[0144] S222, based on the Request granularity, perform Extremum granularity aggregation;

[0145] The size of requests shows a periodic change pattern. Extremum granularity aggregation is based on the request granularity. All requests between two request minimum values ​​are defined as an extreme granularity to obtain the statistical characteristics of the extreme granularity.

[0146] The statistical characteristics of the Extremum particle size include:

[0147] 1) Initial extreme value Request;

[0148] 2) Terminate the extreme value Request;

[0149] 3) Request sequence List<Request> from start to end;

[0150] 4) The total amount of Request downloaded data in the Extremum granularity (including the starting extreme value Request, excluding the ending extreme value Request).

[0151] In order to clearly show the above statistical characteristics, they are now represented in the form of a bar chart:

[0152] Specifically, the captured video transmission data packets are plotted into a bar chart of data block downloads based on the statistical characteristics of the above request granularity, such as Figures 4 to 6 As shown, each column is a data block corresponding to a Request; the top edge of the column is the start time of data block download, that is, the start Request timestamp t start The lower edge of the column is the time when the data block ends downloading, which is also the download completion timestamp t in the Request. download_over , the column width indicates the actual downloading time of the data block, and the column height indicates the size of the data block, that is, the amount of downloaded data in the Request (bytes);

[0153] By observing the download histograms of different video transmission data blocks, the encrypted video download mode is defined into three categories: smooth download mode, stuttering download mode and balanced download mode.

[0154] In smooth download mode, the video plays smoothly. During the video transmission process, the video and audio are transmitted alternately. The size of the request shows a periodic change pattern, which is shown in Figure 4 That is, large data blocks and small data blocks are transmitted alternately, and the size of the data blocks changes periodically, usually for 5s or 10s. Usually, the large data blocks are video data blocks, and the small data blocks are audio data blocks. The download duty cycle is very small, the bandwidth can fully meet the video download requirements, and the video playback is smooth. Figure 4 The three requests in the middle box A (solid line box) are aggregated into one Extremum;

[0155] In the stuttering download mode, the video playback is stuttering, as shown below: Figure 5 That is, large data blocks and small data blocks are transmitted alternately, and the size of the data blocks shows a periodic change pattern (the period size is uncertain, usually greater than 10s, related to the degree of bandwidth limitation). Audio and video are transmitted in large data blocks, and small data blocks only transmit request messages. The download duty cycle is very large, and bandwidth becomes the bottleneck of video transmission, causing video playback to be stuck.

[0156] Balanced download mode, such as Figure 6 As shown in the figure, the data block size does not show an obvious periodic pattern (the data block interval is usually 5s to 10s), and one data block corresponds to a video block and an audio block of a time segment. At this time, the download rate is equivalent to the video playback rate, the download duty cycle is large, and the video playback is normal.

[0157] S23, identification of download mode.

[0158] Download mode identification is to determine whether the download data packets belong to one of the three download modes mentioned above based on the statistics of the download data packets in a certain period of time. Download mode identification requires at least that the data accumulation satisfies the total valid request duration of not less than 3×T0 seconds, forming a set R, where T0 is the video block duration.

[0159] S231, identify valid Request,

[0160] The valid request satisfies the following conditions at the same time:

[0161] 1) The amount of data downloaded by the request is greater than 10kB;

[0162] 2) The non-downloading time of Request is less than T0 / 2;

[0163] Wherein, T0 is the duration of the video block.

[0164] S232, take a video data segment that meets the total valid request duration greater than or equal to 3×T0 seconds, form a set R, and calculate the download time duty ratio r in the set R d The proportion of valid requests with a request duration greater than T0 r s ;

[0165] 1) The download time duty cycle r d : The ratio of the total download duration of valid requests to the total duration of requests in set R;

[0166] The calculation formula is: Where R is the set of valid Requests;

[0167] 2) The proportion of valid requests whose request duration is longer than T0 s : In set R, the proportion of valid requests with a request duration greater than T0 to the total number of requests during the statistical period;

[0168] The calculation formula is:

[0169] in, is the number of valid requests in set R whose request duration is greater than T0;

[0170] is the total number of valid Requests in set R,

[0171] T0 is the download duration of the video block, which is a configurable parameter;

[0172] The method for obtaining the video block download time T0 is: observe the captured video transmission data, determine the size of the audio block and the preset audio sampling rate, and estimate the video block download time T0. Assume that the size of the audio block is H and the audio sampling rate is M, then the video block time T0 = H / M, where H / M is an integer.

[0173] The following is a specific example to illustrate the method of obtaining the video block download duration T0: Considering the video playback index estimation problem as a whole, the amount of downloaded data is known, and the playback rate is to be determined, the video duration must be estimated first. In the smooth download mode, video blocks and audio blocks can be distinguished. The size of the video block varies greatly with the clarity, but the sampling rate of the audio is usually not much different. Therefore, the video duration can be estimated by the size of the audio block and the preset audio sampling rate. Taking the Youtube Android platform as an example, assuming the audio sampling rate M = 128kbps = 16kBps, it is observed from the data that the size of the audio block is usually around H = 161kB, then the video duration is about T0 = H / M = 161 / 16≈10s, that is, T0 = 10s; in the stuck download mode and balanced download mode, the video block duration is also 10s, so the video block duration of 10s is used as the premise for download mode determination and subsequent bit rate estimation.

[0174] S233, based on the download time duty cycle r d The proportion of valid requests with a request duration greater than T0 r s ,, determine the download mode.

[0175] The three download modes can be classified using a binary tree based on the two features extracted above, such as Figure 7 As shown,

[0176] First, the download time duty cycle rd is determined. When rd is less than the threshold th1, it is determined to be a smooth download mode;

[0177] When d ≥ threshold th1, the proportion of Request duration greater than T0 r s To judge, when r s When the value is less than the threshold value th2, it is determined to be in balanced download mode;

[0178] When s When the value is greater than or equal to the threshold value th2, it is determined to be a stuck download mode;

[0179] Wherein, th1 and th2 are configurable parameters, and usually th1=0.8 and th2=0.1.

[0180] S24: Time window granularity data packet aggregation and statistics.

[0181] S241, Identify effective Extremum

[0182] A valid Extremum must meet the following conditions:

[0183] 1) The total amount of Extremum downloaded data is greater than 250kB;

[0184] 2) When the download mode is determined to be the stuck download mode or the balanced download mode, the Extremum download time duty cycle should be greater than 0.8.

[0185] Among them, the Extremum download time duty cycle refers to the proportion of the time spent downloading in Extremum to the total time of Extremum.

[0186] S242, based on the Extremum granularity, aggregate the time window granularity;

[0187] The length of the aggregation time window is taken as n (Usually take th n =5) valid Extremum time lengths, and time length threshold th t (Usually take th t =150)

[0188] To clearly explain the definition of effective Extremum, the following is based on the attached Figures 4 to 6 The three download modes shown give valid Extremum recognition results.

[0189] Smooth download mode, such as Figure 4 As shown, the box B (dashed box) is a time window. This time window contains 5 valid Extremums and 1 invalid Extremum (the amount of downloaded data does not meet condition 1)

[0190] Stuttering download mode, such as Figure 5 As shown, the frame C (dashed line frame) encloses a time window. This time window contains 2 valid extremums and multiple invalid extremums (the condition 1 is not satisfied at the frame D (solid line circle), and the condition 2 is not satisfied at the frame E (dashed line circle)).

[0191] Balanced download mode, such as Figure 6 As shown, the video download duration is 115 ms, with 2 valid extremums and 1 invalid extremum (the condition 2 is not met at the box F (dashed box), and the extremum download time duty cycle should be greater than 0.8); Figure 6 There is only 1 time window in the video download shown.

[0192] S3: For the download mode, estimate the video bit rate within the time window and obtain the video benchmark bit rate R B ;

[0193] The video download duration T0 has been estimated before. The three download modes mentioned above, the smooth download mode and the stuck download mode can be unified into a periodic download mode. Each period corresponds to an Extremum, downloading a video block of duration T0; the stuck download mode corresponds to the non-periodic download mode, and each Request (corresponds to downloading a video block of duration T0. The periodic and non-periodic download modes use different playback rate estimation methods.

[0194] 1) When the download mode is a periodic download mode, the real-time bit rate is estimated by using a method for estimating the real-time bit rate in the periodic download mode; the periodic download mode includes a smooth download mode and a stuck download mode.

[0195] The method for estimating the real-time bit rate in the periodic download mode is as follows: within the time window, the real-time bit rate is estimated based on the Extremum granularity;

[0196] Specifically, in the periodic download mode, there are multiple valid extremums in the time window. The data size of all valid extremums in the time window is calculated, and the median of the valid extremum size is taken as the benchmark value V0 of the video segment size; the video duration is also T0, the same as the video block duration, and the benchmark bit rate R in the time window B =V0 / T0;

[0197] The method for obtaining the median size of the valid Extremum is: among all valid Extremums, the valid Extremums are sorted according to the data volume. When the number of valid Extremums is an odd number, the median size of the valid Extremum is: the data volume value of the valid Extremum in the middle position; when the number of valid Extremums is an even number, the median size of the valid Extremum is: the average of the data volume values ​​of the two valid Extremums in the middle position.

[0198] 2) When the download mode is non-periodic download mode, first, calculate the size variation coefficient c of all valid requests within the time window v The coefficient of variation represents the degree of data dispersion. The larger the coefficient of variation, the more discrete the data.

[0199]

[0200] Among them, σ is the standard deviation of request size, μ is the mean of request size;

[0201] Then, set the threshold value th cv (Usually take th cv =0.5), when c v >th cv When c v ≤th cv When the real-time bit rate estimation method in the non-periodic download mode is used, the real-time bit rate estimation method is continued to be used to estimate the real-time bit rate;

[0202] The non-periodic downloading mode includes a balanced downloading mode.

[0203] The method for estimating the real-time bit rate in the non-periodic download mode is as follows: within the time window, the real-time bit rate is estimated based on the Request granularity;

[0204] Specifically, the data size of all valid requests in the time window is calculated. Among all valid requests, the median of the valid request size is taken as the benchmark value V0 of the video segment size; the video duration is also taken as T0, the same as the video block duration, and the benchmark bit rate R in the time window B =V0 / T0;

[0205] The method for obtaining the median size of the valid Request is: among all the valid Requests, sort the valid Requests according to the size of the data. When the number of valid Requests is an odd number, the median size of the valid Request is: the data size value of the valid Request in the middle position; when the number of valid Requests is an even number, the median size of the valid Request is: the average of the data size values ​​of the two valid Requests in the middle position.

[0206] S4, generating a theoretical playback integral curve;

[0207] S41, assuming that the video is played without interruption, within the time window, the video playback time is used as the horizontal axis and the reference bit rate within the time window is used as the slope to form a theoretical playback integral curve; the vertical axis is the total amount of data accumulated up to the current video playback time;

[0208] S42, connecting the theoretical playback integral curve of each time window drawn in S41 in chronological order to form a theoretical playback integral curve of the entire video.

[0209] S5, estimating the actual playback integral curve;

[0210] The estimation of the actual playback integral curve is based on the theoretical playback integral curve and the download integral curve;

[0211] S51, set buffering time t buf Parameter, (usually t buf = 5s). When the video is initially buffered, it needs to load enough buf Start playing after the video; if there is a freeze, you need to load enough t buf Video resumes after a freeze.

[0212] S52, such as Fig.10 As shown, the theoretical playback integral curve is shifted to the right and placed below the actual download integral curve, and it is ensured that at the initial playback and freeze recovery time points, the download curve is larger than the playback curve by the buffering time t buf The corresponding video data volume can be used to obtain the actual playback integral curve. h0 is the initial buffering time of the video buf The amount of duration data, h1 is the time required to cache the video after it is played back after being stuck at time t1. buf Duration and amount of data.

[0213] Specifically, S521, through the bottom end of the original theoretical playback integral curve, that is, the initial point, draw a line segment X parallel to the x-axis, and the end point X' of the line segment X is the distance to the actual download curve, corresponding to the initial playback or buffering t during the freeze. buf The amount of data corresponding to the duration, and then move the theoretical playback curve rightward to the end point X' of the line segment x, that is, the rightward shift of the theoretical playback curve is completed once, S522, when the theoretical playback integral curve after rightward shift and the actual download integral curve still intersect, intercept the part above the actual download integral curve at the intersection point X", repeat step S521, until the original theoretical integral curve is completely moved rightward to the bottom of the actual playback curve, this is to connect the multiple segments of the theoretical playback curve after rightward shift with the line segment X, and you can get the following Figures 10-11 The actual playback integral curve is shown.

[0214] S6. Calculate various video playback indicators.

[0215] The calculation of the video playback index is performed based on the actual playback curve.

[0216] 1) Video duration: the end time of the theoretical playback curve (e.g. Fig.10 t in v );

[0217] 2) Initial buffering time: the initial download time t on the actual playback curve buf The time point of the video data volume (h0) (e.g. Fig.10 t0 in );

[0218] 3) Freeze time point: the time point corresponding to the intersection of the actual playback curve and the actual download curve, that is, the time point when the actual download data volume = the actual playback data volume (for example Fig.10 t1, t3, t5, t7);

[0219] 4) Jam recovery time point: that is, the download time t after jamming on the actual playback curve buf Video data volume time point (such as Fig.10 t2, t4, t6, t8);

[0220] 5) Freeze duration: The freeze duration of each freeze is calculated by dividing the freeze recovery time point by the freeze time point.

[0221] Among them, in the above-mentioned encrypted video playback index estimation, S2, based on the three granularities of request granularity, extremum granularity, and time window granularity, aggregates the captured video transmission data packets and determines the download mode; S3, for the download mode, estimates the video bit rate within the time window to obtain the video benchmark bit rate R B ; The process of the video bit rate identification method for encrypted video.

[0222] To implement the above method, Fig.12 As shown, the present invention also provides an encrypted video playback index estimation device,

[0223] A download integral curve generating module 1 is used to generate an actual download integral curve by statistics based on captured video transmission data packets;

[0224] The data packet aggregation and statistics module 2 is used to aggregate the captured video transmission data packets based on the three granularities of request granularity, extremum granularity, and time window granularity, and determine the download mode;

[0225] The video bit rate estimation module 3 is used to estimate the video bit rate of videos in different download modes within a time window.

[0226] Theoretical playback integral curve generation module 4, used to generate a theoretical playback integral curve according to the video bit rate estimated in the time window;

[0227] Video playback index estimation module 5: used to estimate the actual playback integral curve based on the actual download integral curve and the theoretical playback integral curve, and calculate the video playback index.

[0228] Among them, the data packet aggregation and statistics module and the video bit rate estimation module are video bit rate estimation devices for encrypted videos.

[0229] Specifically, the data packet aggregation and statistics module 2 is used for request granularity data packet aggregation and statistics, extreme granularity data packet aggregation and statistics, download mode recognition, and time window granularity data packet aggregation and statistics.

[0230] The video bit rate estimation module 3 is used in the periodic download mode to estimate the real-time bit rate based on the Extremum granularity; and is used in the non-periodic download mode to estimate the real-time bit rate based on the Request granularity.

[0231] The above-described embodiments are only preferred embodiments of the present invention, and are not exhaustive of the feasible implementations of the present invention. For those skilled in the art, any obvious changes made thereto without departing from the principles and spirit of the present invention should be considered to be included in the scope of protection of the claims of the present invention.< / request>

Claims

1. A method for identifying the video bit rate of an encrypted video, characterized in that: include Aggregate the captured video transmission data packets based on the three granularities of request granularity, extremum granularity, and time window granularity, and determine the download mode; For the download mode, estimate the video bit rate within the time window and obtain the benchmark bit rate R of the video B ; The method for aggregating captured video transmission data packets based on request granularity, extremum granularity, and time window granularity is as follows: S21, aggregate and count the request-sized data packets; S22, Extremum granularity packet aggregation and statistics; S23, identification of download mode; In S21, the specific method for aggregating and counting request-sized data packets is as follows: Statistics are collected on captured video transmission data packets, two uplink download request data packets and downlink data packets during the two uplink download request data packets are defined as a request statistical granularity, and statistical features of the request granularity are obtained; In S22, the Extremum granularity packet aggregation and statistics method is: S221, identifying the minimum value of Request; S222, based on the Request granularity, perform Extremum granularity aggregation; In S23, the method of identifying the download mode is: S231, identifying a valid Request; S232, take a video data segment that meets the total valid request duration greater than or equal to 3×T0 seconds, form a set R, and calculate the download time duty ratio r in the set R d The proportion of valid requests with a request duration greater than T0 r s ; S233, based on the download time duty cycle r d The proportion of valid requests with a request duration greater than T0 r s , determine the download mode; Among them, T0 is the duration of the video block; For download mode, the method to estimate the video bitrate within the time window is: 1) When the download mode is a periodic download mode, the real-time bit rate is estimated by using a method for estimating the real-time bit rate in the periodic download mode; the periodic download mode includes a smooth download mode and a stuck download mode; 2) When the download mode is non-periodic download mode, first, calculate the size variation coefficient c of all valid requests within the time window v , Among them, σ is the standard deviation of request size, μ is the mean of request size; Then, set the threshold value th cv , when c v >th cv When c v ≤th cv When the real-time bit rate estimation method in the non-periodic download mode is used, the real-time bit rate estimation method is continued to be used to estimate the real-time bit rate; The non-periodic download mode includes a balanced download mode; The method for estimating the real-time bit rate in the periodic download mode is as follows: within the time window, the real-time bit rate is estimated based on the Extremum granularity; The method for estimating the real-time bit rate in the non-periodic download mode is: estimating the real-time bit rate based on the Request granularity within the time window.

2. The method for identifying the video bit rate of an encrypted video according to claim 1, characterized in that: The statistical characteristics of the Extremum particle size include: 1) Initial extreme value Request; 2) Terminate the extreme value Request; 3) Request sequence list from start to end <request> ；< / request> 4) Total amount of Request download data in Extremum granularity.

3. The method for identifying the video bit rate of an encrypted video according to claim 1, characterized in that: The statistical characteristics of the request granularity include: 1) Start Request timestamp t start ; 2) Termination Request timestamp t end ; 3) Download completion timestamp t in Request download_over ; 4) The amount of downloaded data in the Request.

4. The method for identifying the video bit rate of an encrypted video according to claim 1, characterized in that: In S231, the minimum Request value must satisfy the following conditions at the same time: 1) The download data amount of the Request is smaller than the download data amount of the previous Request; 2) The download data volume of the Request is smaller than the download data volume of the next Request; 3) The download data size of the Request is less than 250kB.

5. The method for identifying the video bit rate of an encrypted video according to claim 1, characterized in that: In S222, the method for performing the Extremum granularity aggregation is: defining all Requests between two Request minimum values ​​as an Extremum granularity, and obtaining the statistical characteristics of the Extremum granularity.

6. The method for identifying the video bit rate of an encrypted video according to claim 3, characterized in that: In S231, the valid Request satisfies the following conditions at the same time: 1) The amount of data downloaded by the request is greater than 10kB; 2) The non-downloading time of Request is less than T0 / 2; Among them, T0 is the duration of the video block.

7. The method for identifying the video bit rate of an encrypted video according to claim 6, characterized in that: In S232, the download time duty cycle r d The calculation formula is: Where R is the set of valid Requests; The proportion of valid requests with a request duration greater than T0 is r s The calculation formula is: in, is the number of valid requests in set R whose request duration is greater than T0; The total number of valid Requests in set R.

8. The method for identifying the video bit rate of an encrypted video according to claim 1, characterized in that: In S233, based on the download time duty cycle r d The proportion of valid requests with a request duration greater than T0 r s , the method to determine the download mode is: First determine the download time duty cycle r d , when r d When the value is less than the threshold value th1, it is judged as the smooth download mode; When d ≥ threshold th1, the proportion of Request duration greater than T0 r s To judge, when r s When the value is less than the threshold value th2, it is determined to be in balanced download mode; When s When the value is greater than or equal to the threshold value th2, it is determined to be a stuck download mode.

9. The method for identifying the video bit rate of an encrypted video according to claim 1, characterized in that: The method for aggregating captured video transmission data packets based on request granularity, extremum granularity, and time window granularity also includes S24, time window granularity data packet aggregation and statistics.

10. The method for identifying the video bit rate of an encrypted video according to claim 9, characterized in that: In S24, the method for aggregating and counting packets at the time window granularity is: S241, Identify effective Extremum; S242: Based on the Extremum granularity, aggregate the time window granularity.

11. The method for identifying the video bit rate of an encrypted video according to claim 10, characterized in that: In S241, the effective Extremum must meet the following conditions at the same time: 1) The total amount of Extremum downloaded data is greater than 250kB; 2) When the download mode is determined to be the jamming download mode or the balanced download mode, the Extremum download time duty cycle should be greater than 0.

8.

12. The method for identifying the video bit rate of an encrypted video according to claim 10, characterized in that: In S242, the length of the aggregation time window is th n The effective Extremum time length and the time length threshold th t The smaller value in .

13. The method for identifying the video bit rate of an encrypted video according to claim 10, characterized in that: The method for estimating the real-time bit rate in the periodic download mode is as follows: in the periodic download mode, there are multiple valid extremums in the time window, the data size of all valid extremums in the time window is calculated, and the median of the valid extremum size is taken as the reference value V0 of the video clip size; the reference bit rate R in the time window B =V0 / T0; Among them, T0 is the duration of the video block.

14. The method for identifying the video bit rate of an encrypted video according to claim 1, characterized in that: Calculate the size of all valid request data in the time window, take the median of the valid request size, and use it as the benchmark value V0 of the video clip size; the benchmark bit rate R in the time window B =V0 / T0; Among them, T0 is the duration of the video block.

15. The method for identifying the video bit rate of an encrypted video according to claim 1, characterized in that: The method for obtaining the video block duration T0 is as follows: observing the captured video transmission data, determining the size of the audio block and the preset audio sampling rate, and estimating the video block duration. Assuming the size of the audio block is H and the audio sampling rate is M, the video block duration T0 = H / M, where H / M is an integer.

16. A method for estimating encrypted video playback indicators, characterized in that: include An actual download integral curve is generated by statistics of captured video transmission data packets; Identify the video bit rate using the video bit rate identification method for encrypted video according to any one of claims 1 to 15; Generate theoretical playback integral curve; Estimate the actual playback integral curve; Calculate various video playback metrics.

17. A method for estimating encrypted video playback index according to claim 16, characterized in that: The specific method of generating the actual download integral curve by statistics of the captured video transmission data packets is as follows: in chronological order, the sizes of the data packets downloaded before each time point are accumulated, and a graph is drawn with the download time as the horizontal axis and the total size of the data packets downloaded before each time point as the vertical axis to form a download data volume integral curve.

18. A method for estimating encrypted video playback index according to claim 17, characterized in that: The method for drawing the theoretical playback integral curve is: S41, assuming that the video is played without interruption, within the time window, the video playback time is used as the horizontal axis and the reference bit rate within the time window is used as the slope to form a theoretical playback integral curve; the vertical axis is the total amount of data accumulated up to the current video playback time; S42, connecting the theoretical playback integral curve of each time window drawn in S41 in chronological order to form a theoretical playback integral curve of the entire video.

19. The method for estimating encrypted video playback index according to claim 16, characterized in that: The actual playback integral curve is estimated based on the theoretical playback integral curve and the download integral curve.

20. The method for estimating encrypted video playback index according to claim 19, characterized in that: The method to estimate the actual playback integral curve is: S51, set buffering time t buf parameter, S5.

2. Shift the theoretical playback curve to the right and place it below the actual download integral curve, and ensure that at the initial playback and freeze recovery time points, the download curve is larger than the playback curve by the buffering time t buf The corresponding video data volume can obtain the actual playback integral curve.

21. The method for estimating encrypted video playback index according to claim 16, characterized in that: The video playback indicators include video duration, initial buffering duration, freeze time point, freeze recovery time point, and freeze duration.

22. A method for estimating encrypted video playback index according to claim 21, characterized in that: The calculation of the video playback index is performed based on the actual playback curve.

23. A method for estimating encrypted video playback indicators according to claim 21, characterized in that: 1) Video duration: the end time of the theoretical playback curve; 2) Initial buffering time: the initial download time t on the actual playback curve buf The time point of the video data volume; 3) Freeze time point: the time point corresponding to the intersection of the actual playback curve and the actual download curve, that is, the time point when the actual download data volume = the actual playback data volume; 4) Jam recovery time point: that is, the download time t after jamming on the actual playback curve buf Video data volume time point; 5) Freeze duration: The freeze duration of each freeze is calculated by dividing the freeze recovery time point by the freeze time point.

24. A video bit rate identification device for encrypted video, characterized in that: include The data packet aggregation and statistics module is used to aggregate the captured video transmission data packets based on the three granularities of request granularity, extremum granularity, and time window granularity, and determine the download mode; The video bitrate estimation module is used to estimate the video bitrate of videos in different download modes within the time window. The method of aggregating the captured video transmission data packets based on the request granularity, the extremum granularity, and the time window granularity is as follows: S21, aggregate and count the request-sized data packets; S22, Extremum granularity packet aggregation and statistics; S23, identification of download mode; In S21, the specific method for aggregating and counting request-sized data packets is as follows: Statistics are collected on captured video transmission data packets, two uplink download request data packets and downlink data packets during the two uplink download request data packets are defined as a request statistical granularity, and statistical features of the request granularity are obtained; In S22, the Extremum granularity packet aggregation and statistics method is: S221, identifying the minimum value of Request; S222, based on the Request granularity, perform Extremum granularity aggregation; In S23, the method of identifying the download mode is: S231, identifying a valid Request; S232, take a video data segment that meets the total valid request duration greater than or equal to 3×T0 seconds, form a set R, and calculate the download time duty ratio r in the set R d The proportion of valid requests with a request duration greater than T0 r s ; S233, based on the download time duty cycle r d The proportion of valid requests with a request duration greater than T0 r s , determine the download mode; Among them, T0 is the duration of the video block; 1) When the download mode is a periodic download mode, the real-time bit rate is estimated by using a method for estimating the real-time bit rate in the periodic download mode; the periodic download mode includes a smooth download mode and a stuck download mode; 2) When the download mode is non-periodic download mode, first, calculate the size variation coefficient c of all valid requests within the time window v , Among them, σ is the standard deviation of request size, μ is the mean of request size; Then, set the threshold value th cv , when c v >th cv When c v ≤th cv When the real-time bit rate estimation method in the non-periodic download mode is used, the real-time bit rate estimation method is continued to be used to estimate the real-time bit rate; The non-periodic download mode includes a balanced download mode; The method for estimating the real-time bit rate in the periodic download mode is as follows: within the time window, the real-time bit rate is estimated based on the Extremum granularity; The method for estimating the real-time bit rate in the non-periodic download mode is: estimating the real-time bit rate based on the Request granularity within the time window.

25. The video bit rate identification device for encrypted video according to claim 24, characterized in that: The data packet aggregation and statistics module is used for request granularity data packet aggregation and statistics, extreme granularity data packet aggregation and statistics, download mode recognition, and time window granularity data packet aggregation and statistics.

26. The video bit rate identification device for encrypted video according to claim 24, characterized in that: The video bit rate estimation module is used in the periodic download mode to estimate the real-time bit rate based on the Extremum granularity; and is used in the non-periodic download mode to estimate the real-time bit rate based on the Request granularity.

27. An encrypted video playback index estimation device, characterized in that: include A download integral curve generating module, used for generating an actual download integral curve by statistics from captured video transmission data packets; The video bit rate identification device for encrypted video according to any one of claims 24 to 26; A theoretical playback integral curve generation module, used to generate a theoretical playback integral curve according to the video bit rate estimated within the time window; Video playback index estimation module: used to estimate the actual playback integral curve based on the actual download integral curve and the theoretical playback integral curve, and calculate the video playback index.

Images