Judgment Method, Device, Electronic Device and Readable Storage Medium for Playback Stuttering

Abstract

The present application discloses a method, apparatus, electronic device, and readable storage medium for judging playback stuttering. The method includes: obtaining a media object to be transmitted; obtaining a transmission speed for sending the media object to a receiver, where the transmission speed is the amount of data sent per unit time, and the receiver is used to play the received media object; obtaining an average bit rate of the media object; and judging whether stuttering occurs when the receiver plays the received media object according to the transmission speed and the average bit rate. By means of the present application, the problem that the provider of the CDN in the prior art cannot know whether stuttering will occur when playing the multimedia file sent by it is solved, and thus it is possible to predict the stuttering that occurs when the user plays the multimedia file, providing data support for the optimization of multimedia file transmission.

Description

Technical Field

[0001] This application relates to the field of multimedia processing. Specifically, it relates to a method, apparatus, electronic device, and readable storage medium for judging video freezing during playback. Background Art

[0002] A content delivery network (CDN) is a network built on top of the existing network for content distribution. The CDN performs content distribution through technical means such as load balancing and resource scheduling, enabling users to obtain the required content nearby, reducing network congestion, and improving the speed at which users obtain the required content.

[0003] With the development of technology, more and more users obtain multimedia content such as audio or video through the network. Therefore, in current CDNs, the transmission of audio or video content through on-demand and live broadcast is an important traffic distribution scenario in the CDN, especially video on-demand and live broadcast. According to statistics, the current video on-demand traffic accounts for approximately 60% of the total CDN traffic, and the video live broadcast traffic accounts for approximately 10% of the total CDN traffic.

[0004] During the transmission and playback of audio or video, freezing may occur, which is one of the important reasons affecting the user experience. For example, in the scenario of video transmission, the video service provider will use a CDN provider (i.e., the provider of the CDN) to distribute video content, and the end user will watch the video provided by the video service provider after passing through the distribution service provided by the CDN provider. Usually, the video content provider usually uses the video playback freezing rate as an indicator to compare the service quality of various CDN providers. Therefore, improving the performance of video transmission and reducing the video playback freezing rate of users have become the key for each CDN provider to improve service quality.

[0005] However, video playback is a phenomenon that can only be perceived on the user side through a video player, that is, when the user plays a video. The existing methods for solving video freezing are all solved from the user side. For example, when playing a multimedia file, the cache can be increased to cache more files, so that more file download time can be obtained by using the files in the playback cache to reduce video playback freezing, etc. However, for the CDN provider, it cannot obtain the freezing data of multimedia file playback on the user side, nor can it solve the video freezing problem from the user side perspective. Therefore, in the existing technology, the CDN provider cannot judge whether the multimedia file distribution method it provides will cause freezing during user playback, and thus cannot optimize the multimedia distribution. Summary of the Invention

[0006] The embodiments of the present application provide a method, apparatus, electronic device, and readable storage medium for judging playback jitter, so as to at least solve the problem in the prior art that the provider of the CDN cannot know whether there will be jitter when the multimedia file it sends is played.

[0007] According to one aspect of the present application, a method for judging playback jitter is provided, including: obtaining a media object to be transmitted; obtaining the transmission speed of sending the media object to a receiver, where the transmission speed is the amount of data sent per unit time, and the receiver is used to play the received media object; obtaining the average bit rate of the media object; and judging whether there is jitter when the receiver plays the received media object according to the transmission speed and the average bit rate.

[0008] According to another aspect of the present application, a device for judging playback jitter is further provided, including: a first obtaining module, configured to obtain a media object to be transmitted; a second obtaining module, configured to obtain the transmission speed of sending the media object to a receiver, where the transmission speed is the amount of data sent per unit time, and the receiver is used to play the received media object; a third obtaining module, configured to obtain the average bit rate of the media object; and a judging module, configured to judge whether there is jitter when the receiver plays the received media object according to the transmission speed and the average bit rate.

[0009] According to another aspect of the present application, an electronic device is further provided, including a memory and a processor; wherein, the memory is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the above method steps.

[0010] According to another aspect of the present application, a readable storage medium is further provided, on which computer instructions are stored, and when the computer instructions are executed by a processor, the above method steps are implemented.

[0011] In the embodiments of the present application, the following steps are adopted: obtaining a media object to be transmitted; obtaining the transmission speed of sending the media object to a receiver, where the transmission speed is the amount of data sent per unit time, and the receiver is used to play the received media object; obtaining the average bit rate of the media object; and judging whether there is jitter when the receiver plays the received media object according to the transmission speed and the average bit rate. By means of the present application, the problem in the prior art that the provider of the CDN cannot know whether there will be jitter when the multimedia file it sends is played is solved, and thus it is possible to predict the jitter that occurs when a user plays a multimedia file, providing data support for optimizing the transmission of the multimedia file. Description of the Drawings

[0012] The accompanying drawings, which form a part of this application, are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0013] Figure 1 is a flowchart of a method for judging video playback stuttering according to an embodiment of this application;

[0014] Figure 2 is a schematic diagram for predicting stuttering based on TCP layer data packets according to an embodiment of this application; and,

[0015] Figure 3 is a schematic diagram for comparing stuttering prediction results of different video transmission algorithms according to an embodiment of this application. Detailed implementation manners

[0016] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The following will refer to the accompanying drawings and combine with the embodiments to detail this application.

[0017] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0018] The multimedia content provider is used to provide media objects (for example, audio files or video files) to users, and users play these media objects through the acquisition methods provided by the multimedia content provider. For example, if the multimedia content provider provides a live broadcast service, users can browse the live broadcast room through the application or software provided by the multimedia content provider, and then use the multimedia player in the application or software to play the media object. Another example is that if the multimedia content provider provides an on-demand service, users can access the website provided by the multimedia content provider through a browser, and then play the media object through the player in the browser. It should be noted that since both the user's behavior of browsing multimedia and the user's behavior of playing multimedia are carried out through the services provided by the multimedia content provider, the multimedia content provider can collect the data of the player when playing the video and judge whether the video playback will stutter based on these data. However, for the CDN provider, it is only responsible for the distribution of multimedia content. It cannot collect data from the perspective of the player, and the multimedia content provider will not actively provide the collected data to the CDN provider, which results in the CDN provider being unable to judge whether the way it distributes multimedia files will cause stuttering during playback, thus affecting the service quality of the CDN.

[0019] There are various reasons for the stuttering of multimedia file playback. For example, the stuttering during playback caused by the slow decoding of the media object by the player is due to the player itself or video codec, and this kind of stuttering requires optimization of the player. Therefore, the stuttering that can be predicted in the following implementation does not include the stuttering caused by the player itself. Another kind of stuttering in multimedia file playback is caused by file transmission. Taking the example of playing a video file in an on-demand manner, generally, the file size of a video file is relatively large, so the video file cannot be completely sent to the user for playback in a short time. If waiting for the entire video file to be transmitted before playing, although there will be no stuttering during playback, the user needs to wait for a long time to view the video. To solve this problem, the video file is transmitted in chunks, and the player can start playing after receiving a part of the video file, and can continue to receive the video file during playback. If the video reception speed is greater than the playback speed, then there will be no stuttering in the video during playback due to video transmission.

[0020] For the CDN provider, it provides the distribution of multimedia files, that is, the transmission speed of sending multimedia files (which can also be called the sending speed) can be obtained by the CDN. Considering this, it is possible to predict whether stuttering will occur based on the sending speed, which is similar to the principle of judging whether stuttering will occur on the player side: in the player, if the reception speed is less than the playback speed, after playing all the caches, the player will stop playing to wait for the reception of subsequent files, thus causing stuttering; when the CDN sends multimedia files, if the amount of data sent per unit time is less than the amount of data played by the player per unit time, then stuttering will definitely occur in the player.

[0021] Based on the above principle, a method for judging playback stuttering is provided in the following embodiments. Figure 1 It is a flowchart of the method for judging playback stuttering according to the embodiments of the present application, as Figure 1 shown. The steps included in the Figure 1 method shown below will be described.

[0022] Step S102, obtain the media object to be transmitted. For example, the media object may include an audio file or a video file.

[0023] In this step, as the acquirer of the media object, the receiver first sends a request. After receiving the request, the sender transmits the chunked media object (e.g., a video file or an audio file) to the receiver according to the request. Therefore, the media object to be transmitted in this step can be understood as the chunked video file or audio file sent in response to the request after the receiver sends the request. That is, the media object to be transmitted can be a part of a complete video file or a complete audio file, and the complete video file or complete audio file can be sent to the receiver in multiple times. The content sent each time can be understood as the media object to be transmitted in this step.

[0024] Step S104: Obtain the transmission speed (or also referred to as the sending speed) of sending the media object to the receiver, where the transmission speed is the amount of data sent per unit time, and the receiver is used to play the received media object.

[0025] During the process of data transmission over the network, the sender sends data to the receiver. The speed at which the sender sends is the sending speed, and the speed at which the receiver receives is the receiving speed. If the network between the sender and the receiver is ideal and does not generate any packet loss or network latency, then the sending speed is the same as the receiving speed. However, such an ideal network generally does not exist. Therefore, under normal circumstances, the receiving speed is less than the sending speed. The transmission speed of the sender is used in this step. On the one hand, it is because the CDN provider can obtain this speed. On the other hand, if the sending speed cannot even meet the playback requirements, it will inevitably lead to playback stuttering.

[0026] Step S106: Obtain the average bitrate of the media object.

[0027] The average bitrate is involved in this step. Before introducing the average bitrate, the bitrate is first explained. The media object can include a video file or an audio file. Both the video file and the audio file are time-related files. The following takes a video file as an example for explanation. The bitrate can be understood as the amount of data consumed per second when the video file is played at normal speed at a speed of 1x.

[0028] The bitrate can be calculated based on the resolution and frame rate of the video. For example, for a video file with a 4K resolution, the resolution of each frame is 4096*2160 or 3840*2160. The following calculation is based on a resolution of 3840*2160. A resolution of 3860*2160 means there are 3860*2160 pixel points, each pixel point has 3 sub-pixels (red, blue, green), and each sub-pixel occupies 8 bits (bit). Then the data volume of one frame is 3840*2160*8*3 bits. If the frame rate is 60 frames per second (fps), then 60 frames will be played within 1 second, that is, the data volume consumed by the played images within this second is 3840*2160*8*3*60≈12 Gbps; and this is just the image. If the audio track is considered to account for about 1 / 10, the total is 13 G bits per second (bps), that is, the bitrate within this 1 second is 13 G bits per second. It should be noted that since the content of each frame of the image is different, the size of each frame may be different, that is, the data volume played per 1 second may also be different, that is, the bitrate is constantly changing. Therefore, if the bitrate is used for judgment, it is necessary to calculate the bitrate every 1 second and perform the comparison operation in step S108, which will consume relatively large computing resources.

[0029] Considering the problem that using the bitrate will consume relatively large computing resources, the average bitrate is adopted in this step. The average bitrate is the average value of the bitrates of the video file. In an alternative embodiment, the average bitrate can be calculated based on the size and duration of the video file. For example, if the duration of a video file is N seconds and the size of the video file is M bits, then the average bitrate of the video is M bits / N seconds.

[0030] Step S108, determine whether the receiver experiences stuttering when playing the received media object according to the transmission speed and the average bitrate.

[0031] In the above steps, the transmission speed when sending a file to the receiver is used as the basis for judging lag. This transmission speed can be obtained by the sender. For the bitrate of the file sent out, it can also be obtained by the sender. Therefore, as the sender of the audio file or video file, the CDN provider can obtain the sending speed and average bitrate for judging video lag, and then judge whether the receiver will experience lag during playback based on the sending speed and average bitrate. Through the above steps, the CDN provider does not need data from the player and can predict whether the receiver will experience lag during playback based on the data it can obtain as the sender, thus solving the problem in the prior art that the CDN provider cannot know whether the multimedia file it sends will experience lag during playback. Furthermore, it can predict the lag that users will experience during multimedia file playback, providing data support for the optimization of multimedia file transmission.

[0032] In the above steps, it is necessary to predict whether the receiver will experience lag when playing the received media object based on the sending speed and average bitrate. The calculation method of the average bitrate has been described in step S106. For the calculation of the sending speed, multiple calculation methods can be used. For example, timing can start when starting to send the media object, and then stop timing after sending is completed to obtain the transmission time, and then calculate the transmission speed (i.e., the sending speed) based on the size of the media object to be transmitted and the transmission time. It should be noted that in step S102 above, it is described that the media object to be transmitted can be a part of a complete video file or a complete audio file. Since the media object to be transmitted may be a part of a complete file, it may not be possible to directly obtain the size of the media object to be transmitted through the file attributes of the media object to be transmitted. In this case, as another optional method, the data volume sent by the sender can be used as the basis for calculating the transmission speed, that is, obtaining the transmission speed of sending the media object to the receiver can include the following steps: obtaining the data volume transmitted to the receiver when sending the media object to the receiver; obtaining the transmission time used to send the media object to the receiver; calculating the transmission speed based on the data volume transmitted to the receiver and the transmission time. In this alternative embodiment, the data volume sent to the receiver is used to calculate the sending speed, and this data volume can be obtained by the sender. For example, the process or thread responsible for sending the media object can be monitored, and the data volume sent by this process or thread can be used as the data volume sent to the receiver; or, the data volume of the network interface used to send the media object can also be monitored, and the data volume sent through this network interface can be used as the data volume sent to the receiver.

[0033] Considering that both audio files and video files are sent out after being constructed into data packets according to the network transmission protocol, in order to obtain a more accurate amount of data transmitted to the receiving party, the amount of data sent to the receiving party can be obtained according to the nature of the network transmission protocol. In the CDN network, in order to ensure that the sent data packets can be received by the receiving party, the Transmission Control Protocol (TCP for short) is usually used to send data packets. The following is an introduction to TCP.

[0034] The Internet consists of a set of protocols, which can be divided into four layers: the data link layer, the network layer, the transport layer, and the application layer. Among them, the data link layer is used to implement the network driver of the network card interface to handle the transmission of data on physical media such as Ethernet cables. Different electrical characteristics of different physical networks are hidden in the data link layer, providing a unified interface for the upper-layer protocols. For example, in Ethernet, the Ethernet protocol is used in the data link layer. Routers are used in the network to connect dispersed hosts or local area networks. That is, the two communicating hosts are generally not directly connected but are connected through multiple intermediate nodes (routers), thus forming a network topology connection. Therefore, the network layer is used to determine the communication path between two hosts. That is, the network layer hides the details of the network topology connection from the upper-layer protocols, making it seem to the transport layer that the communicating parties are directly connected. The core protocol used in the network layer is the Internet Protocol (IP for short). Therefore, the network layer is also called the IP layer. The role of the transport layer is to provide end-to-end communication for application programs, that is, to hide the details of data packet jumps from application programs, and to be responsible for the sending and receiving of data packets, link timeout reconnection, etc. The transport layer (also known as the TCP layer) mainly uses the TCP protocol and the User Datagram Protocol (UDP for short). The TCP protocol provides reliable, connection-oriented, and stream-based services for application programs, and uses methods such as timeout retransmission and data confirmation to ensure that data packets are correctly sent to the receiving end. The TCP protocol is mainly used in the CDN; the UDP protocol is the opposite of the TCP protocol. It provides unreliable, connectionless, and datagram-based services for application programs. The application layer is responsible for handling specific operation logics, such as file transfer, network management, etc.

[0035] The TCP protocol is the upper-layer protocol of the Ethernet protocol and the IP protocol, and also the lower-layer protocol of the application-layer protocol. In the Ethernet protocol, it is stipulated how electronic signals form data packets. The IP protocol can connect multiple local area networks. The IP protocol defines a set of its own address rules, called IP addresses. It realizes the routing function and allows host A in a local area network to send messages to host B in another local area network. The IP protocol is just an address protocol and does not guarantee the integrity of data packets. If a router loses a packet, it is necessary to find out which packet is lost and how to resend this packet. This depends on the TCP protocol. The role of the TCP protocol is to ensure the integrity and reliability of data communication and prevent packet loss.

[0036] The size of an Ethernet data packet is fixed. Initially, it was 1518 bytes and later increased to 1522 bytes. Among them, 1500 bytes is the payload and 22 bytes is the head information. The IP data packet is inside the payload of the Ethernet data packet and it also has its own head information, which requires at least 20 bytes. So the maximum payload of the IP data packet is 1480 bytes. The TCP data packet (also called the TCP-layer data message) is inside the payload of the IP data packet. Its head information also requires at least 20 bytes. Therefore, the maximum payload of the TCP data packet is 1480 - 20 = 1460 bytes. Since the IP and TCP protocols often have additional head information, the actual TCP payload is about 1400 bytes and the size of each TCP-layer data message can be obtained. If a TCP-layer data message is 1400 bytes, then when sending a large amount of data at one time, it must be divided into multiple messages. For example, a 10MB file needs to send more than 7100 messages. When sending, the TCP protocol numbers each TCP-layer data message, and this number is called the sequence number (abbreviated as SEQ) so that the receiving party can restore it in order. This sequence number is also the sequence number of the data packet after the TCP-layer data message is packed into a data packet.

[0037] Considering the characteristics of TCP layer data packets, in an alternative embodiment, the characteristics of TCP layer data packets can be utilized to obtain the amount of data transmitted to the receiving party. That is, in this alternative embodiment, obtaining the amount of data transmitted to the receiving party when the media object is sent to the receiving party may include the following steps: obtaining the starting sequence number and the ending sequence number of the TCP layer data packets when the media object is sent to the receiving party, obtaining the number of TCP layer packets or data packets based on the starting sequence number and the ending sequence number (since one data packet carries one TCP layer packet, the number of data packets is the same as the number of TCP layer packets), and obtaining the amount of data transmitted to the receiving party based on the number and size of the TCP layer packets or data packets; and / or, obtaining the transmission time taken to send the media object to the receiving party may include the following steps: obtaining the first time when the data packet carrying the starting sequence number (which can also be referred to as the first data packet) is sent and the second time when the data packet carrying the ending sequence number (which can also be referred to as the last data packet) is sent to the receiving party, and obtaining the transmission time taken to send the media object to the receiving party based on the difference between the second time and the first time; wherein, the data packets of the media object are data packets formed by packing the TCP layer packets and capable of being transmitted in the network. For example, the time to send the first data packet is T1, and the time to send the 10th data packet is T2. It should be noted that at time T2, the 10th data packet has not been transmitted to the receiving party through the network yet. Therefore, the time obtained using T2 - T1 does not include the transmission time of the 10th data packet. The transmission time of one data packet is relatively short. If the amount of data transmitted is the amount of 1000 data packets, and the transmission time is calculated using the time to send the 1000th data packet minus the time to send the first data packet at this time, although there will be an error, the error is not very large. In this example, 10 data packets are sent. If T2 is directly used for calculation, the error in the calculated transmission speed will be relatively large. In an alternative embodiment, the second time when the last data packet is sent to the receiving party (i.e., the time when the receiving party receives the 10th data packet) can be used to calculate the transmission time, which will make the calculation of the transmission time more accurate. In this alternative embodiment, the TCP protocol data packets can be used to obtain the transmission time and the amount of data transmitted to the receiving party, so that the calculated transmission speed of sending the media object to the receiving party is more accurate.

[0038] When using the TCP protocol, since the service provided by the TCP protocol is reliable, both parties communicating using the TCP protocol must first establish a TCP connection, and the receiving party will also confirm the received data packets, that is, the receiving party will send a confirmation message to the sending party after receiving the data packets. For the sending party, after receiving the confirmation message sent by the receiving party, it can determine that the data packet corresponding to the sequence number in the confirmation message has been received by the receiving party according to the sequence number carried in the confirmation message. In an alternative embodiment, the sending party can calculate the amount of data transmitted and the transmission time based on the received confirmation message. That is, in this alternative embodiment, obtaining the amount of data transmitted to the receiving party and the transmission time when the media object is sent to the receiving party may include the following steps: receiving a first confirmation message from the receiving party, where the first confirmation message is used to confirm that the receiving party has received the data packet carrying the first sequence number; receiving a second confirmation message from the receiving party, where the confirmation message is used to confirm that the receiving party has received the data packet carrying the second sequence number; taking the first sequence number as the starting sequence number and the second sequence number as the ending sequence number, and obtaining the amount of data transmitted to the receiving party according to the number and size of the TCP layer packets or data packets between the first sequence number and the second sequence number; taking the time difference between the second time and the first time as the transmission time, where the second time = the time of receiving the second confirmation message - (the time of receiving the second confirmation message - the time of sending the data packet carrying the second sequence number) / 2, and the first time is the time of sending the data packet carrying the first sequence number. For example, after sending the data packet with sequence number 1, receiving the confirmation message for the data packet with sequence number 1 (the first data packet) indicates that the data packet has been received by the receiving party; after sending the data packet with sequence number 10 (the tenth data packet), receiving the confirmation message for the data packet with sequence number 10 indicates that the data packet has also been received. The time of sending the first data packet is T1, the time of sending the tenth data packet is T2, and the time of receiving the confirmation message for the tenth data packet is T3. Between T2 and T3, the tenth data packet is sent to the receiving party, and at the same time, the confirmation message for the tenth data packet is sent to the sending party. Generally, the transmission time from the sending party to the receiving party and from the receiving party to the sending party is the same. Then (T3 - T2) / 2 is the transmission time of the tenth data packet from the sending party to the receiving party. Therefore, the transmission time = (T3 - (T3 - T2) / 2) - T1, where (T3 - (T3 - T2) / 2) is the second time and T1 is the first time. After obtaining the transmission time, the transmission speed can be calculated.It should be noted that if the transmission speed is calculated every time a confirmation message is received, it will consume a large amount of resources for the sender, affect the transmission speed of the sender, and is also unnecessary. Therefore, the transmission speed can be calculated at intervals of a predetermined number of data packets or at intervals of a predetermined time duration, that is, there is a predetermined number of data packets between the data packet carrying the second sequence number and the data packet carrying the first sequence number, or there is a predetermined time duration between the time of sending the data packet carrying the second sequence number and the time of sending the data packet carrying the first sequence number. For example, after receiving the confirmation message for the data packet with the sequence number 1001, calculate the transmission time for sending the data packets with sequence numbers from 1 to 1000, and then calculate the transmission speed every 1000 data packets; for another example, calculate the transmission speed every 60 seconds, etc.

[0039] For the TCP protocol, after establishing a TCP connection, some necessary data structures are saved in the kernel of the operating system to maintain the connection, such as the state of the connection, the attribute information of the data packets, etc. When the communication ends, both parties must close the connection to release these data saved in the kernel. Therefore, when obtaining the time of sending the data packet and the sequence number of the data packet at the TCP layer, it can be collected from the kernel of the sender's operating system, that is, collect the starting sequence number and the ending sequence number in the kernel of the sender's operating system, and / or collect the first time and the second time in the operating system kernel.

[0040] Figure 2 is a schematic diagram of predicting lags based on TCP-layer data packets according to an embodiment of the present application, as Figure 2As shown in the figure, the function of predicting lags based on TCP layer packet data is divided into three modules: a data collection module, a video analysis module, and a lag prediction module. Among them, the data collection module is embedded in the operating system kernel of the CDN server. During the transmission process, it collects the size of the transmitted file and the transmission time of the file. Among them, the size of the transmitted file is obtained by calculating the difference between the starting sequence number and the ending sequence number of the data packets transmitted at the TCP layer. The transmission time of the file is obtained by calculating the difference between the time of sending the first data packet and the time of sending the last data packet. After obtaining the above data, the data collection module stores the data for subsequent use by the lag prediction model. Through this collection method, the starting sequence number, ending sequence number, first time, and second time can be obtained more accurately, so that the transmission speed can be accurately calculated. In addition, the method of collecting data through the kernel can also improve the accuracy of data acquisition. The video analysis module is used to calculate and analyze the average bit rate of the video. For example, the average bit rate is calculated based on information such as resolution and frame rate. Through the video analysis module and the data collection module, the transmission speed and average bit rate of sending the media object to the receiving party can be obtained, and then it can be determined whether there is a lag based on the transmission speed and average bit rate. The lag prediction module can perform lag prediction based on the information provided by the above two modules.

[0041] In an alternative embodiment, it can be considered that if the average bit rate is greater than the transmission speed, it can be determined that there will be a lag when playing at the receiving party. That is Figure 2 the lag prediction module in can perform lag prediction based on the information provided by the above two modules. First, calculate the average transmission speed, and obtain the transmission speed by dividing the size of the transmitted file by the transmission time. Then compare the size of the video average bit rate with the transmission speed. If the video average bit rate > transmission speed, output lag. For the case where the video average bit rate is less than or equal to the transmission speed, it can directly output no lag.

[0042] In the above embodiments, it is pointed out that for the case where the average video bitrate is less than or equal to the transmission speed, it can be directly output without stuttering. Or, as another alternative embodiment, considering that the TCP protocol is used for transmission in the CDN, in order to ensure that all the sent data packets are received by the receiving party, when packet loss occurs due to network reasons, the lost data packets will be resent by the sending party. If the network transmission condition is good, the number of lost packets will not be large. If the network condition is bad, a relatively large number of lost packets will occur. In the above embodiments, the sending party uses the amount of data sent to measure the sending speed. If it is considered that the amount of data sent includes the retransmitted data packets, the number of data packets sent by the sending party is greater than the number of data packets received by the receiving party. In this case, the retransmitted data packets in the data packets to be sent can be deleted, so that the sending speed can be calculated according to the amount of data of the data packets actually received by the receiving party. That is, when the average bitrate is less than or equal to the transmission speed, obtain the total amount of data transmitted to the receiving party when sending the media object and the amount of retransmitted data, where the amount of retransmitted data is generated by retransmitting the amount of data not received by the receiving party; recalculate the transmission speed according to the difference obtained by subtracting the amount of retransmitted data from the total amount of data; when the average bitrate is greater than the recalculated transmission speed, determine that the receiving party will stutter during playback, and when the average bitrate is less than or equal to the recalculated transmission speed, determine that the receiving party will not stutter during playback. By excluding the discarded data packets in this way and recalculating the sending speed, the calculated sending speed is close to the receiving speed of the receiving party, so that the predicted stuttering will be more accurate.

[0043] For example, the average bitrate of the video file to be sent is 900 Kbps. If the retransmitted data packets are not considered, the calculated transmission speed is 1000 Kbps. At this time, the transmission speed is greater than the average bitrate of the file, and it can be considered that the receiving party will not stutter during playback. If it is found that the retransmission rate of the data packets is higher than the pre-configured threshold, the retransmitted data packets need to be considered to recalculate the transmission speed. At this time, the obtained retransmission rate of the data packets is 20%, that is, among the 100 data packets sent before, the receiving party only received 80. Therefore, the recalculated transmission speed is 800 Kbps, and this recalculated transmission speed is less than the average bitrate of the video file, and it is determined that the receiving party will stutter during playback.

[0044] Through the above method, it can be determined whether the receiving party stutters, and then the percentage of stuttering of different data transmission algorithms within a predetermined time can be counted, so that the advantages and disadvantages of different data transmission algorithms can be compared. Figure 3Schematic diagram for comparing stuttering prediction results of different video transmission algorithms according to embodiments of the present application. As shown in Figure 3 , the stuttering percentage is used as an indicator for evaluating video transmission quality in CDN video transmission services. In Figure 3 , different video transmission algorithms are configured according to whether the last digit of the IP address of the client receiving the video file is odd or even. The superiority and inferiority of the two algorithms can be compared using the predicted stuttering results. As the receiving client, after sending a request, it determines whether the last digit of the IP address of the client from which the request originated is odd or even. If it is odd, it uses the first transmission algorithm to send the video file or audio file to the client, and after sending, it determines whether this sending will cause stuttering during the client's playback. If it is even, it uses the second transmission algorithm to send the video file to the client and determines whether this sending will cause stuttering during the client's playback. Then, the number of requests and the number of predicted stuttering occurrences are counted, and the percentage of the number of stuttering occurrences and the number of requests per day is calculated as an indicator for measuring the superiority and inferiority of the first transmission algorithm and the second transmission algorithm. After the receiving party sends a request, in response to the request, the segmented video file or audio file to be sent, that is, the media object to be transmitted can be a part of a complete video file or a complete audio file. The complete video file or complete audio file can be sent to the receiving party in multiple times, and the content sent each time can be understood as the media object to be transmitted in this step. In Figure 3 , the stuttering percentage predicted by the first transmission algorithm corresponds to predicted stuttering_odd, and the stuttering percentage predicted by the second transmission algorithm corresponds to predicted stuttering_even. Judging from the trend shown in Figure 3 , the predicted stuttering percentage increases with the date, but the second transmission algorithm is better than the first transmission algorithm.

[0045] In addition to evaluating the superiority and inferiority of different transmission algorithms, the predicted stuttering can also be adjusted in the CDN based on the predicted stuttering to eliminate stuttering as much as possible. In an alternative embodiment, after determining that the receiving party will experience stuttering during playback, the method further includes: adjusting the bitstream of the media object requested by the receiving party and sending the media object with the adjusted bitstream to the receiving party.

[0046] For example, if the average bitrate of a video file to be sent is 900 Kbps and the calculated transmission speed is 800 Kbps, it is determined that the receiver will experience stuttering during playback. At this time, if a request to continue transmitting the video file is received from the receiver, a file with a lower average bitrate corresponding to the video file, such as 700 Kbps, can be obtained, and then the file with the adjusted average bitrate is sent to the receiver. On the other hand, if the calculated transmission speed is greater than the average bitrate, a file with a higher average bitrate can be obtained at this time. If the average bitrate of the file with the higher average bitrate is still less than the calculated transmission speed, a file with a higher average bitrate (i.e., a media object) can be transmitted to the receiver. Through this example, the average bitrate of the sent file can be flexibly adjusted to reduce stuttering during playback on the one hand and improve the picture playback quality to a certain extent on the other hand.

[0047] In another alternative embodiment, a threshold can be set. This threshold serves as a reference for adjusting the average bitrate. If the average bitrate of the video file is greater than the sum of the transmission speed and the threshold, the average bitrate of the transmitted video file is reduced. If the average bitrate of the video file is less than the difference between the transmission speed and the threshold, the average bitrate of the transmitted video file is increased. If the average bitrate of the video file is greater than or equal to the difference between the transmission speed and the threshold and less than or equal to the sum of the transmission speed and the threshold, the average bitrate of the transmitted video file is not adjusted. The threshold can be pre-configured. If real stuttering data of the client can be obtained, the threshold can be adjusted according to the real stuttering data of the client to reduce the occurrence of stuttering.

[0048] For another example, computing resources and network resources can also be added to the sender to increase the sending speed of the sender, which can also reduce the occurrence of stuttering.

[0049] It should be noted that the above-described embodiment described by taking a video file as an example is also applicable to audio files.

[0050] Through the above embodiments, the problem that the provider of the CDN in the prior art cannot know whether there will be stuttering during the playback of the multimedia file it sends is solved. Furthermore, it is possible to predict the stuttering that occurs when the user plays the multimedia file, providing data support for the optimization of multimedia file transmission. In addition, in the CDN scenario, the results obtained through the above stuttering prediction can assist in optimizing the transmission algorithm of the CDN and improving the service quality of the CDN.

[0051] In this embodiment, an electronic device is provided, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method in the above embodiments.

[0052] The above program can run in a processor or can also be stored in a memory (or referred to as a computer-readable medium). A computer-readable medium includes permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0053] These computer programs can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate computer-implemented processing. Thus, the instructions executed on the computer or other programmable device provide for implementing the steps for the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 The steps for the functions specified in one block or multiple blocks can be implemented by different modules corresponding to different steps.

[0054] In one embodiment, such a device is provided. The device is called a device for judging playback jitter and includes: a first acquisition module for acquiring a media object to be transmitted; a second acquisition module for acquiring the transmission speed of sending the media object to a receiving party, where the transmission speed is the amount of data sent per unit time, and the receiving party is used to play the received media object; a third acquisition module for acquiring the average bit rate of the media object; and a judgment module for judging whether the receiving party has jitter when playing the received media object according to the transmission speed and the average bit rate.

[0055] The system or device is used to implement the functions of the method in the above embodiment. Each module in the system or device corresponds to each step in the method, and those that have been described in the method will not be elaborated here.

[0056] For example, the second obtaining module is configured to obtain the amount of data transmitted to the recipient when the media object is sent to the recipient; obtain the transmission time taken to send the media object to the recipient; and calculate the transmission speed based on the amount of data transmitted to the recipient and the transmission time.

[0057] Optionally, the second obtaining module is configured to obtain the starting sequence number and the ending sequence number of the Transmission Control Protocol (TCP) layer data packets when the media object is sent to the recipient, obtain the number of TCP layer packets or data packets based on the starting sequence number and the ending sequence number, and obtain the amount of data transmitted to the recipient based on the number and size of the TCP layer packets or data packets; and / or, obtain the first time when a data packet carrying the starting sequence number is sent and the second time when a data packet carrying the ending sequence number is sent to the recipient, and obtain the transmission time taken to send the media object to the recipient based on the difference between the second time and the first time; wherein, the data packets of the media object are data packets formed by packing the TCP layer packets and capable of being transmitted in the network.

[0058] Optionally, the second obtaining module is configured to receive a first acknowledgment message from the recipient, where the first acknowledgment message is used to confirm that the recipient has received a data packet carrying a first sequence number; receive a second acknowledgment message from the recipient, where the acknowledgment message is used to confirm that the recipient has received a data packet carrying a second sequence number; use the first sequence number as the starting sequence number and the second sequence number as the ending sequence number, and obtain the amount of data transmitted to the recipient based on the number and size of the TCP layer packets or data packets between the first sequence number and the second sequence number; use the time difference between the second time and the first time as the transmission time, where the second time = the time when the second acknowledgment message is received - (the time when the second acknowledgment message is received - the time when the data packet carrying the second sequence number is sent) / 2, and the first time is the time when the data packet carrying the first sequence number is sent.

[0059] For another example, the determination module is configured to determine that the receiver will experience buffering when the average bitrate is greater than the transmission speed; the determination module is configured to determine that the receiver will not experience buffering when the average bitrate is less than or equal to the transmission speed; or, when the average bitrate is less than or equal to the transmission speed, obtain the total amount of data transmitted to the receiver when sending the media object and the amount of retransmitted data, where the retransmitted data is generated by retransmitting the amount of data not received by the receiver; recalculate the transmission speed according to the difference obtained by subtracting the retransmitted data amount from the total data amount; determine that the receiver will experience buffering when the average bitrate is greater than the recalculated transmission speed, and determine that the receiver will not experience buffering when the average bitrate is less than or equal to the recalculated transmission speed.

[0060] For another example, the above device may further include: an adjustment module, configured to adjust the bitstream of the media object requested by the receiver after determining that the receiver will experience buffering during playback, and send the media object with the adjusted bitstream to the receiver.

[0061] Through the above embodiments, based on the detailed data of the video file or audio file collected by the data collection module in the kernel during the transmission process, the calculated data transmission speed is more accurate; by comparing the data transmission speed with the average video bitrate to predict whether the client will experience buffering, the accuracy of buffering prediction is improved, enabling comparable key metrics among video transmission optimization algorithms and improving the optimization efficiency.

[0062] The above are only the embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A method for judging playback jitter, applied to a sender, where the sender is a content distribution network provider, including: Obtain a media object to be transmitted; Obtain the transmission speed of sending the media object to a receiver, where the transmission speed is the amount of data sent by the sender per unit time; Obtain the average bitrate of the media object; the average bitrate is calculated by the sender based on the size and duration of the media object; When the average bitrate is less than or equal to the transmission speed, obtain the total amount of data transmitted to the receiver when sending the media object and the amount of retransmitted data, where the amount of retransmitted data is generated by retransmitting the data not received by the receiver; recalculate the transmission speed according to the difference obtained by subtracting the amount of retransmitted data from the total amount of data; judge whether the receiver has jitter when playing the received media object according to the recalculated transmission speed and the average bitrate.

2. The method according to claim 1, wherein, Obtaining the transmission speed of sending the media object to the receiver includes: Obtain the amount of data transmitted to the receiver when sending the media object to the receiver; Obtain the transmission time used to send the media object to the receiver; Calculate the transmission speed according to the amount of data transmitted to the receiver and the transmission time.

3. The method according to claim 2, wherein Obtaining the amount of data transmitted to the receiver when sending the media object to the receiver includes: obtaining the start sequence number and end sequence number of the transmission control protocol layer data packet when sending the media object to the receiver, obtaining the number of the transmission control protocol layer data packets or data packets according to the start sequence number and the end sequence number, and obtaining the amount of data transmitted to the receiver according to the number and size of the transmission control protocol layer data packets or data packets; wherein, the data packet is a data packet formed by packing the transmission control protocol layer data packet and capable of being transmitted in the network; and / or, Obtaining the transmission time used to send the media object to the receiver includes: obtaining the first time of sending the data packet carrying the start sequence number and the second time of sending the data packet carrying the end sequence number to the receiver, and obtaining the transmission time used to send the media object to the receiver according to the difference between the second time and the first time.

4. The method according to claim 3, wherein, Obtaining the amount of data transmitted to the receiver when sending the media object to the receiver and the transmission time includes: Receive a first acknowledgment message from the receiver, where the first acknowledgment message is used to confirm that the receiver has received the data packet carrying the first sequence number; Receive a second acknowledgment message from the receiver, where the acknowledgment message is used to confirm that the receiver has received the data packet carrying the second sequence number; Using the first serial number as the starting serial number and the second serial number as the ending serial number, obtain the amount of data transmitted to the receiving party based on the number and size of Transmission Control Protocol (TCP) layer data packets or data frames between the first serial number and the second serial number. Using the time difference between the second time and the first time as the transmission time, where the second time = the time of receiving the second acknowledgment message - (the time of receiving the second acknowledgment message - the time of sending the data packet carrying the second serial number) / 2, and the first time is the time of sending the data packet carrying the first serial number.

5. The method according to claim 4, wherein, There is a predetermined number of data packets between the data packet carrying the second serial number and the data packet carrying the first serial number, or there is a predetermined time interval between the time of sending the data packet carrying the second serial number and the time of sending the data packet carrying the first serial number.

6. The method according to any one of claims 3 to 5, wherein Collect the starting serial number and the ending serial number in the operating system kernel of the sender, and / or collect the first time and the second time in the operating system kernel, where the sender is the party that sends the media object to the receiving party.

7. The method according to any one of claims 1 to 5, wherein In the case where the average bitrate is greater than the transmission speed, it is determined that the receiving party will experience buffering during playback.

8. A buffering determination device applied to a sender, where the sender is a content delivery network provider, comprising: A first acquisition module for acquiring a media object to be transmitted; A second acquisition module for acquiring the transmission speed of sending the media object to a receiving party, where the transmission speed is the amount of data sent by the sender per unit time, and the receiving party is used for playing the received media object; A third acquisition module for acquiring the average bitrate of the media object; the average bitrate is calculated by the sender based on the size and duration of the media object; A judgment module for, in the case where the average bitrate is less than or equal to the transmission speed, acquiring the total amount of data transmitted to the receiving party and the amount of retransmitted data when sending the media object, where the amount of retransmitted data is generated by retransmitting the amount of data not received by the receiving party; recalculating the transmission speed based on the difference obtained by subtracting the amount of retransmitted data from the total amount of data; and judging whether the receiving party experiences buffering when playing the received media object based on the recalculated transmission speed and the average bitrate.

9. The device according to claim 8, wherein The second acquisition module is configured to acquire the amount of data transmitted to the receiving party when sending the media object to the receiving party; acquire the transmission time used to send the media object to the receiving party; and calculate the transmission speed based on the amount of data transmitted to the receiving party and the transmission time.

10. The device according to claim 9, wherein The second obtaining module is configured to obtain the starting sequence number and the ending sequence number of the Transmission Control Protocol (TCP) layer data packets when sending the media object to the recipient, obtain the number of the TCP layer data packets or data packets according to the starting sequence number and the ending sequence number, and obtain the amount of data transmitted to the recipient according to the number and size of the TCP layer data packets or data packets; and / or obtain the first time when sending the data packet carrying the starting sequence number and the second time when sending the data packet carrying the ending sequence number to the recipient, and obtain the transmission time used for sending the media object to the recipient according to the difference between the second time and the first time; wherein the data packet is a data packet formed by packing the TCP layer data packets and capable of being transmitted in the network.

11. The apparatus according to claim 10, wherein The second obtaining module is configured to receive a first acknowledgment message from the recipient, where the first acknowledgment message is used to confirm that the recipient has received the data packet carrying the first sequence number; receive a second acknowledgment message from the recipient, where the acknowledgment message is used to confirm that the recipient has received the data packet carrying the second sequence number; use the first sequence number as the starting sequence number and the second sequence number as the ending sequence number, and obtain the amount of data transmitted to the recipient according to the number and size of the TCP layer data packets or data packets between the first sequence number and the second sequence number; use the time difference between the second time and the first time as the transmission time, where the second time = the time of receiving the second acknowledgment message - (the time of receiving the second acknowledgment message - the time of sending the data packet carrying the second sequence number) / 2, and the first time is the time of sending the data packet carrying the first sequence number.

12. The apparatus according to any one of claims 8 to 11, wherein The determining module is configured to determine that the recipient will experience stuttering during playback when the average bit rate is greater than the transmission speed.

13. An electronic device, comprising a memory and a processor; wherein, The memory is configured to store one or more computer instructions, where the one or more computer instructions are executed by the processor to implement the method steps according to any one of claims 1 to 7.

14. A readable storage medium having computer instructions stored thereon, wherein, The computer instructions, when executed by the processor, implement the method steps according to any one of claims 1 to 7.

Images