Method for detecting quality of wireless network based on airplay projection screen

Abstract

The invention discloses a method for detecting the quality of a wireless network based on airplay screen projection. The method comprises the following steps: starting a receiving thread, wherein the receiving thread is used for receiving a tcp data stream returned from a network card; starting a heartbeat detection thread; the heartbeat detection thread is used for providing accurate timing for the system and performing network jitter algorithm operation on a timestamp of a received heartbeat packet; and judging the calculated jitter value, if the jitter value is greater than a set threshold value, reporting that the network is abnormal, and if the jitter value is smaller than the set threshold value, reporting that the network is recovered. According to the invention, through the cooperation of the receiving thread and the heartbeat detection thread, the system can monitor the state of the network in real time and timely discover network fluctuation and abnormal conditions, and when the network jitter value exceeds the set threshold value, the system can rapidly report the network abnormity. And when the network jitter value is recovered to a normal range, the system can report a network recovery state to help a user to know the network condition.

Description

Technical Field

[0001] The present invention belongs to the field of network transmission technology, and specifically is a method for detecting wireless network quality based on airplay screen projection. Background Art

[0002] In today's society, casting the screen of a mobile phone to a TV has become an indispensable function in work or life, but it is easy to experience poor user experience such as freezes and screen distortion when using video projection. There are many reasons for freezes and screen distortion, such as low encoding performance of the mobile phone, abnormal decoding or insufficient performance of the TV end, and the biggest problem may be high network latency or packet loss. Therefore, timely and effective judgment and reminder of the current network reception status can effectively improve the sensory experience during the projection process.

[0003] With the widespread use of mobile screen projection technology, in order to effectively improve the smoothness of the projection effect, mobile phone manufacturers often push videos at a higher frame rate, and high frame rate videos also mean that higher network bandwidth is required. Since mobile phone screen projection uses WiFi as the network connection method for projection, the network environment for projection is relatively complex. When other applications in the network occupy high bandwidth or the mobile phone is far away from the router, the transmission efficiency of the video stream will become unstable. In order to ensure the transmission quality, users can only purchase router hardware with higher bandwidth to achieve better projection effects, and the problem of projection freeze cannot be completely avoided. Summary of the invention

[0004] The purpose of the present invention is to provide a method for detecting the quality of a wireless network based on airplay projection, so as to solve the problem in the prior art proposed in the background technology that when a user projects the screen through a mobile phone, the transmission efficiency is unstable and the projection quality cannot be guaranteed.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A method for detecting wireless network quality based on airplay projection includes the following steps:

[0007] Step S1, start the receiving thread: the receiving thread is used to receive the TCP data stream returned from the network card;

[0008] Step S2, start the heartbeat detection thread; the heartbeat detection thread is used to provide accurate time timing for the system and perform network jitter algorithm calculation on the timestamp of the received heartbeat packet;

[0009] Step S3, calculating the jitter value by network jitter algorithm operation;

[0010] Step S4, judging the jitter value calculated in step S3, if the jitter value is greater than the set threshold, reporting network abnormality, if the jitter value is less than the set threshold, reporting network recovery.

[0011] According to the above technical solution, in step S1, receiving the TCP data stream returned from the network card is specifically:

[0012] First, you need to parse out the packet type, then determine the length and category information of the current packet from the header information of the data packet, and then send the audio, video, and heartbeat data to the corresponding cache respectively for network quality calculation.

[0013] According to the above technical solution, in step S2, the specific time is obtained as follows:

[0014] By calling the system's clock_gettime, you can get the precise time in nanoseconds after the system starts. The following algorithm can be used to convert it into the system's nanosecond time:

[0015] long tickCount=ts.tv_sec*1000+ts.tv_nsec / 1000000

[0016] Where tickCount is the system's precise time in nanoseconds.

[0017] According to the above technical solution, in step S3, the network jitter algorithm operation includes calculating the first initialization time variable, updating the timestamp and calculating the heartbeat jitter.

[0018] According to the above technical solution, the calculation of the first initialization time variable is specifically as follows: first, define and initialize the timestamp structure jitter; and initialize the value of jitter to 0.

[0019] According to the above technical solution, the specific steps for updating the timestamp are as follows: each time a heartbeat packet is received, a MIRROR_TIMESTAMP object needs to be added to the heartbeat detection thread to save the relevant information of the current heartbeat; the TICKCOUNT variable can be used to obtain the current time through the algorithm in step S2, and HEARTBEATID can be directly added by 1 using the global counting variable.

[0020] According to the above technical solution, the heartbeat jitter is calculated as follows:

[0021]

[0022] In the formula, tickcount is the TICKCOUNT variable of the structure object, timestamp is the TIMESTAMP variable of the structure object, n represents the number of heartbeat packet structure buffers, w is the weight value of the heartbeat packets received in the order of reception, and the sum of all buffered w values ​​is equal to 1.

[0023] According to the above technical solution, in step S4, after calculating the latest JITTER value, it is determined whether the JITTER is greater than the set threshold threshhold. When the JITTER value is greater than threshhold, the network abnormal state needs to be reported to apk, and the apk reminds the user that the current network abnormal state causes screen projection to freeze;

[0024] When the JITTER value is less than threshhold, the normal network status needs to be reported to apk. If apk is currently reminding the user of network abnormalities, the current network abnormality reminder information needs to be cleared.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] In the present invention, through the cooperation of the receiving thread and the heartbeat detection thread, the system can monitor the status of the network in real time, promptly discover network fluctuations and abnormal conditions, and when the network jitter value exceeds the set threshold, the system can quickly report the network abnormality so that timely measures can be taken.

[0027] By collecting and analyzing jitter values ​​and their changing trends, the system can provide data support for subsequent network optimization and help identify potential problems and improvement points. By effectively monitoring and managing wireless network quality, the user experience when using AirPlay can be significantly improved, reducing problems such as freezes and delays, and improving fluency. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of the detection method of the present invention;

[0029] Figure 2 This is a schematic diagram of a normal heartbeat;

[0030] Figure 3 This is a schematic diagram of an abnormal heartbeat. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Embodiment 1

[0033] like Figure 1 As shown, a method for detecting wireless network quality based on airplay projection includes the following steps:

[0034] Step S1, start the receiving thread: the receiving thread is used to receive the TCP data stream returned from the network card (a data transmission channel will be established between the mobile phone and the Android TV when casting the screen, and both the sending end and the receiving end send or receive data from the network card) (the data stream here is all the data directly obtained from the network card, including all the data of the entire airplay casting protocol (such as audio data, video data, heartbeat data, control command data, graphic data, etc.);

[0035] Step S2, start the heartbeat detection thread; the heartbeat detection thread is used to provide accurate time timing for the system and perform network jitter algorithm calculation on the timestamp of the received heartbeat packet;

[0036] Step S3, calculating the jitter value by network jitter algorithm operation;

[0037] Step S4, judging the jitter value calculated in step S3, if the jitter value is greater than the set threshold, reporting network abnormality, if the jitter value is less than the set threshold, reporting network recovery.

[0038] In the present invention, through the cooperation of the receiving thread and the heartbeat detection thread, the system can monitor the status of the network in real time, and promptly discover network fluctuations and abnormalities. When the network jitter value exceeds the set threshold, the system can quickly report the network abnormality, so that timely measures can be taken, such as adjusting the network configuration, switching to a more stable network, etc., thereby reducing the impact on user experience. When the network jitter value returns to the normal range, the system can report the status of network recovery, which can help users understand the recovery of the network and enhance users' trust in the system.

[0039] By collecting and analyzing jitter values ​​and their changing trends, the system can provide data support for subsequent network optimization and help identify potential problems and improvement points. By effectively monitoring and managing wireless network quality, the user experience when using AirPlay can be significantly improved, reducing problems such as freezes and delays, and improving fluency.

[0040] Embodiment 2

[0041] This embodiment is a further refinement of the first embodiment.

[0042] In step S1, receiving the TCP data stream returned from the network card is specifically as follows:

[0043] First, you need to parse the packet type Type, then determine the length and category information of the current packet from the header information of the data packet (the category information includes audio, video and heartbeat data), and then send the audio, video and heartbeat data to the corresponding cache respectively for network quality calculation.

[0044] In step S2, the specific time is obtained as follows:

[0045] By calling the system's clock_gettime, you can get the precise time in nanoseconds after the system starts. The following algorithm can be used to convert it into the system's nanosecond time:

[0046] long tickCount=ts.tv_sec*1000+ts.tv_nsec / 1000000

[0047] Where tickCount is the system's precise time in nanoseconds.

[0048] In step S3, the network jitter algorithm operation includes calculating the first initialization time variable, updating the timestamp, and calculating the heartbeat jitter.

[0049] The calculation of the first initialization time variable is as follows: The initialization time variable mainly performs the following operations:

[0050] MIRROR_TIMESTAMP*CURTIMESTAMP; / / Define and initialize timestamp structure;

[0051] int JITTERVALUE = 0; / / Initialize jitter value to 0

[0052] double ws[3] = {0.5, 0.3, 0.2}; / / This weight value is a relatively reasonable weight ratio value found in the current test. Different weight values ​​can be adjusted according to the actual calculation.

[0053] The specific method of updating the timestamp is as follows: each time a heartbeat packet is received, a MIRROR_TIMESTAMP object needs to be added to the heartbeat detection thread to save the relevant information of the current heartbeat; the TICKCOUNT variable can be used to obtain the current time through the algorithm in step S2, and HEARTBEATID directly uses the global count variable plus 1.

[0054] The calculation of heartbeat jitter is as follows:

[0055]

[0056] In the formula, tickcount is the TICKCOUNT variable of the structure object, timestamp is the TIMESTAMP variable of the structure object, n represents the number of heartbeat packet structure buffers, w is the weight value of the heartbeat packets received in the order of reception, and the sum of all buffered w values ​​is equal to 1.

[0057] In step S4, after calculating the latest JITTER value, it is determined whether the JITTER is greater than the set threshold threshhold. When the JITTER value is greater than threshhold, the network abnormality state needs to be reported to apk, and the apk reminds the user that the current network abnormality state causes screen projection freeze;

[0058] When the JITTER value is less than threshhold, the normal network status needs to be reported to apk. If apk is currently reminding the user of network abnormalities, the current network abnormality reminder information needs to be cleared.

[0059] Embodiment 3

[0060] This embodiment provides a specific implementation method.

[0061] Step 1: Start the receiving thread. After the screen projection session is officially started, the TCP data stream will be returned from the network card in the receiving thread. First, you need to parse the packet type Type according to the following structure, and then determine the length, category and other information of the current packet from the header information, and then send the audio, video and heartbeat data to the corresponding cache respectively. The structure definition of the header information is as follows:

[0062] typedef struct_MIRROR_HEAD

[0063] {

[0064] UINT32 Size; / / Package size

[0065] UCHAR Type; / / Package type 0: H264 data 1: spspps information 2: audio data 5: heartbeat packet information

[0066] }MIRROR_HEAD,*PMIRROR_HEAD;

[0067] When receiving data sent back by the network card, first convert the first 5 bytes into the above structure to determine the size and type of the current packet. Here, since we only care about the heartbeat packet information, we only need to filter out the data with Type 5 to obtain the heartbeat packet and timestamp information.

[0068] Step 2, start the heartbeat detection thread. The function of the heartbeat detection thread is to provide accurate time timing and perform jitter algorithm calculations on the timestamp of the received heartbeat packet. The accurate time timing is to call the system's clock_gettime to obtain the precise time in nanoseconds after the system is started. This method can obtain the variable ts with a timespec structure, which can be converted into the system's nanosecond time through the following algorithm. The final tickCount is the precise time of the system in nanoseconds.

[0069] long tickCount=ts.tv_sec*1000+ts.tv_nsec / 1000000;

[0070] Step 3: Network jitter algorithm operation principle description. Since the Apple screen projection protocol has defined that the heartbeat packet is sent once every 1 second, theoretically the screen projection receiver will receive a heartbeat packet data accurately every 1 second. The received heartbeat packet time is as follows: Figure 2 shown.

[0071] However, the real network environment is complicated, and the Apple protocol uses the TCP protocol to transmit data. Since the TCP protocol itself has an error checking mechanism and will not lose packets, the heartbeat packet data we receive is definitely complete, but there may be a lot of jitter in the timing. Often the heartbeat packet time we receive is as follows: Figure 3 shown.

[0072] By comparing the above normal heartbeat and abnormal heartbeat, we can easily see that the time interval between two heartbeat packets of normal heartbeat is accurately received at an interval of 1 second. This situation indicates that the current network situation is good and there is no transmission congestion in the video stream and audio stream. The abnormal heartbeat diagram shows that although the screen projection sender sends a heartbeat packet strictly at an interval of 1 second, the time interval for receiving the heartbeat packet at the receiving end is less than 1 second or greater than 1 second. This indicates that the network situation is not very good, and the transmission of audio and video streams is also blocked. The effect seen by the screen projection receiver is either stuck or jitter. At this time, it is necessary to calculate the jitter value based on the network jitter algorithm to decide whether it is necessary to report to remind the user of network abnormalities or clear network abnormality prompts.

[0073] Step 4: Initialize the time variable for the first time. The following structure is needed to store the heartbeat packet information and system time information when calculating the heartbeat jitter.

[0074]

[0075]

[0076] The above structure is the main structure that needs to create a memory object to store relevant information every time a heartbeat packet is received. The TICKCOUNT variable is the current system time value calculated from the algorithm in step 2 when the heartbeat packet is received, TIMESTAMP is the system timestamp set when the screen projection sender sends it out of the heartbeat packet, HEARTBEATID is the ID number used to describe the current object, JITTER is the jitter value calculated based on TICKCOUNT and TIMESTAMP when the heartbeat packet is received, and PREV is a pointer to the previous MIRROR_TIMESTAMP object used to implement a linked list. When initializing the time variable for the first time, we only need to set the global current timestamp pointer MIRROR_TIMESTAMP*CURTIMESTAMP to NULL.

[0077] Step 5, update the timestamp. Every time a heartbeat packet is received, a MIRROR_TIMESTAMP object needs to be added to the heartbeat detection thread to save the current heartbeat related information. The TICKCOUNT variable can directly retrieve the current time according to the algorithm in step 2. HEARTBEATID can be directly added by 1 to the global count variable. TIMESTAMP needs to be taken from the Apple screen projection protocol. According to the Apple data transmission protocol, the timestamp of the heartbeat packet is from the 8th to the 16th bytes of the HEAD packet. Since the data storage method is big-endian storage, the 8 bytes obtained can be converted as follows to obtain the timestamp of the heartbeat packet, where bb is the 8th to 16th byte data of the HEAD packet.

[0078] uint64_t timestamp=((((uint64_t)bb[7]&0xff)<<56)

[0079] |(((uint64_t)bb[6]&0xff)<<48)

[0080] |(((uint64_t)bb[5]&0xff)<<40)

[0081] |(((uint64_t)bb[4]&0xff)<<32)

[0082] |(((uint64_t)bb[3]&0xff)<<24)

[0083] |(((uint64_t)bb[2]&0xff)<<16)

[0084] |(((uint64_t)bb[1]&0xff)<<8)

[0085] |(((long long int)bb[0]&0xff)<<0));

[0086] Step 6: Calculate the heartbeat jitter. Calculating the JITTER variable value in the MIRROR_TIMESTAMP object is the core algorithm, and its calculation formula is as follows:

[0087]

[0088] Where tickcount is the TICKCOUNT variable of the structure object, timestamp is the TIMESTAMP variable of the structure object, n represents the number of heartbeat packet structure caches, w is the weight value of the heartbeat packets received in the order of reception, and the sum of all cached w values ​​is equal to 1.

[0089] In general, the value of n in the formula is 3, that is, only the values ​​of the latest three heartbeat packet objects are used to calculate the JITTER value of the current latest heartbeat object. Whenever a new heartbeat packet is received, a new MIRROR_TIMESTAMP object value is created to store the latest timestamp information, where the PREV pointer is set to the previous MIRROR_TIMESTAMP object. After setting the PREV pointer, it is necessary to first release the structure objects greater than n according to the value of n in the formula to avoid memory leaks, and then calculate the JITTER value based on the latest n MIRROR_TIMESTAMP objects and store it in the latest MIRROR_TIMESTAMP object. The following is the logic code for calculating JITTER.

[0090]

[0091]

[0092] Step 7, determine the jitter threshold. After calculating the latest JITTER value through the above steps, directly determine whether JITTER is greater than the set threshold threshhold (generally set to 2 based on user experience). When the JITTER value is greater than threshhold, the network abnormality status needs to be reported to apk, and apk will remind the user that the current network abnormality may cause screen projection to freeze. When the JITTER value is less than threshhold, the normal network status needs to be reported to apk. If apk is currently reminding the user of network abnormality, the current network abnormality reminder information needs to be cleared.

[0093] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0094] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting wireless network quality based on airplay projection, characterized in that: The following steps are involved: Step S1, start the receiving thread: the receiving thread is used to receive the TCP data stream returned from the network card; Step S2, start the heartbeat detection thread; the heartbeat detection thread is used to provide accurate time timing for the system and perform network jitter algorithm calculation on the timestamp of the received heartbeat packet; Step S3, calculating the jitter value by network jitter algorithm operation; Step S4, judging the jitter value calculated in step S3, if the jitter value is greater than the set threshold, reporting network abnormality, if the jitter value is less than the set threshold, reporting network recovery.

2. According to claim 1, a method for detecting wireless network quality based on airplay projection is characterized in that: In step S1, receiving the TCP data stream returned from the network card is specifically as follows: First, you need to parse out the packet type, then determine the length and category information of the current packet from the header information of the data packet, and then send the audio, video, and heartbeat data to the corresponding cache respectively for network quality calculation.

3. According to claim 2, a method for detecting wireless network quality based on airplay projection is characterized in that: In step S2, the specific time is obtained as follows: By calling the system's clock_gettime, you can get the precise time in nanoseconds after the system starts. The following algorithm can be used to convert it into the system's nanosecond time: long tickCount=ts.tv_sec*1000+ts.tv_nsec / 1000000 Where tickCount is the system's precise time in nanoseconds.

4. According to claim 3, a method for detecting wireless network quality based on airplay projection is characterized in that: In step S3, the network jitter algorithm operation includes calculating the first initialization time variable, updating the timestamp, and calculating the heartbeat jitter.

5. According to claim 4, a method for detecting wireless network quality based on airplay projection is characterized in that: The calculation of the first initialization time variable is as follows: first, define and initialize the timestamp structure jitter; and initialize the value of jitter to 0.

6. According to claim 5, a method for detecting wireless network quality based on airplay projection is characterized in that: The specific method of updating the timestamp is as follows: each time a heartbeat packet is received, a MIRROR_TIMESTAMP object needs to be added to the heartbeat detection thread to save the relevant information of the current heartbeat; the TICKCOUNT variable can be used to obtain the current time through the algorithm in step S2, and HEARTBEATID directly uses the global count variable plus 1.

7. According to claim 6, a method for detecting wireless network quality based on airplay projection is characterized in that: The calculation of heartbeat jitter is as follows: In the formula, tickcount is the TICKCOUNT variable of the structure object, timestamp is the TIMESTAMP variable of the structure object, n represents the number of heartbeat packet structure buffers, w is the weight value of the heartbeat packets received in the order of reception, and the sum of all buffered w values ​​is equal to 1.

8. According to claim 7, a method for detecting wireless network quality based on airplay projection is characterized in that: In step S4, after calculating the latest JITTER value, it is determined whether the JITTER is greater than the set threshold threshhold. When the JITTER value is greater than threshhold, the network abnormality state needs to be reported to apk, and the apk reminds the user that the current network abnormality state causes screen projection freeze; When the JITTER value is less than threshhold, the normal network status needs to be reported to apk. If apk is currently reminding the user of network abnormalities, the current network abnormality reminder information needs to be cleared.

Images