A data processing method and an electronic device

Abstract

The present application provides a data processing method and an electronic device, relating to the technical field of network transmission. This method can jointly adjust the length of the buffer queue according to the degree of network jitter and service settings, while smoothing the delay jitter without causing performance waste. The method includes: receiving data packets transmitted by a sending end; processing the data packets of a first service based on the actual length k of the buffer queue and the expected length K of the buffer queue; wherein, the expected length K of the buffer queue is the number of data packets expected to be stored in the buffer queue, the expected length K of the buffer queue is associated with the data transmission delay, the service setting delay, and the jitter buffer delay, the data transmission delay is the time required for the sending end to send a data packet until the receiving end receives the corresponding data packet, the service setting delay is associated with the first service and the data transmission delay, and the jitter buffer delay is related to the network state between the receiving end and the sending end and the degree of jitter of the data transmission delay.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of network transmission technologies, and in particular, to a data processing method and an electronic device. Background Art

[0002] During the process of data transmission, delay jitter may occur. Currently, a buffer queue mechanism is generally used to smooth delay jitter. In the buffer queue mechanism, data packets are pre-stored in the buffer queue to maintain the packet delivery period.

[0003] However, in this solution, the length of the buffer queue is relatively fixed. If the network environment and service type change, it may be impossible to smooth the delay jitter due to the too short buffer queue, or performance waste may be caused due to the too long buffer queue. Summary of the Invention

[0004] Embodiments of the present application provide a data processing method and an electronic device, which can jointly adjust the length of the buffer queue according to the network jitter degree and service settings, and while smoothing the delay jitter, will not cause performance waste.

[0005] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a data processing method, which is applied to a receiving end. The receiving end is wirelessly connected to a sending end. The method includes: receiving a session establishment request sent by the sending end, where the session establishment request carries identification information of a first service; in response to the session establishment request, establishing a first session with the sending end, where the first session is used to transmit data packets of the first service; receiving data packets transmitted by the sending end based on the first session; processing data packets of the first service based on the actual length k of the buffer queue and the expected length K of the buffer queue; where the actual length k of the buffer queue is the number of data packets actually stored in the buffer queue, the expected length K of the buffer queue is the number of data packets expected to be stored in the buffer queue, the buffer queue is used to store data packets of the first service, the expected length K of the buffer queue is associated with the data transmission delay, the service setting delay, and the jitter buffer delay. The data transmission delay is the time required for the sending end to send a data packet to the receiving end to receive the corresponding data packet. The service setting delay is associated with the first service and the data transmission delay. The jitter buffer delay is related to the network state between the receiving end and the sending end and the degree of jitter of the data transmission delay.

[0007] That is to say, the embodiments of the present application can obtain the data transmission delay between the receiving end and the sending end, then jointly determine the length of the buffer queue according to the service setting delay, the data transmission delay, and the network state, and finally process the data packets by adjusting the length of the buffer queue, which can smooth the delay caused by the network.

[0008] In an implementation provided in the first aspect, the method further includes: in response to receiving a data packet of a first service, sending a transmission completion frame to a sending end at a preset time interval; wherein, the transmission completion frame carries the number of data packets received by the receiving end within the corresponding time interval, and the time when each data packet is received; receiving a transmission completion acknowledgment frame; wherein, the transmission completion acknowledgment frame is feedback by the sending end in response to the transmission completion frame, and the transmission completion acknowledgment frame carries a waiting duration, the waiting duration is the difference between a first moment and a second moment, the first moment is the moment when the sending end sends a first data packet, the first data packet is the first data packet received by the receiving end within the corresponding time interval, and the second moment is the moment when the sending end receives the transmission completion frame; calculating a round-trip delay based on a third moment and a fourth moment; wherein, the third moment is the moment when the receiving end sends the transmission completion frame, and the fourth moment is the moment when the receiving end receives the transmission completion acknowledgment frame; determining a data transmission delay based on the waiting duration, the time interval, and the round-trip delay. In this way, the receiving end can detect the current transmission delay of the data packets of the first service in real time and accurately. Furthermore, the length of the buffer queue can be calculated more accurately, and the delay jitter can be better smoothed. Moreover, there is no need to add extra data frames to detect the transmission delay, which can reduce power consumption and save power.

[0009] In an implementation provided in the first aspect, the data transmission delay, the waiting duration, the time interval, and the round-trip delay satisfy the formula: S1 = W - D - RTT / 2; wherein, S1 is the data transmission delay, W is the waiting duration, D is the time interval, and RTT is the round-trip delay.

[0010] In an implementation provided in the first aspect, the method further includes: obtaining a first threshold and a second threshold corresponding to the first service, wherein the first threshold is less than the second threshold; determining a service setting delay based on the first threshold, the second threshold corresponding to the first service, and the data transmission delay. That is to say, the service setting delay can be dynamically adjusted according to the data transmission delay.

[0011] In an implementation provided in the first aspect, determining the service setting delay based on the first threshold, the second threshold corresponding to the first service, and the data transmission delay includes: if the data transmission delay is less than the first threshold, determining the service setting delay as the first threshold; if the data transmission delay is greater than or equal to the first threshold, determining the service setting delay as the second threshold. That is to say, when the network state is good, no jitter buffer is introduced, thus no performance waste is caused.

[0012] In an implementation provided in the first aspect, the method further includes: determining a network jitter value according to multiple data transmission delays obtained within a preset duration; obtaining network information and determining a jitter coefficient based on the network information, where the network information is used to indicate the relationship between the frequency points and channels of the receiving end and the sending end; calculating a jitter buffer delay according to the network jitter value and the jitter coefficient.

[0013] In an implementation provided in the first aspect, the method further includes: determining a network jitter value according to multiple data transmission delays obtained within a preset duration; obtaining network information and determining a jitter coefficient based on the network information, where the network information is used to indicate the relationship between the frequency points and channels of the receiving end and the sending end; calculating a first parameter according to the network jitter value and the jitter coefficient.

[0014] If the first parameter is greater than a preset maximum jitter buffer delay value, determining that the jitter buffer delay is the preset maximum jitter buffer delay value, where the preset maximum jitter buffer delay value corresponds to the first service; if the first parameter is less than or equal to the packet sending interval, determining that the jitter buffer delay is 0, where the packet sending interval is the time interval for the sending end to send data packets; if the first parameter is greater than the packet sending interval and less than the preset maximum jitter buffer delay value, determining that the jitter buffer delay is the first parameter.

[0015] In an implementation provided in the first aspect, processing data packets of the first service based on the actual length k of the buffer queue and the expected length K of the buffer queue includes: based on the actual length k of the buffer queue and the expected length K of the buffer queue, adjusting the processing policy of the data packets already cached in the buffer queue until there are K data packets cached in the buffer queue, and then processing the data packets cached in the buffer queue according to the packet sending interval.

[0016] In an implementation provided in the first aspect, adjusting the processing policy of the data packets already cached in the buffer queue based on the actual length k of the buffer queue and the expected length K of the buffer queue includes: when k < K, sending the data packets in the buffer queue according to the packet sending interval and re-sending the lost data packets; when k > K, sending the data packets in the buffer queue according to a target interval or performing packet loss processing on the data packets in the buffer queue, where the target interval is less than the packet sending interval; when k = 0, after caching a data packet in the buffer queue, immediately sending the corresponding data packet. It can be seen that different processing policies are adopted in different states, which can maintain the speed of processing data packets during delay jitter and make the length of the buffer queue recover to K as soon as possible.

[0017] In an implementation provided in the first aspect, the expected length K of the buffer queue, the data transmission delay, the service setting delay, and the jitter buffer delay satisfy: K=(S yw -S1+S N ) / S; where S ywSet the latency for the service. S1 is the data transmission latency, and S N is the jitter buffer latency, and S is the packet sending interval.

[0018] In an implementation provided in the first aspect, the receiving end includes a communication module server. Receiving the data packets of the first service transmitted by the sending end includes: the communication module server receives the data packets of the first service transmitted by the sending end; the communication module server processes the data packets of the first service based on the actual length k of the buffer queue and the expected length K of the buffer queue.

[0019] In an implementation provided in the first aspect, the receiving end further includes a network detection module; in response to receiving the data packets of the first service, sending a transmission completion frame to the sending end at a preset time interval includes: in response to receiving the data packets of the first service, the communication module server sends a transmission completion frame to the sending end at a preset time interval; receiving a transmission completion acknowledgment frame includes: the communication module server receives a transmission completion acknowledgment frame; the method further includes: the communication module server sends first time information to the network detection module; where the first time information includes a waiting duration, a first moment, and a second moment; calculating the round-trip latency based on a third moment and a fourth moment includes: the network detection module calculates the round-trip latency based on the third moment and the fourth moment; determining the data transmission latency based on the waiting duration, the time interval, and the round-trip latency includes: the network detection module determines the data transmission latency based on the waiting duration, the time interval, and the round-trip latency.

[0020] In an implementation provided in the first aspect, the receiving end further includes a dynamic buffer algorithm module; the method further includes: the network detection module sends the data transmission latency to the dynamic buffer algorithm module; determining the service setting latency based on a first threshold and a second threshold corresponding to the first service and the data transmission latency includes: the dynamic buffer algorithm module determines the service setting latency based on the first threshold and the second threshold corresponding to the first service and the data transmission latency.

[0021] In an implementation provided in the first aspect, the method further includes: the network detection module determines the network jitter value based on multiple data transmission latencies obtained within a preset duration; the network detection module sends the network jitter value to the dynamic buffer algorithm module; the receiving end further includes a dynamic buffer algorithm module; determining the jitter coefficient based on network information includes: the dynamic buffer algorithm module determines the jitter coefficient based on network information; calculating the jitter buffer latency based on the network jitter value and the jitter coefficient includes: the dynamic buffer algorithm module calculates the jitter buffer latency based on the network jitter value and the jitter coefficient.

[0022] In an implementation provided in the first aspect, before the dynamic caching algorithm module determines the service setting delay according to the first threshold, the second threshold corresponding to the first service, and the data transmission delay, the method further includes: the communication module on the server side receives a request for establishing a channel sent by the sending end; wherein, the request for establishing a channel carries the identification information of the first service; the communication module on the server side queries and obtains the first delay information according to the identification information of the first service, and the first delay information includes the first threshold and the second threshold corresponding to the first service; the communication module on the server side obtains network information; the communication module on the server side sends the first delay information and the network information to the dynamic caching algorithm module.

[0023] In an implementation provided in the first aspect, processing the data packets of the first service based on the actual length k of the cache queue and the expected length K of the cache queue includes: the communication module on the server side adjusts the processing policy of the data packets cached in the cache queue based on the actual length k of the cache queue and the expected length K of the cache queue, and after K data packets are cached in the cache queue, processes the data packets cached in the cache queue according to the packet sending interval.

[0024] In an implementation provided in the first aspect, the receiving end further includes a big data reporting module, and the method further includes: the big data reporting module obtains the lag information, and the lag information includes the number of lags, the lag duration, and the data transmission delay during the receiving end's processing of the first service; the big data reporting module sends the lag information to the cloud server after the receiving end stops processing the first service.

[0025] In the second aspect, an embodiment of the present application provides a data processing method applied to the sending end. The sending end establishes a wireless connection with the receiving end. The method includes: sending a request for establishing a session to the receiving end, where the request for establishing a session carries the identification information of the first service; receiving a response for establishing a session fed back by the receiving end, and establishing a first session with the receiving end, where the first session is used to transmit the data packets of the first service; sending data packets to the receiving end based on the first session, so that the receiving end processes the data packets of the first service based on the actual length k of the cache queue and the expected length K of the cache queue; wherein, the actual length k of the cache queue is the number of data packets actually stored in the cache queue, the expected length K of the cache queue is the number of data packets expected to be stored in the cache queue, the cache queue is used to store the data packets of the first service, the expected length K of the cache queue is associated with the data transmission delay, the service setting delay, and the jitter buffer delay, the data transmission delay is the time required for the sending end to send a data packet to the receiving end to receive the corresponding data packet, the service setting delay is associated with the first service and the data transmission delay, and the jitter buffer delay is related to the network state between the receiving end and the sending end and the jitter degree of the data transmission delay.

[0026] In an implementation provided in the second aspect, the sending end includes a communication module client and a network detection module. The method further includes: The communication module client receives a transmission completion frame sent by the receiving end; wherein, the transmission completion frame carries the number of data packets received by the receiving end and the time when each data packet is received; The communication module client sends a transmission completion acknowledgment frame to the receiving end; wherein, the transmission completion acknowledgment frame carries a waiting duration, and the waiting duration is the difference between a first time and a second time. The first time is the time when the sending end sends the first data packet, and the first data packet is the first data packet received by the receiving end indicated in the transmission completion acknowledgment frame. The second time is the time when the communication module client receives the transmission completion frame.

[0027] In an implementation provided in the second aspect, the sending end further includes a QoE reporting module. The method further includes: In response to receiving the transmission completion frame, the communication module client sends data processing information and network parameters to the QoE reporting module; The QoE reporting module determines a stuttering level according to the data processing information; When the stuttering level changes, the QoE reporting module sends stuttering information to the first service. The stuttering information carries the stuttering levels before and after the change and the network parameters; In response to receiving the stuttering information, the first service adjusts the bitrate or frame rate.

[0028] In a third aspect, an embodiment of the present application provides an electronic device, which includes: a communication module, a memory, and one or more processors; The communication module, the memory, and the processor are coupled; The memory is used to store computer program code, and the computer program code includes computer instructions. When the computer instructions are executed by the electronic device, the electronic device executes the methods in the first aspect, the second aspect, and any one of their items.

[0029] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions run on the electronic device, the electronic device executes the methods in the first aspect, the second aspect, and any one of their items.

[0030] In a fifth aspect, an embodiment of the present application provides a computer program product, including computer programs / instructions. When the computer programs / instructions are executed by a processor, the steps of the methods in the first aspect, the second aspect, and any one of their items are implemented.

[0031] Among them, the technical effects brought by any one of the design methods in the second aspect to the fifth aspect can refer to the technical effects brought by different design methods in the first aspect, which will not be elaborated here. Description of the Drawings

[0032] Figure 1 It is a schematic diagram of an existing cache queue solution provided by an embodiment of the present application;

[0033] Figure 2 It is a schematic flowchart of a delay detection provided by an embodiment of the present application;

[0034] Figure 3 It is a schematic hardware structure diagram of an electronic device provided by an embodiment of the present application;

[0035] Figure 4 It is a hierarchical architecture diagram of an electronic device provided by an embodiment of the present application;

[0036] Figure 5 It is a schematic flowchart of a data processing method provided by an embodiment of the present application Figure 1 ;

[0037] Figure 6 It is a schematic flowchart of a data processing method provided by an embodiment of the present application Figure 2 ;

[0038] Figure 7 It is a schematic interaction diagram between a sending end and a receiving end provided by an embodiment of the present application;

[0039] Figure 8 It is a schematic diagram of calculating transmission delay provided by an embodiment of the present application;

[0040] Figure 9 It is a schematic diagram of each state of a cache queue provided by an embodiment of the present application;

[0041] Figure 10 It is a schematic diagram of rapid recovery of an abnormal state of a cache queue provided by an embodiment of the present application;

[0042] Figure 11 It is a schematic flowchart of a data processing method provided by an embodiment of the present application Figure 3 ;

[0043] Figure 12 It is a schematic structure diagram of a QoE management system provided by an embodiment of the present application. Detailed implementation manners

[0044] The following describes the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular forms "a", "the", "above-mentioned", "this" and "this one" are also intended to include, for example, the expression form of "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one" and "one or more" mean one or more than two (including two). The term "and / or" is used to describe the association relationship of associated objects and means that three relationships can exist; for example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally means that the associated objects before and after are in an "or" relationship.

[0045] Reference to "one embodiment" or "some embodiments" etc. described in this specification means that a specific feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprise", "include", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise stated. "First" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features.

[0046] In the embodiments of the present application, words such as "exemplarily" or "for example" are used to give examples, illustrations or explanations. Any embodiment or design solution described as "exemplarily" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplarily" or "for example" is intended to present relevant concepts in a specific manner.

[0047] Users can use different types of streaming services under near - field conditions. Among them, streaming services refer to real - time services such as audio and video playback in the form of streaming media. Streaming media refers to a media format that uses streaming transmission. Before playing, the entire file is not downloaded. Instead, through the method of downloading, caching, and playing simultaneously, the media data is correctly output. Streaming services are unidirectional transmissions, including a sending end and a receiving end. The sending end transmits the data packets of this type of service to the receiving end, and the receiving end processes the received data packets, that is, submits the data packets to the upper layer to execute this type of service.

[0048] Currently, multiple types of streaming services with different transmission requirements are supported (such as super mouse and keyboard, super call, heterogeneous source screen mirroring, etc.). If completely relying on the air interface network for transmission, it will lead to a poor actual experience for services that are sensitive to network jitter, such as scenarios like mouse and keyboard same - frequency different - channel, different - frequency different - channel, and streaming transmission services with external interference.

[0049] Delay jitter is an important indicator affecting the quality of service (QoS) of streaming services. Among them, delay jitter refers to the change in delay. When data packets leave the sending end, they are sent evenly at a certain interval. However, when passing through the network, this even interval is disrupted due to different delays experienced by the data packets, thus generating jitter. In some existing solutions, a cache queue scheme can be adopted to resist network jitter.

[0050] Please refer to Figure 1 , Figure 1 for a schematic diagram of a cache queue streaming transmission model. As Figure 1 shown, the sending end sends data packets to the receiving end. To resist network jitter and provide the ability of packet - order preservation and dynamic frame rate upward, after receiving the packets, the receiving end does not immediately submit them, but first inserts them into the cache queue. After caching a certain number of data packets, it then submits the data packets upward.

[0051] Delay jitter will cause the receiving end to have a window period without receiving packets and a large window period with receiving packets. During the window period without receiving packets caused by delay jitter, the receiving end submits the data packets in the cache queue to the upper layer in order according to the packet - sending interval S. As Figure 1 shown, when the receiving end receives the fourth data packet, it starts to enter the window period without receiving packets. At this time, the receiving end starts to submit the first data packet in the cache queue. Among them, the time from when the first data packet enters the cache queue to when it is submitted by the receiving end is called the first - packet cache delay. The packet - submission period R refers to the time interval for the receiving end to submit data packets, which is set to 4 milliseconds here. As Figure 1As shown, after the end of the packet receiving window period, the receiving end starts to enter a large packet receiving window period, and the receiving end pre-stores the received data packets into the buffer queue. At this time, the receiving end is still delivering the data packets cached in the buffer queue according to the packet delivery cycle. Thus, during the period when the packet receiving at the receiving end is abnormal due to delay jitter, the buffer queue at the receiving end can cache the data packets, so that the delay jitter will not affect the delivery of data packets at the receiving end.

[0052] In Figure 1 the packet delivery policy shown, the user experience delay Tu is the sum of the first packet cache delay Tc and the transmission delay Ts, that is, Tu = Tc + TsTs (Equation 1), where K is the buffer queue depth, and the buffer queue depth can also be called the length of the buffer queue, specifically the number of data packets that can be cached in the buffer queue.

[0053] However, in this scheme, the length of the buffer queue is relatively fixed. If the network environment and service type change, it may be impossible to smooth the delay jitter due to the too short buffer queue, or performance waste may be caused due to the too long buffer queue.

[0054] In some existing other schemes, a dynamic buffer queue scheme can be adopted to resist network jitter. Among them, the WIFI negotiation rates of 433 - 1200 Mbps, 100 - 433 Mbps, and 0 - 100 Mbps can be defined as three intervals of good, medium, and poor. When the WIFI negotiation rate changes across intervals, the sending end sends a delay detection frame for delay detection. Then, according to the result of the delay detection, the buffer queue is adjusted.

[0055] Please refer to Figure 2 , Figure 2 for a schematic diagram of delay detection. As Figure 2 shown, the sending end sends a delay detection frame to the receiving end. After receiving the delay detection frame, the receiving end can immediately send back a delay detection acknowledgment (ACK) frame to the sending end. The sending end records the moment when it receives the delay detection acknowledgment frame, and the difference between this moment and the moment when the delay detection frame is sent is the round-trip delay (Round-Trip Time, RTT). Finally, the sending end transmits the delay notification frame carrying the RTT to the receiving end. Thus, the transmission delay Ts can be obtained, that is, Ts = RTT / 2. Among them, substituting Ts = RTT / 2 into the above Equation 1 and transforming it, it can be obtained that the buffer queue depth K satisfies: K = (Tu - RTT / 2) / S.

[0056] When the WIFI negotiation rate changes across intervals, delay detection can be re-initiated, that is, the transmission delay is recalculated, and then the buffer queue depth is obtained. Finally, according to the actual data and the length of the re-obtained buffer queue, the packet delivery policy is adjusted.

[0057] However, in the above solution, the delay detection frame is only sent when the network negotiation rate changes, and it cannot sense network jitter and delay in a timely manner, which in turn leads to inaccurate cache depth and inability to smooth jitter. In addition, the sizes of the delay detection frame, the delay detection acknowledgment frame, and the actual service frame are different, resulting in inaccurate detected delay. Moreover, the cache depth is determined only by service settings and network delay, and the length of the cache queue cannot be adjusted according to the degree of network jitter. Also, the above solution introduces a certain cache delay even when the network conditions are good, causing performance waste.

[0058] To solve the above problems, an embodiment of the present application provides a data processing method. This method can detect the specific degree of network jitter by obtaining the transmission delay between the receiving end and the sending end. And according to the service-set delay, the transmission delay and the degree of network jitter jointly determine the cache depth of the cache queue. Finally, the packet delivery policy is determined by adjusting the cache queue depth.

[0059] The following will describe in detail the implementation manner of the embodiment of the present application with reference to the accompanying drawings. A data processing method provided by an embodiment of the present application is applied to an electronic device (which can be a sending end or a receiving end). In an embodiment of the present application, taking the above electronic device as a mobile phone as an example, the hardware structure of the electronic device is introduced.

[0060] As Figure 3 shown, the electronic device 200 may include: a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone interface 270D, a sensor module 280, a key 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc.

[0061] Among them, the processor 210 may include one or more processing units. For example, the processor 210 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. The processor 210 may be the nerve center and command center of the electronic device 200. The processor 210 may generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0062] A memory may also be provided in the processor 210 for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. This memory may save the instructions or data that the processor 210 has just used or recycled. If the processor 210 needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated accesses, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0063] In the embodiments of the present application, the processor 210 may execute instructions or algorithms related to the data processing method to complete the transfer of data packets.

[0064] In some embodiments, the processor 210 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0065] The external memory interface 220 may be used to connect an external memory card, such as a Micro SD card, to implement the storage capacity expansion of the electronic device 200. The external memory card communicates with the processor 210 through the external memory interface 220 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0066] The internal memory 221 may be used to store computer-executable program code, and the executable program code includes instructions. The processor 210 executes various functional applications and data processing of the electronic device 200 by running the instructions stored in the internal memory 221. For example, in the embodiments of the present application, the processor 210 may execute the instructions stored in the internal memory 221. The internal memory 221 may include a storage program area and a storage data area.

[0067] Among them, the storage program area may store an operating system, applications required for at least one function (such as a service recommendation function, etc.). The storage data area may store data created during the use of the electronic device 200 (such as the maximum / minimum latency corresponding to different services, etc.). In addition, the internal memory 221 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0068] The wireless communication module 260 may provide wireless communication solutions applied to the electronic device 200, including WLAN (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), etc.

[0069] The wireless communication module 260 may be one or more devices integrating at least one communication processing module. The wireless communication module 260 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 210. The wireless communication module 260 may also receive the signals to be sent from the processor 210, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 2 and radiate them out.

[0070] It can be understood that the interface connection relationships between the modules illustrated in this embodiment are only illustrative and do not constitute a structural limitation on the electronic device 200. In other embodiments, the electronic device 200 may also include more or fewer modules than those provided in the above embodiments, and different interface connection methods or combinations of multiple interface connection methods may also be adopted between the various modules.

[0071] In the embodiments of the present application, the sender and the receiver may adopt Figure 4 the hierarchical architecture shown. In other words, Figure 4 the architecture shown can be applied to both the sender and the receiver at the same time. As Figure 4 shown, the electronic device includes a service layer, an application layer, and a transport layer.

[0072] Among them, the service layer includes a series of business-related application programs, such as super mouse and keyboard, super call, heterogeneous screen mirroring, super connection, etc.

[0073] The application layer includes communication applications that provide communication functions for the service layer, such as MagicLink, etc.

[0074] The transport layer includes a communication module for providing data transmission services, which can be implemented based on communication protocols such as the User Datagram Protocol (UDP) or the Transmission Control Protocol (TCP). Taking the example that the communication module is implemented based on UDP, the communication module can add a UDP header to the data packet to be passed to the service layer and then transmit it to the peer device.

[0075] In the embodiments of the present application, the transport layer further includes a network detection module, a dynamic cache algorithm module, a Quality of Experience (QoE) reporting module, and a big data reporting module.

[0076] Among them, the network detection module is used to monitor the transmission delay of data packets and network jitter, etc.

[0077] The dynamic cache algorithm module can determine the expected depth of the cache queue and dynamically adjust the length of the cache queue.

[0078] The QoE reporting module can be used to monitor the stuttering situation, and when the stuttering situation changes, report the stuttering situation and network situation to the sending device, so that the sending device can adjust the bit rate or frame rate.

[0079] The big data reporting module can be used to report the number of stutters, transmission delay, stutter duration, network jitter, etc. in this session to the cloud server after a session ends, which is convenient for the cloud server to record and analyze.

[0080] It should be noted that the above architecture is only an example. In other implementation manners, the architecture may include more modules than those Figure 4 shown, and no specific limitation is made here.

[0081] The following will specifically describe the data processing method provided in the embodiments of the present application in conjunction with the accompanying drawings and based on the Figure 4 architecture shown.

[0082] Refer to Figure 5 , which is a flowchart of a data processing method provided in the embodiments of the present application Figure 1 , and specifically describes the process of establishing a session between the sending device and the receiving device. Among them, the sending device includes Service 1 (which can also be called the first service), a communication application client, and a UDP client, and the receiving device includes Service 1, a communication application server, a UDP server, a dynamic cache algorithm module, and a big data reporting module. It should be noted that Service 1 is a streaming type service, such as heterogeneous screen mirroring, etc. In addition, both the communication application client and the communication application server can pass through Figure 4In the communication application implementation at the application layer, the UDP client and the UDP server can pass through Figure 4 The communication module implemented based on UDP at the transport layer in.

[0083] Such as Figure 5 As shown, the data processing method provided by the embodiments of the present application includes S501 to S516.

[0084] S501, Service 1 receives Operation 1 of the user and sends a request to establish a session to the communication application client.

[0085] Among them, the identity information of Service 1 is carried in the request to establish a session, such as the name of Service 1 (such as yytp) or the identity identifier (such as 001).

[0086] In a possible design, Service 1 can call the opensession interface to send the request to establish a session.

[0087] S502, The communication application client sends a request to establish a channel to the communication application server.

[0088] Among them, the identity information of Service 1, service parameters, and the software and hardware version information of the local end are carried in the request to establish a channel.

[0089] In a possible design, the communication application client can call the OpenChannel function to send a request to establish a channel.

[0090] In the embodiments of the present application, before the communication application client sends a request to establish a channel to the communication application server, it can first perform an initialization operation, which can initialize the applications (such as communication applications), modules (such as communication modules), threads, dynamic files, etc. that will be used. Among them, the communication application client can call the DTURBO_StreamMgmtInit function to implement the initialization operation.

[0091] S503, The communication application server queries and obtains Delay Information 1 according to the identity information of Service 1.

[0092] The Delay Information 1 is the delay information corresponding to Service 1. In the embodiments of the present application, the delay information includes the maximum fixed value of the service-set delay, the minimum fixed value of the service-set delay, and the maximum jitter buffer delay.

[0093] Among them, the service setup delay refers to the time it takes for a service to expect to send a data packet from the sending end until the receiving end starts to process the corresponding data packet. Therefore, the minimum fixed value and the maximum fixed value of the service setup delay respectively refer to the minimum expected delay and the maximum expected delay set by the service according to the service requirements corresponding to different service types under different network conditions. For example, for a service that is less sensitive to delay such as a voice call service, the maximum fixed value of the service setup delay can be set larger, such as 100 milliseconds. For a service that is more sensitive to delay such as screen mirroring, the maximum fixed value of the service setup delay also needs to be set smaller, such as 50 milliseconds.

[0094] Among them, the service setup delay can also be set to 0. For example, for some services that have very low requirements for delay and do not think it is necessary to set the buffer depth, the minimum fixed value and the maximum fixed value of the service setup delay corresponding to this service can both be set to 0.

[0095] The jitter buffer delay refers to the delay caused by network jitter (how to determine the jitter buffer delay will be described in detail in S613 later), and the maximum jitter buffer delay is the maximum value of the jitter buffer delay.

[0096] In the embodiments of the present application, the receiving end pre-stores delay information corresponding to multiple services. Exemplarily, multiple services and their corresponding delay information can be as shown in Table 1:

[0097] Table 1

[0098]

[0099] For example, if the identification information of Service 1 is "Super Voice Call", the communication application server can query the delay information 1 in Table 1. The delay information 1 indicates that the maximum fixed value of the service setup delay is B1 milliseconds, the minimum fixed value is B0 milliseconds, and the maximum jitter buffer delay is B2 milliseconds.

[0100] It should be noted that in other embodiments, the correspondence between services and delay information may not be presented in the form of a table.

[0101] S504, the communication application server obtains network information.

[0102] In the embodiments of the present application, the network information, which can also be called frequency band characteristic information, is used to indicate the network state between the receiving end and the sending end. Among them, according to the frequency points and channels used by the receiving end and the sending end, the network state can include the same frequency and the same channel, the same frequency and different channels, and different frequencies and different channels.

[0103] Among them, same frequency and same channel means that the receiving end and the sending end use the same channel and the same frequency band to transmit data, for example, both the receiving end and the sending end use the 2.4G frequency band of WIFI to transmit data. Same frequency and different channels means that the receiving end and the sending end use different channels and the same frequency band to transmit data packets, for example, the sending end uses the WIFI channel and the P2P channel to transmit data, and the frequency bands of the WIFI channel and the P2P channel are the same. Different frequencies and different channels means that the receiving end and the sending end use different channels and different frequencies to transmit data packets, for example, the sending end uses the 2.4G frequency band of WIFI to transmit data, and the receiving end uses the P2P channel to transmit data.

[0104] The network information can be used to determine the cache delay caused by network jitter. For details, see S613, which will not be described here.

[0105] S505, the communication application server sends an enable instruction 1 to the dynamic cache algorithm module, where the enable instruction 1 carries delay information 1 and network information.

[0106] In this way, it is convenient for the subsequent dynamic cache algorithm module to dynamically adjust the target cache delay based on the delay information 1 and the network information (see S613 below for details).

[0107] S506, the communication application server sends an opening instruction 2 to the big data reporting module.

[0108] In the embodiment of the present application, the start instruction 2 carries the identification information of the service 1. In this way, it is convenient for the subsequent big data reporting module to actively report the jamming situation and other information (see below for details). Figure 11 related content).

[0109] S507, the communication application server sends a notification 1 of allocating a port to the UDP server.

[0110] The notification 1 of the allocated port carries the identification information of the service 1 .

[0111] In a possible design, the communication application server can call the DTURBO_StreamServer function to deliver the notification 1 of the allocated port.

[0112] It should be noted that there is no strict execution order between S504, S506 and S507. The communication application server can execute S504, S506 and S507 at the same time, or in the order of Figure 5 The steps may be performed in the order shown, or in any other order without any specific limitation.

[0113] In a possible design, before the communication application server executes S504, S506, and S507, it can first perform an initialization operation. This initialization operation can initialize the applications (such as communication applications), modules (such as communication modules), threads, dynamic files, etc. that will be used. Among them, the communication application server can call the DTURBO_StreamMgmtInit function to implement the initialization operation.

[0114] S508, the UDP server sends port information to the communication application server.

[0115] In the embodiment of the present application, the UDP server can, in response to the notification 1 of port allocation, allocate a port to Service 1 as the port information of the UDP server. The UDP server can also send the corresponding port information to the communication application server, and the port information can specifically be the port number.

[0116] S509, the communication application server sends a confirmation of establishing a channel to the communication application client.

[0117] Among them, the confirmation of establishing a channel carries the identification information of Service 1, the port number, the version information of the receiving end, etc.

[0118] In a possible design, the communication application server can pass the confirmation of establishing a channel to the communication application client through the OpenChannel function.

[0119] S510, the communication application client sends a notification 2 of port allocation to the UDP client.

[0120] Among them, the notification 2 of port allocation carries the identification information of Service 1, the port information of the UDP server, etc.

[0121] In a possible design, the communication application client can call the DTURBO_StreamClient function to pass the notification 2 of port allocation.

[0122] S511, the UDP client sends a setting frame to the UDP server.

[0123] In the embodiment of the present application, in response to receiving the notification 2 of port allocation, the UDP client can also allocate a port number to Service 1 as the port information of the UDP client. Then, the UDP client can send a setting frame to the UDP server based on the port information of the UDP server.

[0124] Among them, the setting frame carries the port information of the sending end, which is convenient for the receiving end to feedback information to the sending end.

[0125] S512, the UDP server sends a setup confirmation frame to the UDP client.

[0126] Among them, the UDP server can send a setup confirmation frame to the UDP client based on the port information of the UDP client. By exchanging the setup frame and the setup confirmation frame, the UDP server and the UDP client can confirm that a channel has been successfully established with the peer device and a session can be conducted.

[0127] S513, the UDP server sends a notification of creating a new session to the communication application server.

[0128] In a possible design, the communication application server can register to listen for the setup frame through the MP_ON_NEW_SESSION function. In this way, after the UDP server receives the setup frame, it can call back the MP_ON_NEW_SESSION function and send a notification of creating a new session to the communication application server.

[0129] S514, the communication application server sends a notification of creating a new session to Service 1.

[0130] In a possible design, the communication application server can call the onSessionOpen function to pass the notification of creating a new session.

[0131] S515, the UDP client sends a notification of creating a new session to the communication application client.

[0132] In a possible design, the communication application client can register to listen for the setup confirmation frame through the MP_ON_CONNETCTED function. In this way, after the UDP server receives the setup confirmation frame, it can call back the MP_ON_CONNETCTED function and send a notification of creating a new session to the communication application server.

[0133] S516, the communication application client sends a notification of creating a new session to Service 1.

[0134] In a possible design, the communication application client can call the onSessionOpen function to pass the notification of creating a new session.

[0135] In the embodiments of this application, different delay information is set for different services, and during the process of establishing a session between the sending end and the receiving end, the delay information (such as the above-mentioned delay information 1) involved in this service (such as Service 1) can be passed to the dynamic cache algorithm module, and at the same time, the network information is also transmitted and passed to the dynamic cache algorithm module during the process of establishing a session, which is beneficial for the subsequent dynamic cache algorithm module to adjust the target cache delay in a timely manner according to the network situation and delay information 1.

[0136] After establishing a session for Service 1 between the sender and the receiver, the sender can send data packets of Service 1 to the receiver, and the receiver can process the received data packets. Taking Service 1 as an example of heterogeneous screen mirroring, the sender is the screen mirroring device (such as a mobile phone), and the receiver is the device to be screen mirrored (such as a TV), and the data packets can be data to be screen mirrored.

[0137] See Figure 6 , which is a schematic flowchart of a data processing method provided by an embodiment of the present application Figure 2 , and specifically describes the process of the sender sending data packets to the receiver and the receiver processing the data packets. Among them, the sender further includes a network detection module 1 and a QoE reporting module 1, and the receiver further includes a network detection module 2. It should be noted that both the network detection module 1 and the network detection module 2 can be implemented by Figure 4 the network detection module located in the transport layer in Figure 4 the QoE reporting module located in the transport layer in

[0138] As Figure 6 shown, the data processing method provided by the embodiment of the present application further includes S601~S617.

[0139] S601, Service 1 sends a data packet to the communication application client.

[0140] In the embodiment of the present application, Service 1 can send data packets evenly at a certain time interval S. For example, Service 1 can send a data packet every 4 milliseconds, which is equivalent to S = 4 milliseconds. Of course, in other embodiments, the time interval for Service 1 to send data packets can also be uneven, and no specific limitation is made here.

[0141] S602, the communication application client sends the data packet to the UDP client.

[0142] That is to say, the communication application client can forward the data packet sent by Service 1 to the UDP client.

[0143] S603, the UDP client sends a data frame to the UDP server.

[0144] In the embodiment of the present application, every time the UDP client receives a data packet, it can encapsulate information such as the data packet, the port number of the UDP client, and the port number of the UDP server in a certain format to obtain a data frame. In other words, the data frame carries the data packet, the port number of the UDP client, and the port number of the UDP server.

[0145] Among them, the UDP client can send data frames to the UDP server based on the port number of the UDP server. In addition, the UDP client can also attach a timestamp to each data frame to record the time when the data frame is sent.

[0146] S604. The UDP server sends a Transfer Done frame to the UDP client at preset time intervals starting from time 1.

[0147] In the embodiment of this application, the UDP server can record the time when each data frame is received. Among them, time 1 is the time when the UDP server receives the first data frame. That is to say, starting from the time when the UDP server receives the first data frame, a Transfer Done frame is sent to the UDP client every time the duration corresponding to the time interval elapses.

[0148] In a possible design, the Transfer Done frame carries the number of data frames received by the UDP server within the corresponding time interval, as well as the sending time and receiving time (collectively referred to as the sending and receiving time) corresponding to each data frame. Among them, the sending time corresponding to the data frame refers to the time when the UDP client sends the data frame, and the receiving time corresponding to the data frame is the time when the UDP server receives the data frame.

[0149] Exemplarily, taking time 1 as T1 and the time interval as 100 ms as an example, the UDP server can send Transfer Done frame 1 to the UDP client at the time of T1 + 100 ms, send Transfer Done frame 2 to the UDP client at the time of T1 + 200 ms, send Transfer Done frame 3 to the UDP client at the time of T1 + 300 ms... Among them, Transfer Done frame 1 includes the number of data frames received by the UDP server during T1 to T1 + 100 and the sending and receiving time corresponding to each data frame, Transfer Done frame 2 includes the number of data frames received by the UDP server during T1 + 100 to T1 + 200 and the sending and receiving time corresponding to each data frame, and Transfer Done frame 3 includes the number of data frames received by the UDP server during T1 + 200 to T1 + 300 and the sending and receiving time corresponding to each data frame.

[0150] In another possible design, the Transfer Done frame may not carry the sending time corresponding to each data frame.

[0151] S605. The UDP client sends time information 1 to the network detection module 1 in response to receiving the Transfer Done frame.

[0152] That is to say, every time the UDP client receives a Transfer Done frame, it sends time information 1 to the network detection module 1 once. This time information 1 includes time 2 and time 3.

[0153] Among them, time 2 is the transmission time corresponding to the first data frame received by the UDP server within the corresponding time interval (for example, data frame 1), or it can be understood as the transmission time corresponding to the first data frame (for example, data frame 1) confirmed and received in the transmitted completed frame, that is, the time when the UDP client sends this data frame 1. Time 3 is the time when the UDP client receives the transmitted completed frame.

[0154] It can be understood that the UDP client can extract the transmission time corresponding to each data frame from the transmitted completed frame and determine the transmission time corresponding to data frame 1 from it.

[0155] In the case where the transmitted completed frame does not carry the transmission time corresponding to each data frame, the UDP client can determine time 2 according to the number of data frames received by the UDP server carried in each received transmitted completed frame (which will be specifically described later in combination with Figure 7 ).

[0156] It can be understood that since the data frames confirmed and received by each transmitted completed frame are different and the transmission times corresponding to each transmitted completed frame are also different, the time 2 and time 3 sent by the UDP client to the network detection module 1 each time are also different.

[0157] S606, the network detection module 1 calculates the waiting duration according to the time information 1.

[0158] Among them, the waiting duration is the difference between time 3 and time 2, which is used to indicate the time consumed by the UDP client (i.e., the sending end) from sending data frame 1 to receiving the transmitted completed frame.

[0159] S607, the network detection module 1 sends the waiting duration to the UDP client.

[0160] S608, the UDP client sends a transmitted completed acknowledgement (ACK) frame to the UDP server in response to receiving the transmitted completed frame.

[0161] It can be seen that every time the UDP client receives a transmitted completed frame, it can receive the waiting duration transmitted by the network detection module 1, and then send a transmitted completed acknowledgement frame to the UDP server. In other words, every time the UDP client receives a transmitted completed frame, it sends a transmitted completed acknowledgement frame to the UDP server once, and the transmitted completed acknowledgement frame carries the corresponding waiting duration.

[0162] Exemplarily, refer to Figure 7 , which is an interaction schematic diagram of the sending end and the receiving end. As Figure 7As shown, the sender sends data frames to the receiver in sequence. After receiving the first data frame, the receiver sends a transmission completed frame to the sender every time interval D1. After receiving the transmission completed frame each time, the sender returns a transmission completed acknowledgment frame to the receiver.

[0163] Taking the example that the receiver receives the first data frame at time T1, the receiver can send the transmission completed frame 1 to the sender at T1 + D1, send the transmission completed frame 2 to the sender at T1 + 2D1, and send the transmission completed frame 3 to the sender at T1 + 3D1. Among them, the transmission completed frame 1 is used to confirm that 2 data frames have been received (i.e., the first data frame and the second data frame), and carries the reception times of the first data frame and the second data frame; the transmission completed frame 2 is used to confirm that 2 data frames have been received (i.e., the third data frame and the fourth data frame), and carries the reception times of the third data frame and the fourth data frame; the transmission completed frame 3 is used to confirm that 1 data frame has been received (i.e., the fifth data frame), and carries the reception time of the fifth data frame.

[0164] In response to receiving the transmission completed frame 1, the transmission completed frame 2, and the transmission completed frame 3 respectively, the sender sends the transmission completed acknowledgment frame 1, the transmission completed acknowledgment frame 2, and the transmission completed acknowledgment frame 3 to the receiver. Among them, since the transmission completed frame 1 is used to indicate that the receiver has received 2 data frames (i.e., the first data frame and the second data frame), the sender can confirm that the first data frame it sent is the first data frame received by the receiver during T1~T1 + D1. Therefore, the waiting duration carried in the transmission completed acknowledgment frame 1 is the difference between the time when the sender receives the transmission completed frame 1 and the time when the sender sends the first data frame, which is the waiting duration W1 in the figure.

[0165] Since the transmission completed frame 2 is used to indicate that the receiver has received 2 data frames (the third data frame and the fourth data frame), and combined with the fact that the transmission completed frame 1 also indicates that the receiver has received 2 data frames, the sender can confirm that the third data frame it sent is the first data frame received by the receiver during T1 + D1~T1 + 2D1. Therefore, the waiting duration carried in the transmission completed acknowledgment frame 2 is the difference between the time when the sender receives the transmission completed frame 2 and the time when the sender sends the third data frame, which is the waiting duration W2 in the figure.

[0166] Since the transmission completed frame 3 is used to indicate that the receiving end has received 1 data frame, combined with the fact that both the transmission completed frame 1 and the transmission completed frame 2 indicate that the receiving end has received 2 data frames, the sending end can confirm that the 5th data frame it sent is the 1st data frame received by the receiving end during the period from T1 + 2D1 to T1 + 3D1. Therefore, the waiting duration carried in the transmission completed confirmation frame 3 is the difference between the moment when the sending end receives the transmission completed frame 3 and the moment when the 5th data frame is sent, which is the waiting duration W3 in the figure.

[0167] S609, the UDP server responds to the received transmission completed confirmation frame and sends time information 2 to the network detection module 2.

[0168] Among them, the time information 2 carries the waiting duration, the moment 4 when the UDP server sends the transmission completed frame, and the moment 5 when the UDP server receives the transmission completed confirmation frame. It should be noted that the waiting duration, the transmission completed frame, and the transmission completed confirmation frame are generated during one interaction process, that is, there are no other transmission completed frames and transmission completed confirmation frames between this transmission completed frame and the transmission completed confirmation frame.

[0169] Exemplarily, still as Figure 7 shown, the time information 2 can carry the moment when the receiving end sends the transmission completed frame 1, the moment when the receiving end receives the transmission completed confirmation frame 1, and the waiting duration W1; or, the time information 2 can carry the moment when the receiving end sends the transmission completed frame 2, the moment when the receiving end receives the transmission completed confirmation frame 2, and the waiting duration W2; or, the time information 2 can carry the moment when the receiving end sends the transmission completed frame 3, the moment when the receiving end receives the transmission completed confirmation frame 3, and the waiting duration W3.

[0170] S610, the network detection module 2 determines the data transmission delay 1 based on the time information 2 and the time interval.

[0171] Among them, the data transmission delay 1 can be understood as the time required for the sending end to send a data frame until the receiving end receives the corresponding data frame.

[0172] In the embodiment of the present application, the network detection module 2 can parse the time information 2 to obtain the time 4 and the time 5, and then subtract the time 4 from the time 5 to obtain the round-trip delay (RTT) of the signaling frame. The round-trip delay RTT of the signaling frame is the sum of the transmission delay of the transmitted completed frame and the transmission delay of the transmitted completed acknowledgment frame. Since the sizes of the transmitted completed frame and the transmitted completed acknowledgment frame are almost the same, and the UDP client and the network detection module 1 are located at the same layer, the transmission delay between them can be almost ignored. Therefore, it can be considered that the transmission delays of the transmitted completed frame and the transmitted completed acknowledgment frame are the same. Thus, the network detection module 2 can calculate that the transmission delay of the transmitted completed frame is RTT / 2.

[0173] As Figure 8 shown, taking the time interval as D, the waiting duration as W, and the data transmission delay 1 as S1 as an example, S1 = W - D - RTT / 2. Among them, the data transmission delay 1 is specifically the transmission delay of the first data frame.

[0174] It can be understood that since the UDP client and the UDP server exchange the transmitted completed frame and the transmitted completed acknowledgment frame (referred to as the signaling frame) at the above time interval, every time the UDP server and the UDP client exchange a signaling frame, the network detection module 2 can determine the data transmission delay 1 once.

[0175] Exemplarily, in Figure 7 the scenario shown, the sender and the receiver exchange at least 3 signaling frames, then the network detection module 2 can obtain at least 3 data transmission delays 1, which are the transmission delay of the first data frame, the transmission delay of the third data frame, and the transmission delay of the fifth data frame respectively.

[0176] Thus, the receiver can detect the transmission delay of the data frame of service 1 in real time and accurately. Moreover, there is no need to add extra data frames to detect the transmission delay, which can reduce power consumption and save power.

[0177] S611, the network detection module 2 calculates the network jitter value based on multiple data transmission delays 1.

[0178] Among them, the network jitter value can be used to reflect the jitter degree of the network within a period of time. Among them, the higher the network jitter value, the stronger the network jitter, that is, the more unstable the network; the smaller the network jitter value, the weaker the network jitter, that is, the more stable the network.

[0179] In a possible design, the network detection module 2 can calculate the average deviation σ of multiple data transmission delays 1 and use the average deviation σ as the network jitter value. The calculation formula of the average deviation σ is as follows:

[0180]

[0181] Wherein, n is the number of data transmission delays 1, is the i-th data transmission delay 1, i = 1 to n, is the average value of n data transmission delays 1. The difference between each data transmission delay 1 and the average value is the deviation, and the average deviation σ is the average value of the absolute values of each deviation. In the embodiments of the present application, the receiving end determines the average deviation of the calculated transmission delay as the jitter indication value of the transmission delay, that is, it can reflect the jitter degree of the transmission delay within a preset time period.

[0182] In a possible design, the network detection module 2 can calculate the network jitter value every preset time interval starting from the moment when the first time information 2 is received, based on all the data transmission delays 1 determined within the preset time interval. For example, the network detection module 2 calculates the network jitter value every 4 seconds starting from the moment when the first time information 2 is received, based on the data transmission delays 1 determined within the 4 seconds, to reflect the network jitter degree within the 4 seconds.

[0183] In another possible design, the network detection module 2 can count the data of the determined data transmission delays 1. Every time the network detection module 2 determines a preset number of data transmission delays 1, it can calculate the network jitter value based on the preset number of data transmission delays 1.

[0184] It should be noted that in other embodiments, the network jitter value can also be the average deviation of the data transmission delay 1, or other values, such as the standard deviation of the data transmission delay 1, as long as it can more accurately reflect the jitter degree of the data transmission delay 1.

[0185] S612. The network detection module 2 sends the data transmission delay 2 and the network jitter value to the dynamic cache algorithm module.

[0186] Wherein, the data transmission delay 2 can be the average value of multiple data transmission delays 1 within a preset time interval, or other values that can reflect the transmission delay of the data frame.

[0187] S613. The dynamic cache algorithm module determines the target cache delay based on the data transmission delay 2, the network jitter value, the delay information 1, and the network information.

[0188] Wherein, the target cache delay can be understood as the time used by the UDP server (receiving end) from receiving the first data frame to delivering the first data frame.

[0189] In the embodiments of the present application, the target cache delay (for example, S T) can include two parts. One part is the latency set to meet the actual requirements of different services, which can be called service cache latency (for example, for S B ) ; the other part is the latency caused by network jitter, which can be called jitter cache latency (for example, for S N ). That is to say, the target cache latency S T = service cache latency S B + jitter cache latency S N .

[0190] Therefore, the dynamic cache algorithm module can determine the service cache latency and the jitter cache latency based on the data transmission latency 2, the network jitter value, the latency information 1, and the network information, and then use the sum of the service cache latency and the jitter cache latency as the target cache latency. The processes of the dynamic cache algorithm module determining the service cache latency and the jitter cache latency will be described separately below.

[0191] (1) The dynamic cache algorithm module can determine the service cache latency based on the data transmission latency 2, the maximum fixed value of the service setting latency carried in the latency information 1, and the minimum fixed value of the service setting latency.

[0192] As described above, the service setting latency refers to the time it takes for a service to expect to send a data packet from the sender to the receiver to start processing the corresponding data packet. It can be seen that this service setting latency actually includes two parts. One part is the transmission latency of the data frame (i.e., the data transmission latency), and the other part is the service cache latency. Therefore, subtracting the data transmission latency from the service setting latency can obtain the service cache latency.

[0193] It can be understood that to a certain extent, the data transmission latency can represent the network condition. The smaller the data transmission latency, the better the network condition. That is, the minimum fixed value of the service setting latency is the service expected latency corresponding to the case when the network condition is good, and the maximum fixed value is the service expected latency corresponding to the case when the network state is poor. Thus, the dynamic cache algorithm module can determine the value of the service setting latency to be the minimum fixed value or the maximum fixed value according to the actual data transmission latency (i.e., the data transmission latency 2).

[0194] Among them, due to the limitation of the service setting latency, even when the network condition is very good, that is, when the data transmission latency is very small, the service may also need a certain cache latency. Therefore, when the data transmission latency 2 is less than the minimum fixed value of the service setting latency, the service setting latency is determined to be the minimum fixed value, and thus the service cache latency is the difference between the minimum fixed value and the data transmission latency 2.

[0195] Moreover, even when the network condition is poor, that is, when the data transmission delay 2 is relatively large, the service cannot increase the caching delay indefinitely. Therefore, when the data transmission delay 2 is not less than (greater than or equal to) the minimum fixed value of the service setting delay, the service setting delay is determined to be the maximum fixed value, so that the service caching delay is the difference between the maximum fixed value and the data transmission delay 2.

[0196] Specifically, when the data transmission delay 2 is greater than the maximum fixed value of the service setting delay, due to the limitation of the maximum fixed value, the service caching delay is 0.

[0197] Exemplarily, taking the minimum fixed value A1 of the service setting delay as 30 milliseconds and the maximum fixed value A2 of the service setting delay as 50 milliseconds as an example, when the data transmission delay 2 is 20 milliseconds (less than 30 milliseconds), the dynamic caching algorithm module can determine that the service setting delay is 30 milliseconds, and the service caching delay is 30 - 20 = 10 milliseconds. When the data transmission delay 2 is 35 milliseconds (greater than 30 milliseconds and less than 50 milliseconds), the dynamic caching algorithm module can determine that the service setting delay is 50 milliseconds, and the service caching delay is 50 - 35 = 15 milliseconds. When the data transmission delay 2 is 55 milliseconds, since this data transmission delay 2 is greater than the maximum fixed value of 50 milliseconds, the dynamic caching algorithm module can directly determine that the service caching delay is 0 milliseconds.

[0198] (2) The dynamic caching algorithm module can determine the jitter caching delay based on the network jitter value and network information.

[0199] In the embodiment of the present application, the dynamic caching algorithm module can determine the jitter coefficient according to the network information, and this jitter coefficient is used to reflect the impact of the network state on the jitter caching delay. Among them, the larger the jitter coefficient, the greater the impact of the network state on the jitter caching delay.

[0200] In the embodiment of the present application, when the sending end and the receiving end are of the same frequency and the same channel, the jitter coefficient p1 can be 0; when the sending end and the receiving end are of the same frequency and different channels, the jitter coefficient p2 can be 3; when the sending end and the receiving end are of different frequencies and different channels, the jitter coefficient p3 can be 6. It should be noted that in other implementation manners, the jitter coefficients p1, p2, and p3 can also be other values, as long as p1 < p2 < p3 is satisfied, and no specific limitation is made here.

[0201] It can be understood that since there will be a situation similar to network jitter in the case of the same frequency and different channels due to the antenna time-division transmission, compared with the same frequency and the same channel, the jitter coefficient will be a little larger. Then, compared with the same frequency and different channels, the jitter coefficient in the case of different frequencies and different channels will be even larger.

[0202] Then, the dynamic caching algorithm module can calculate the delay S3 based on the jitter coefficient and the network jitter value, and use the delay S3 as the jitter buffer delay. Among them, the delay S3 = the jitter coefficient p * the network jitter value σ.

[0203] In a possible design, the dynamic caching algorithm module can further determine the jitter buffer delay according to the delay S3, the packet sending interval S, and the maximum jitter buffer delay.

[0204] Among them, when the delay S3 is less than the packet sending interval S of the sending end, it can be considered that the network condition is relatively good at this time, and there is no need to introduce a jitter buffer. That is to say, when the delay S3 is less than the packet sending interval S, the dynamic caching algorithm module can set the jitter buffer delay to 0. In this way, when the network state is good, the network jitter buffer will not be introduced, thus avoiding performance waste.

[0205] When the delay S3 is greater than or equal to the packet sending interval S of the sending end and less than the maximum jitter buffer delay, the dynamic caching algorithm module can set the jitter buffer delay to the delay S3.

[0206] When the delay S3 is greater than or equal to the maximum jitter buffer delay, the dynamic caching algorithm module sets the jitter buffer delay to the maximum jitter buffer delay.

[0207] In the embodiment of the present application, when service 1 is just initiated and the receiving end has not calculated the data transmission delay 2 yet, at this time, it can be considered that the data transmission delay is 0 and the network jitter value is also 0, that is, the jitter buffer delay is 0. According to the relationship between the data transmission delay and the service setting delay, it can be determined that the service setting delay is the minimum fixed value at this time. Then it can be known that the target cache delay at this time is the minimum fixed value.

[0208] S614, the dynamic caching algorithm module sends the target cache delay to the UDP server.

[0209] S615, the UDP server determines the expected length K of the cache queue according to the target cache delay.

[0210] It can be understood that after receiving the data frame, the UDP server can parse the data frame to obtain the data packet carried in the data frame and temporarily store it in the cache queue. Among them, the expected length K of the cache queue is the number of data packets expected to be cached in the cache queue.

[0211] In the embodiment of the present application, the expected length K of the cache queue satisfies the formula: S T = K × S. Among them, S is the time interval for the sending end to send data packets, which can be simply referred to as the packet sending interval. In a possible design, the packet sending interval S can be the average value of the packet sending intervals used by the sending end to send multiple data packets.

[0212] Further, since S T = S yw - S1 + S N , then K = (S yw - S1 + S N ) / S. Among them, S yw is the delay set for the service, S1 is the data transmission delay (such as the above data transmission delay 1 or data transmission delay 2), S N is the jitter buffer delay, and S is the packet sending interval.

[0213] In the embodiments of the present application, the UDP server (receiver) can adjust the length of the buffer queue (i.e., determine the expected length K of the buffer queue) when service 1 is just initiated, rather than delivering packets based on a fixed buffer queue length, which can achieve a better anti-delay jitter effect.

[0214] S616, the UDP server sends data packets to the communication application server according to the actual length k of the buffer queue and the expected length K of the buffer queue.

[0215] Among them, the actual length k of the buffer queue is the number of data packets actually cached in the buffer queue.

[0216] It can be understood that in the actual application process, the actual length k of the buffer queue and the expected length K of the buffer queue are not necessarily the same. Among them, as Figure 9 shown, in the actual scenario, according to the different relationships between the actual length k of the buffer queue and the expected length K of the buffer queue, the buffer queue may include the following states. State 1: In a good network state, the receiver receives packets evenly. At this time, the actual length k of the buffer queue = K. State 2: In the case of network jitter, the receiver does not receive packets. At this time, the actual length k of the buffer queue < K. State 3: In the case of network jitter, the receiver receives a large number of packets in a short time. At this time, the actual length k of the buffer queue > K. State 4: The actual length k of the buffer queue = 0. State 5: The actual length k of the buffer queue = the maximum value MAX. Among them, the maximum value MAX refers to the maximum number of data packets that the buffer queue can accommodate.

[0217] Among them, in different states, the UDP server can send data packets to the communication application server in different ways.

[0218] State 1, the actual length k of the buffer queue = K, and the UDP server delivers packets evenly according to the packet sending interval S (sends data packets to the communication application server).

[0219] State 2, the actual length k of the buffer queue < K, the UDP server delivers packets evenly according to the packet sending interval S, and records the lost data packets in the buffer queue and retransmits the lost data packets.

[0220] In state 3, the actual length k of the storage queue is greater than K. The UDP server delivers packets evenly at a target interval or discards the packets in the buffer queue, where the target interval is less than the packet sending interval S.

[0221] In state 4, the actual length k of the buffer queue is 0. Within the preset duration after entering state 4, the UDP server can immediately deliver the received packets; after the preset duration of entering state 4, the UDP server restarts the buffer queue.

[0222] In state 5, the actual length k of the buffer queue is at the maximum value. The UDP server immediately delivers the received packets.

[0223] It can be understood that the above several states can be transformed into each other.

[0224] Please refer to Figure 10 , Figure 10 which is a schematic diagram of fast recovery of abnormal states of a buffer queue provided by an embodiment of the present application. As Figure 10 shown, in the initial state, the receiving end periodically detects the network state, determines the expected length K of the buffer queue, and enters state 1 after caching K packets. During this period, the UDP server delivers packets in the manner corresponding to the above state 1; if the receiving end cannot receive packets due to network jitter during state 1, it enters state 2. During this period, the UDP server delivers packets in the manner corresponding to the above state 2; if the receiving end still cannot receive packets due to network jitter during state 2, it enters state 4. During this period, if the UDP server receives packets, it immediately delivers them. After a period of time, the state can be switched to state 2. If the UDP server still does not receive packets, it restarts the buffer and returns to the initial state; if the receiving end receives a large number of packets due to network jitter during state 2, it enters state 3. During this period, the UDP server delivers packets in the manner corresponding to the above state 3. After a period of time, the state can be switched to state 1; if the receiving end still receives a large number of packets during state 3, it enters state 5. During this period, the UDP server delivers packets in the manner corresponding to the above state 5. After a period of time, the state can be switched to state 1.

[0225] It can be seen that by adopting different packet delivery methods in different states, the receiving end can quickly recover to the normal state, ensuring that the receiving end can maintain the speed of processing packets during delay jitter.

[0226] S617. The communication application server sends a data packet to Service 1.

[0227] Among them, after receiving the data packet, Service 1 can perform corresponding processing to meet the user's needs.

[0228] In a possible design, the network detection module 2 may also send the data transmission delay 2 and the network jitter value (which can be referred to as network parameters) to the UDP server. In this way, the UDP server may add the network parameters and the actual caching delay (which can also be referred to as the actual delivery time) corresponding to each data packet to the transmitted completed frame sent to the UDP client. The actual delivery time is the time required for the data packet to be received by the UDP server and then transmitted by the UDP server to the communication application server.

[0229] In this case, still as Figure 6 shown, the data processing method provided by the embodiment of the present application further includes S618~S622.

[0230] S618, the UDP client sends the data processing information and network parameters to the QoE reporting module 1.

[0231] The data processing information includes the actual delivery time and the should-be delivery time corresponding to each data packet. The should-be delivery time of the data packet may be the time interval for the sender to send the data packet.

[0232] It should be noted that each time the UDP client receives a transmitted completed frame, it may send the data processing information and network parameters to the QoE reporting module 1 once.

[0233] S619, the QoE reporting module 1 determines the stuttering level according to the data processing information.

[0234] In a possible design, the QoE reporting module 1 may determine the stuttering level by the ratio of the actual delivery time of the data packet to the should-be delivery time of the data packet.

[0235] For example, the stuttering level includes four levels: no stuttering, slight stuttering, moderate stuttering, and severe stuttering. When the ratio is between 1 and 2, it means that the actual delivery time of the data packet is relatively close to the should-be delivery time of the data packet, and the stuttering level can be determined as no stuttering; when the ratio is between 2 and 3, it means that the actual delivery time of the data packet is slightly greater than the should-be delivery time of the data packet, and the stuttering level can be determined as slight stuttering; when the ratio is between 3 and 4, it means that the actual delivery time of the data packet is greater than the should-be delivery time of the data packet, and the stuttering level can be determined as moderate stuttering; when the ratio is above 4, it means that the actual delivery time of the data packet is much greater than the should-be delivery time of the data packet, and the stuttering level can be determined as severe stuttering.

[0236] It can be understood that the QoE reporting module 1 may also determine the stuttering level through other parameters, such as the number of stuttering times, data transmission delay, stuttering duration, and transmission jitter (which can also be referred to as network jitter situation), etc., which are not specifically limited here.

[0237] S620. When the buffering level of the QoE reporting module 1 meets Condition 1, the buffering information 1 is sent to the communication application client.

[0238] Among them, the buffering level meeting Condition 1 may include: the buffering level changes, or the buffering level always remains at a buffering level other than non-buffered. Among them, the change in the buffering level includes the change of the buffering level from a lighter level to a heavier level, for example, from non-buffered to slightly buffered, from slightly buffered to moderately buffered, etc., and the change of the buffering level from a heavier level to a lighter level, for example, from severely buffered to slightly buffered. The buffering level always remaining at a buffering level other than non-buffered is, for example, the buffering level always being slightly buffered, moderately buffered, or severely buffered.

[0239] In the embodiment of the present application, the buffering information 1 may include information such as the buffering levels before and after the change, the actual delivery time of the data packet, and the above network parameters.

[0240] S621. The communication application client sends the buffering information 1 to Service 1.

[0241] S622. Service 1 adjusts the frame rate or bit rate according to the buffering information 1.

[0242] For example, in the case where the buffering information 1 indicates that the buffering situation becomes more serious, for example, the buffering level changes from non-buffered to moderately buffered, then Service 1 may reduce the frame rate or bit rate. Also, for example, in the case where the buffering information 1 indicates that the buffering situation eases, for example, the buffering level changes from severely buffered to slightly buffered, then Service 1 may increase the frame rate or bit rate.

[0243] It should be noted that when Service 1 adjusts the frame rate, it should combine parameters reflecting the network state such as the network jitter value and data transmission delay carried in the buffering information 1. When the network jitter value is large or the data transmission delay is high, it indicates that the current network state is poor. In this case, Service 1 may not increase the bit rate because increasing the bit rate may further cause data frame congestion, further extend the delivery time of the data frame, and exacerbate the buffering.

[0244] In a possible design, the sending end may also not include the network detection module 1, so that the sending end may not execute S605~S607. The UDP client carries the time information 1 in the transmission completed acknowledgment frame, and the UDP server sends the time information 1 and the time information 2 to the network detection module 2, and the same effect as the above embodiment can be achieved.

[0245] In a possible design, the receiving end may include a QoE reporting module 2. The UDP server may send the actual delivery time and the supposed delivery time corresponding to each data frame to the QoE reporting module 2, and the network detection module 2 may send the data transmission delay 2 and the network jitter value to the QoE reporting module 2. When the QoE reporting module 2 monitors a change in the stuttering level, it sends stuttering information such as the current stuttering level, the data transmission delay 2, and the network jitter value through the UDP server, the UDP client, and the communication application client.

[0246] It can be seen that in the embodiment of the present application, by considering the service requirements and the actual network conditions to set the target cache delay, the target cache delay can meet the cache requirements of different services in different network states, and can better resist the problem of poor service experience caused by network jitter.

[0247] Refer to Figure 11 , which is a schematic flowchart of a data processing method provided by the embodiment of the present application Figure 3 , and specifically describes the process of the sending end and the receiving end closing the session (session) and the big data reporting module reporting service information after closing the session. Among them, the receiving end also includes a QoE reporting module 2. As Figure 11 shown, the data processing method provided by the embodiment of the present application includes S1101~S1113.

[0248] S1101, Service 1 sends a request to close the session to the communication application client.

[0249] In the embodiment of the present application, the request to close the session carries the identification information of Service 1.

[0250] It should be noted that when receiving an interruption operation from the user, such as stopping screen mirroring, shutting down the device, etc., or when all data frames of Service 1 have been transmitted, Service 1 can send a request to close the session to the communication application client.

[0251] In a possible design, Service 1 can call the CloseSession function to pass the request to close the session.

[0252] S1102, The communication application client sends a request to disconnect the channel to the communication application server.

[0253] The request to disconnect the channel is used to indicate disconnecting the channel established between the sending end and the receiving end for transmitting data of Service 1. In the embodiment of the present application, the request to disconnect the channel carries the identification information of Service 1.

[0254] In a possible design, the communication application client can call CloseChannel to pass the request to disconnect the channel.

[0255] S1103, The communication application server sends an instruction to close the session to the UDP server.

[0256] The instruction to close the session is used to indicate the closing of the session of Service 1 between the sender and the receiver. Among them, the instruction to close the session carries the identification information of Service 1.

[0257] In a possible design, the communication application server can call the NSTACKX_DstreamCloseSession function to send the instruction to close the session.

[0258] S1104, The UDP server sends an acknowledgement 1 of closing the session to the communication application server.

[0259] In the embodiment of the present application, in response to receiving the instruction to close the session, the UDP server can release the port corresponding to Service 1 and send an acknowledgement 1 of closing the session to the communication application server. It can be understood that after the UDP server releases the port corresponding to Service 1, the sender cannot send data frames to the receiver according to the previously obtained port number of the receiver.

[0260] In a possible design, the UDP server can send the acknowledgement 1 of closing the session to the communication application server by means of renturn or calling back the NSTACKX_DstreamCloseSession function.

[0261] S1105, The communication application server sends a notification of closing the session to Service 1.

[0262] Thus, the receiver closes the session of Service 1.

[0263] In a possible design, the communication application server can send the notification of closing the session through the onSessionClose function.

[0264] S1106, The communication application server sends a notification of closing the session to the big data reporting module.

[0265] In the embodiment of the present application, the notification of closing the session carries the identification information of Service 1.

[0266] S1107, The big data reporting module sends a query instruction to the QoE reporting module 2.

[0267] It can be understood that when the big data reporting module receives the notification of closing the session, it can determine that the session corresponding to Service 1 has ended. Therefore, it sends a query instruction to the QoE reporting module 2 to query the running status of Service 1.

[0268] S1108, the QoE reporting module 2 sends the freezing information 2 to the big data reporting module.

[0269] Among them, the freezing information 2 may include all freezing data during the operation of service 1, all determined network jitter values, all determined data transmission delays, the time when each data frame is received / submitted, and other information.

[0270] It should be noted that the QoE reporting module 2 can monitor the freezing data in the same way as the QoE reporting module 1 in Figure 6 , record the data transmission delay 2, network jitter value, etc. Therefore, the process of the QoE reporting module 2 obtaining these parameters will not be specifically described here.

[0271] S1109, the big data reporting module uploads the service operation information.

[0272] In a possible design, the big data reporting module can upload the service operation information to the cloud server. The service operation information carries the above-mentioned freezing information 2 and the identification information of service 1, so that the cloud server can perform operations such as statistics and analysis based on the freezing information for different services.

[0273] S1110, the communication application server sends a response to disconnect the channel to the communication application client.

[0274] In the embodiment of the present application, the response to disconnect the channel carries the identification information of service 1.

[0275] It should be noted that S1110 only needs to be executed after S1104, and it has no temporal association with S1105 - S1109.

[0276] S1111, the communication application client sends an instruction to close the session to the UDP client.

[0277] Among them, the instruction to close the session carries the identification information of service 1.

[0278] In a possible design, the communication application client can call the NSTACKX_DstreamCloseSession function to send the instruction to close the session.

[0279] S1112, the UDP client sends an acknowledgement 2 to close the session to the communication application client.

[0280] In a possible design, the UDP client can send the acknowledgement 2 to close the session through the onDisconnect function.

[0281] S1113, the communication application client sends a notification to close the session to service 1.

[0282] In a possible design, the communication application client can send a notification of closing the session through the onSessionClose function.

[0283] At this point, both the sender and the receiver have closed the session of Service 1, and they no longer transmit data frames based on this session.

[0284] The embodiment of the present application also provides a QoE management system, as Figure 12 shown. This QoE management system includes a QoE algorithm module, a dynamic caching algorithm module, a network detection module, a QoE reporting module, a streaming service capability support module,

[0285] Among them, the streaming service capability support module can be used to provide the basic capabilities of streaming services, such as dynamic frame rate, in-order delivery, priority configuration, etc.

[0286] Among them, the network detection module can be used to detect transmission delay, estimate network jitter value, obtain frequency band characteristic information, etc. Then the network detection module can provide parameters such as transmission delay, network jitter value, and frequency band characteristic information to the dynamic caching algorithm module, and the dynamic caching algorithm module can determine the service caching delay and jitter caching delay based on these parameters.

[0287] At the same time, the network detection module can provide parameters such as transmission delay, network jitter, and frequency band characteristic information to the QoE reporting module. The QoE reporting module can determine stuttering metrics such as the number of stutters, transmission jitter, stutter duration, and transmission delay, and determine the stuttering level based on these stuttering metrics. Then, when the stuttering level changes, the QoE reporting module can report the stuttering metrics, and the QoE algorithm module can adjust the bit rate or frame rate.

[0288] The embodiment of the present application also provides a chip system, which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected by a line. For example, the interface circuit can be used to receive signals from other devices (such as the memory of an electronic device). For another example, the interface circuit can be used to send signals to other devices (such as the processor). Exemplarily, the interface circuit can read the instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can execute each step in the above embodiments. Of course, the chip system may also include other discrete devices, and the embodiments of the present application do not make specific limitations on this.

[0289] The embodiment of the present application also provides a computer storage medium, which includes computer instructions. When the computer instructions run on the above electronic device, the electronic device is enabled to execute each function or step performed by the receiver / sender in the above method embodiments.

[0290] The embodiments of the present application also provide a computer program product. When the computer program product runs on a computer, the computer is caused to execute each function or step executed by the receiving end / sending end in the above method embodiments.

[0291] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0292] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0293] The units described as separate components may or may not be physically separated. The components displayed as units may be one physical unit or multiple physical units, that is, they can be located in one place, or they can be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0294] In addition, each functional unit in the various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0295] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0296] The above content is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A data processing method, characterized in that: Applied to a receiving end, the receiving end is wirelessly connected to a transmitting end, and the method includes: receiving a request for establishing a session sent by the sending end, wherein the request for establishing the session carries identification information of the first service; In response to the request to establish a session, establish a first session with the sending end, where the first session is used to transmit data packets of the first service; receiving a data packet transmitted by the sending end based on the first session; Processing a data packet of the first service based on an actual length k of the cache queue and an expected length K of the cache queue; The actual length k of the cache queue is the number of data packets actually stored in the cache queue, the expected length K of the cache queue is the number of data packets expected to be stored in the cache queue, the cache queue is used to store data packets of the first service, and the expected length K of the cache queue satisfies the data transmission delay, service setting delay and jitter cache delay: K=(S yw -S1+S N ) / S,S yw The delay is set for the service, S1 is the data transmission delay, S N is the jitter buffer delay, S is the packet sending interval, the data transmission delay is the time required from the sending end sending the data packet to the receiving end receiving the corresponding data packet, the service setting delay is associated with the first service and the data transmission delay, the jitter buffer delay is related to the network status between the receiving end and the sending end and the jitter degree of the data transmission delay, the jitter buffer delay is calculated based on the network jitter value and the jitter coefficient, the network jitter value is determined based on multiple data transmission delays obtained within a preset time length, the jitter coefficient is determined based on the acquired network information, and the network information is used to indicate the relationship between the frequency and channel of the receiving end and the sending end.

2. The method according to claim 1, characterized in that The method further comprises: In response to receiving a data packet of the first service, sending a transmission completed frame to the transmitting end at a preset time interval; wherein the transmission completed frame carries the number of data packets received by the receiving end within the corresponding time interval and the time when each data packet is received; Receive a transmission completion confirmation frame; wherein the transmission completion confirmation frame is fed back by the transmitting end in response to the transmission completion frame, and the transmission completion confirmation frame carries a waiting time, and the waiting time is the difference between a first moment and a second moment, the first moment is the moment when the transmitting end sends a first data packet, the first data packet is the first data packet received by the receiving end within a corresponding time interval, and the second moment is the moment when the transmitting end receives the transmission completion frame; The round trip delay is calculated according to the third moment and the fourth moment; wherein the third moment is the moment when the receiving end sends the transmission completion frame, and the fourth moment is the moment when the receiving end receives the transmission completion confirmation frame; The data transmission delay is determined according to the waiting time, the time interval and the round-trip delay.

3. The method according to claim 2, characterized in that The data transmission delay, the waiting time, the time interval, and the round-trip delay satisfy the formula: S1=WD-RTT / 2; wherein S1 is the data transmission delay, W is the waiting time, D is the time interval, and RTT is the round-trip delay.

4. The method according to claim 2 or 3, characterized in that: The method further comprises: Acquire a first threshold and a second threshold corresponding to the first service, wherein the first threshold is less than the second threshold; The service setting delay is determined according to a first threshold and a second threshold corresponding to the first service and the data transmission delay.

5. The method according to claim 4, characterized in that The determining the service setting delay according to the first threshold and the second threshold corresponding to the first service and the data transmission delay includes: If the data transmission delay is less than the first threshold, determining the service setting delay to be the first threshold; If the data transmission delay is greater than or equal to the first threshold, the service setting delay is determined to be the second threshold.

6. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Calculate a first parameter according to the network jitter value and the jitter coefficient; If the first parameter is greater than a preset maximum jitter buffer delay value, determine that the jitter buffer delay is a preset maximum jitter buffer delay value; wherein the preset maximum jitter buffer delay value corresponds to the first service; If the first parameter is less than or equal to the packet transmission interval, the jitter buffer delay is determined to be 0; wherein the packet transmission interval is the time interval for the sender to send data packets; If the first parameter is greater than the packet sending interval and less than a preset maximum value of the jitter buffer delay, the jitter buffer delay is indeed the first parameter.

7. The method according to any one of claims 1 to 3, characterized in that: The processing of the data packet of the first service based on the actual length k of the cache queue and the expected length K of the cache queue includes: Based on the actual length k of the cache queue and the expected length K of the cache queue, the processing strategy of the cached data packets in the cache queue is adjusted until K data packets are cached in the cache queue and then the cached data packets in the cache queue are processed according to the packet sending interval.

8. The method according to claim 7, characterized in that The adjusting the processing strategy of the cached data packets in the cache queue based on the actual length k of the cache queue and the expected length K of the cache queue includes: In the case where k<K, the data packets in the cache queue are sent according to the packet sending interval, and the lost data packets are resent; In the case of k>K, sending the data packets in the cache queue according to the target interval or performing packet loss processing on the data packets in the cache queue, and the target interval is smaller than the packet sending interval; In the case of k=0, after the buffer queue buffers a data packet, the corresponding data packet is sent immediately.

9. The method according to claim 4, characterized in that The receiving end includes a communication module server, and the receiving of the data packet transmitted by the sending end based on the first session includes: The communication module server receives the data packet transmitted by the sending end based on the first session; The processing of the data packet of the first service based on the actual length k of the cache queue and the expected length K of the cache queue includes: The communication module server processes the data packet of the first service based on the actual length k of the cache queue and the expected length K of the cache queue.

10. The method according to claim 9, characterized in that The receiving end also includes a network detection module; The step of sending a transmission completed frame to the transmitting end at a preset time interval in response to receiving the data packet of the first service includes: In response to receiving the data packet of the first service, the communication module server sends a transmission completed frame to the sending end at a preset time interval; The received transmission completion confirmation frame includes: The communication module server receives a transmission completion confirmation frame; The method further comprises: The communication module server sends first time information to the network detection module; wherein the first time information includes the waiting time, the first moment and the second moment; The round trip delay is calculated according to the third moment and the fourth moment, comprising: The network detection module calculates the round trip delay according to the third moment and the fourth moment; The determining the data transmission delay according to the waiting time, the time interval and the round-trip delay comprises: The network detection module determines the data transmission delay according to the waiting time, the time interval and the round-trip delay.

11. The method according to claim 10, characterized in that The receiving end also includes a dynamic cache algorithm module; The method further comprises: The network detection module sends the data transmission delay to the dynamic cache algorithm module; The determining the service setting delay according to the first threshold and the second threshold corresponding to the first service and the data transmission delay includes: The dynamic cache algorithm module determines the service setting delay according to a first threshold and a second threshold corresponding to the first service and the data transmission delay.

12. The method according to claim 11, characterized in that The method further comprises: The network detection module determines a network jitter value according to a plurality of data transmission delays obtained within a preset time length; The network detection module sends the network jitter value to the dynamic cache algorithm module; The receiving end also includes a dynamic cache algorithm module; The dynamic cache algorithm module determines a jitter coefficient based on the network information; The dynamic cache algorithm module calculates the jitter cache delay according to the network jitter value and the jitter coefficient.

13. The method according to claim 11, characterized in that Before the dynamic cache algorithm module determines the service setting delay according to the first threshold and the second threshold corresponding to the first service and the data transmission delay, the method further includes: The communication module server receives a request for establishing a channel sent by the sender; wherein the request for establishing the channel carries identification information of the first service; The communication module server obtains first delay information according to the identification information of the first service, where the first delay information includes a first threshold and a second threshold corresponding to the first service; The communication module server obtains the network information; The communication module server sends the first delay information and the network information to the dynamic cache algorithm module.

14. The method according to any one of claims 9 to 13, characterized in that: The processing of the data packet of the first service based on the actual length k of the cache queue and the expected length K of the cache queue includes: The communication module server adjusts the processing strategy of the data packets cached in the cache queue based on the actual length k of the cache queue and the expected length K of the cache queue, until K data packets are cached in the cache queue and then processes the data packets cached in the cache queue according to the packet sending interval.

15. The method according to any one of claims 9 to 13, characterized in that: The receiving end also includes a big data reporting module, and the method also includes: The big data reporting module obtains jamming information, where the jamming information includes the number of jamming times, jamming duration, and data transmission delay during processing of the first service by the receiving end; The big data reporting module sends the freeze information to the cloud server after the receiving end stops processing the first service.

16. A data processing method, characterized in that: Applied to a transmitting end, the transmitting end establishes a wireless connection with a receiving end, and the method includes: Sending a request for establishing a session to the receiving end, wherein the request for establishing the session carries identification information of the first service; receiving a response to establishing a session fed back by the receiving end, and establishing a first session with the receiving end, where the first session is used to transmit data packets of the first service; Sending a data packet to the receiving end based on the first session, so that the receiving end processes the data packet of the first service based on the actual length k of the cache queue and the expected length K of the cache queue; The actual length k of the cache queue is the number of data packets actually stored in the cache queue, the expected length K of the cache queue is the number of data packets expected to be stored in the cache queue, the cache queue is used to store data packets of the first service, and the expected length K of the cache queue satisfies the data transmission delay, service setting delay and jitter cache delay: K=(S yw -S1+S N ) / S,S yw Set the delay for the service, S1 is the data transmission delay, S N is the jitter buffer delay, S is the packet sending interval, the data transmission delay is the time required from the sending end sending the data packet to the receiving end receiving the corresponding data packet, the service setting delay is associated with the first service and the data transmission delay, the jitter buffer delay is related to the network status between the receiving end and the sending end and the jitter degree of the data transmission delay, the jitter buffer delay is calculated based on the network jitter value and the jitter coefficient, the network jitter value is determined based on multiple data transmission delays obtained within a preset time length, the jitter coefficient is determined based on the acquired network information, and the network information is used to indicate the relationship between the frequency and channel of the receiving end and the sending end.

17. The method according to claim 16, characterized in that The transmitting end includes a communication module client, and the method further includes: The communication module client receives the transmission completed frame sent by the receiving end; wherein the transmission completed frame carries the number of data packets received by the receiving end and the time when each data packet is received; The communication module client sends a transmission completion confirmation frame to the receiving end; wherein the transmission completion confirmation frame carries a waiting time, and the waiting time is the difference between a first moment and a second moment, the first moment is the moment when the sending end sends a first data packet, the first data packet is the first data packet received by the receiving end indicated in the transmission completion confirmation frame, and the second moment is the moment when the sending end receives the transmission completion frame.

18. The method according to claim 17, characterized in that The transmitting end further includes a QoE reporting module, and the method further includes: In response to receiving the transmission completed frame, the communication module client sends data processing information and network parameters to the QoE reporting module; The QoE reporting module determines the jam level according to the data processing information; When the jam level changes, the QoE reporting module sends jam information to the first service, where the jam information carries the jam levels before and after the change and the network parameters; In response to receiving the freeze information, the first service adjusts the bit rate or frame rate.

19. An electronic device, characterized in that: The electronic device comprises: a communication module, a memory and one or more processors; the communication module, the memory and the processor are coupled; the memory is used to store computer program code, and the computer program code comprises computer instructions, and when the computer instructions are executed by the electronic device, the electronic device executes the method as described in any one of claims 1-18.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed in an electronic device, the electronic device executes the method according to any one of claims 1 to 18.

21. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 18 are implemented.

Images