Data transmission method, system and related device

Abstract

The present application discloses a method for data transmission, which can be applied in a communication system including a sending end and a receiving end, wherein the sending end can be a server / electronic device and the receiving end can be an electronic device / server. In the process of transmitting data from the server to the electronic device, the server can use a transport layer protocol in a corresponding protocol stack to transmit data according to the data type, and different transport layer protocols can support reliability requirements of different types of data.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a method, system and related device for data transmission. Background Art

[0002] With the rapid development of communication technology, more and more application scenarios involve the real-time transmission of various types of data. For example, cloud gaming, high-definition real-time audio and video such as live broadcasting, and the interaction of multiple virtual reality (VR) devices and augmented reality (AR) devices all involve the transmission of audio and video data, control data, and other types of data. How to ensure the reliability requirements of different types of data transmission to meet the integrity and smoothness of the user experience is an issue that needs to be studied. Summary of the invention

[0003] The present application provides a data transmission method, system and related devices, which can meet the reliability requirements of different types of data during the data transmission process.

[0004] In a first aspect, the present application provides a data transmission method, which is applied at a sending end, and the sending end can determine a first protocol based on a first data type to which first data belongs; the process of transmitting data using the first protocol satisfies a user's reliability requirement for data belonging to the first data type; the first data type is determined based on content provided by the first data and the user's experience of the content; the sending end uses the first protocol to send the first data to a receiving end.

[0005] By implementing the method provided in the first aspect, the sending end can send data using a protocol that matches the reliability requirements for transmission of different types of data according to the user's requirements, thereby meeting user needs and improving user experience.

[0006] In combination with the first aspect, in one embodiment, the above method may further include the following steps: the sending end determines a second protocol based on a second data type to which the second data belongs; the process of transmitting data using the second protocol satisfies a user's reliability requirement for data belonging to the second data type; the second data type is determined based on content provided by the second data and the user's experience of the content; the first data type and the second data type are different; and the sending end uses the second protocol to send the second data to the receiving end.

[0007] In this way, when the sender has multiple data to be sent, different protocols can be used to send the data according to different types of data. Different protocols can support the transmission of data with different reliability requirements, thereby meeting user needs and improving user experience.

[0008] In combination with the first aspect, in one implementation, the first data type is any one of the following: control data, critical audio and video data, non-critical audio and video data, and advertisement data.

[0009] In this way, the sender can divide the data into multiple types according to the content provided by the user and the user's experience of the content. Different types have different reliability requirements, thereby meeting the user's personalized needs.

[0010] In combination with the first aspect, in one embodiment, before the sending end uses the first protocol to send the first data to the receiving end, the method also includes: the sending end places the first data into a first sending queue according to the first data type to which the first data belongs; wherein, when the first data type is control data, the length of the first sending queue is the first value.

[0011] In this way, the sending end can use a shorter sending queue for control data based on the impact of the reliability of control data during transmission on user experience, thereby reducing the queuing delay of control data and improving transmission real-time performance, thereby meeting the user's personalized needs.

[0012] In combination with the first aspect, in one embodiment, before the sending end uses the first protocol to send the first data to the receiving end, the method also includes: the sending end puts the first data into a second sending queue according to the data volume of the first data; when the data volume of the first data is less than the second value, the length of the second sending queue is a third value; when the data volume of the first data is greater than the second value, the length of the second sending queue is a fourth value; the fourth value is greater than the third value, and the third value is greater than the first value.

[0013] In this way, since the transmission delay of data with small data volume is shorter, and the transmission delay of data with large data volume is longer, by placing it in a sending queue corresponding to the data volume, the queuing delay of the data can be reduced, the real-time data transmission can be met, and the user experience can be improved.

[0014] In combination with the first aspect, in one embodiment, before the sending end uses the first protocol to send the first data to the receiving end, the method also includes: the sending end inputs the first data into the first sending queue in a first order according to the first priority of the first data; the correspondence between the first priority and the first order is that the higher the first priority, the higher the priority of the first order.

[0015] In this way, according to the real-time requirements of users for different data, the priority of data with high real-time requirements can be set high, and then the data can be input into the sending queue in order from high to low priority, so that high-priority data can occupy the sending queue, thereby meeting the real-time requirements of users for different data and improving user experience.

[0016] In combination with the first aspect, in one embodiment, before the sending end uses the first protocol to send the first data to the receiving end, the method also includes: specifically including: the sending end outputs the first data in the first sending queue from the first sending queue in a second order according to the second priority of the first sending queue; the corresponding relationship between the second priority and the second order is that the higher the second priority, the higher the priority of the second order.

[0017] In this way, according to the user's real-time requirements for different data, the priority of the sending queue with high real-time requirements can be set high, and then the data can be output from the sending queue in order from high to low priority, so that the data in the high-priority sending queue is sent first, further meeting the user's real-time requirements for different data and improving the user experience.

[0018] In combination with the first aspect, in one implementation, the first data is multiple types of data involved in the real-time audio and video services provided by the sender to the receiver, and the real-time audio and video services include: cloud gaming services, online live broadcast services or VR / AR services.

[0019] In this way, in scenarios involving real-time transmission of multiple types of data, the data transmission provided by itself can be used to improve user experience.

[0020] In the second aspect, the present application provides a data transmission method, which is applied to a communication system including a sending end and a receiving end, the sending end may be an electronic device / server, and the receiving end may be a server / electronic device. In this method, the sending end may determine a first protocol based on a first data type to which the first data belongs; the receiving end may monitor the protocol used by the sending end to send the first data; wherein, the process of transmitting data using the first protocol satisfies the user's reliability requirements for data belonging to the first data type; the first data type is determined based on the content provided by the first data and the user's experience of the content; the sending end uses the first protocol to send the first data to the receiving end.

[0021] By implementing the method provided in the second aspect, the sending end can send data using a protocol that matches the reliability requirements for transmission of different types of data according to the user's requirements, and the receiving end can receive data using a protocol equivalent to the sending end, thereby meeting user needs and improving user experience.

[0022] In combination with the second aspect, in one embodiment, the above method may further include the following steps: the sending end determines a second protocol based on a second data type to which the second data belongs; the process of transmitting data using the second protocol satisfies the user's reliability requirements for data belonging to the second data type; the second data type is determined based on the content provided by the second data and the user's experience of the content; the first data type and the second data type are different; and the sending end uses the second protocol to send the second data to the receiving end.

[0023] In this way, when the sender has multiple data to be sent, different protocols can be used to send the data according to different types of data. Different protocols can support the transmission of data with different reliability requirements, thereby meeting user needs and improving user experience.

[0024] In combination with the second aspect, in one implementation, the first data type is any one of the following: control data, critical audio and video data, non-critical audio and video data, and advertisement data.

[0025] In this way, the sender can divide the data into multiple types according to the content provided by the user and the user's experience of the content. Different types have different reliability requirements, thereby meeting the user's personalized needs.

[0026] In combination with the second aspect, in one embodiment, before the sending end uses the first protocol to send the first data to the receiving end, the method also includes: the sending end places the first data into a first sending queue according to the first data type to which the first data belongs; wherein, when the first data type is control data, the length of the first sending queue is the first value.

[0027] In this way, the sending end can use a shorter sending queue for control data based on the impact of the reliability of control data during transmission on user experience, thereby reducing the queuing delay of control data and improving transmission real-time performance, thereby meeting the user's personalized needs.

[0028] In combination with the second aspect, in one embodiment, before the sending end uses the first protocol to send the first data to the receiving end, the method also includes: the sending end puts the first data into a second sending queue according to the data volume of the first data; when the data volume of the first data is less than the second value, the length of the second sending queue is a third value; when the data volume of the first data is greater than the second value, the length of the second sending queue is a fourth value; the fourth value is greater than the third value, and the third value is greater than the first value.

[0029] In this way, since the transmission delay of data with small data volume is shorter, and the transmission delay of data with large data volume is longer, by placing it in a sending queue corresponding to the data volume, the queuing delay of the data can be reduced, the real-time data transmission can be met, and the user experience can be improved.

[0030] In combination with the second aspect, in one embodiment, before the sending end uses the first protocol to send the first data to the receiving end, the method also includes: the sending end inputs the first data into the first sending queue in a first order according to the first priority of the first data; the correspondence between the first priority and the first order is that the higher the first priority, the higher the priority of the first order.

[0031] In this way, according to the real-time requirements of users for different data, the priority of data with high real-time requirements can be set high, and then the data can be input into the sending queue in order from high to low priority, so that high-priority data can occupy the sending queue, thereby meeting the real-time requirements of users for different data and improving user experience.

[0032] In combination with the second aspect, in one embodiment, before the sending end uses the first protocol to send the first data to the receiving end, the method also includes: specifically including: the sending end outputs the first data in the first sending queue from the first sending queue in a second order according to the second priority of the first sending queue; the corresponding relationship between the second priority and the second order is that the higher the second priority, the higher the priority of the second order.

[0033] In this way, according to the user's real-time requirements for different data, the priority of the sending queue with high real-time requirements can be set high, and then the data can be output from the sending queue in order from high to low priority, so that the data in the high-priority sending queue is sent first, further meeting the user's real-time requirements for different data and improving the user experience.

[0034] In combination with the second aspect, in one implementation, the first data is multiple types of data involved in the real-time audio and video services provided by the sender to the receiver, and the real-time audio and video services include: cloud gaming services, online live broadcast services, or VR / AR services.

[0035] In this way, in scenarios involving real-time transmission of multiple types of data, the data transmission provided by itself can be used to improve user experience.

[0036] In a third aspect, the present application provides an electronic device, comprising one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code comprises computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method described in the implementation manner of the first aspect above.

[0037] In a fourth aspect, the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the method described in the implementation manner of the first aspect above.

[0038] In a fifth aspect, the present application provides a computer-readable storage medium, which includes instructions. When the instructions are executed on an electronic device, the electronic device executes the method described in the implementation of the first aspect above.

[0039] In a sixth aspect, the present application provides a communication system, which includes a transmitting end and a receiving end. When the transmitting end runs on an electronic device, the electronic device executes the method described in the implementation scheme of the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of a communication system 10 provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of the server hardware structure provided in the embodiment of the present application;

[0042] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0044] Figure 5 A flow chart of a data transmission method provided in an embodiment of the present application;

[0045] Figure 6 A schematic diagram of a data transmission strategy provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text only describes the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0047] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0048] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0049] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that the user can recognize. The commonly used form of the user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icon, button, menu, tab, text box, dialog box, status bar, navigation bar, widget, etc. displayed on the display screen of an electronic device. The terms used in the implementation method section of the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0050] At present, in the application scenarios of real-time audio and video transmission, various types of data transmission are involved. For example, in the application scenarios of cloud games, online live broadcasts, multiple virtual reality (VR) devices VR / augmented reality (AR) device interactions, the interaction between the server and the electronic device involves the transmission of various data types, and different types of data have different reliability requirements during the transmission process. For example, control data must ensure reliable transmission, that is, control data cannot be wrong or lost during the transmission process, so as to ensure the normal and orderly operation of the program on the electronic device side. For another example, audio and video data includes: key audio and video data and non-key audio and video data. Key audio and video data needs to be reliably transmitted, that is, key audio and video data cannot be wrong or lost during the transmission process, so as to ensure the integrity of the user experience. Non-key audio and video data can support semi-reliable transmission, that is, non-key audio and video data allows a certain degree of loss or error, so as not to affect the integrity of the user experience, the smoothness of the user experience can be guaranteed. For another example, advertising data can support unreliable transmission, that is, non-key audio and video data can be allowed to be lost or wrong.

[0051] How to meet the different reliability requirements of the above-mentioned different types of data is a problem that needs to be solved at present.

[0052] In some embodiments, the communication field sends data by using a protocol stack, such as a TCP / IP protocol stack or a real-time audio and video data transmission standard protocol (RTP) stack. A protocol stack may include protocols used by each layer in a network model, such as a TCP / IP protocol stack may include a HyperText Transfer Protocol (HTTP) or a File Transfer Protocol (FTP) commonly used in an application layer, a Transmission Control Protocol (TCP) or a User Datagram Protocol (UDP) commonly used in a transport layer, an Internet Protocol (IP) commonly used in a network layer, and the like.

[0053] When a server provides a service to an electronic device, such as the cloud games, live streaming, and multi-VR / AR device interaction mentioned above, the same protocol is used at each layer for transmission of all data involved in the above services. For example, in the transport layer, data can only use TCP or UDP in the TCP / IP protocol stack, or RTP in the RTP protocol stack. However, there are many types of data involved in the above services, and different types of data have different transmission reliability requirements. Therefore, using only one transport layer protocol, TCP, UDP, or RTP, to transmit data will have the following defects:

[0054] The above TCP, UDP or RTP can only support one level of reliable data transmission, TCP can only support reliable transmission, UDP can only support unreliable transmission, and RTP can only support unreliable transmission. Therefore, using TCP, UDP or RTP in the transport layer has the following defects:

[0055] 1. The reliability requirements of different types of data cannot be guaranteed.

[0056] Since the above-mentioned TCP, UDP or RTP can only support one type of data transmission with reliability, specifically, TCP can only support reliable transmission, UDP can only support unreliable transmission, and RTP can only support unreliable transmission. When any one of these protocols is used to transmit data, only one level of reliable data transmission can be achieved. In the data transmission process, multiple data types of different types will be involved, such as control data, critical audio and video data, non-critical audio and video data, and advertising data. Users have different reliability requirements for different types of data transmission, so the TCP / IP protocol stack or RTP protocol stack that simply uses the protocol cannot guarantee the reliability requirements of different types of data.

[0057] 2. The real-time nature of data transmission cannot be guaranteed.

[0058] When the server sends data to an electronic device, the server needs to first put the data into a sending queue, and then output it from the sending queue and send it to the electronic device. Since the sending queue is one or more fixed-length queues when the above-mentioned TCP, UDP or RTP is used to transmit data, all data is sent to the electronic device in the order of first-in, first-out in the sending queue. When all the sending queues have no remaining cached data, if the server has a new data request to send, the new data needs to queue up and wait to enter the sending queue. No matter which sending queue it is waiting in, the queuing delay is the same, so the real-time data transmission cannot be guaranteed.

[0059] In order to solve the defects of the above-mentioned data transmission method, an embodiment of the present application provides a method for data transmission, which can be applied in a communication system including a sending end and a receiving end, wherein the sending end can be a server / electronic device and the receiving end can be an electronic device / server. The embodiment of the present application takes the server as the sending end and the electronic device as the receiving end as an example to introduce the method.

[0060] In this method, when the server transmits data to the electronic device, the server can use the transport layer protocol in the corresponding protocol stack to transmit data according to the data type, and different transport layer protocols can support the reliability requirements of different types of data.

[0061] Furthermore, the server can also determine the different sending queues to be used when using different protocol stacks for data transmission based on the data type and / or data volume. The lengths of different sending queues are different, and the queuing delays of different sending queues are different, thereby ensuring the real-time transmission of data of different data types and / or data volumes.

[0062] The explanations of the nouns and terms involved in the embodiments of the present application are as follows:

[0063] In an embodiment of the present application, data types may include: control data, key audio and video data, non-key audio and video data, and advertising data. Among them, control data refers to data used to specify the execution order, operation process, etc. of various functions executed during program operation. Audio and video data refers to data used to provide audio and video content to users. Among them, key audio and video data refers to the audio and video content provided by this type of data is the content that the user focuses on, while the audio and video content provided by non-key audio and video data is the content that the user does not pay much attention to. Advertising data refers to data provided by advertisers to promote a brand or a service to increase its popularity. This type of data is usually content that users are not interested in. The above-mentioned data types are only examples. In specific implementations, embodiments of the present application may also include more data types.

[0064] In the embodiment of the present application, users have different transmission reliability requirements for different types of data. For example, the reliability requirements for different types of data refer to:

[0065] Users want high reliability requirements for control data and key audio and video data; medium reliability requirements for non-key audio and video data; and low reliability requirements for advertising data. This is determined by the degree of impact of the reliability of different types of data during transmission on the user experience. For example, the reliability requirements of control data during transmission have a greater impact on the user experience. This is because when the reliability of control data transmission is low, if the data is lost or erroneous, the user will clearly perceive the abnormality of the currently running program on the electronic device side, such as the user operation cannot be responded to, which greatly affects the user experience. For another example, if key audio and video data is lost or erroneous, the user will clearly perceive the audio and video being watched on the electronic device side as being stuck or reporting errors, which greatly affects the user experience. For another example, if non-key audio and video data is lost or erroneous, even if the non-key audio and video content watched by the user is stuck or cannot be watched, it has little impact on the user experience. For another example, if advertising data is lost or erroneous, even if the advertisement watched by the user is stuck or cannot be watched, it has no impact on the user experience.

[0066] In the embodiment of the present application, the transport layer protocol may include multiple protocols supporting different reliabilities. Exemplarily, the protocols supporting different reliabilities may include: a reliable transmission protocol supporting reliable transmission in protocol stack 1, a semi-reliable transmission protocol supporting semi-reliable transmission in protocol stack 2, and an unreliable transmission protocol 3 supporting unreliable transmission in protocol stack 3, etc. Among them, the principles of the above three protocol stacks supporting different reliability requirements are as follows:

[0067] In protocol stack 1, the reliable transmission protocol is TCP used by the transport layer to implement reliable transmission. Specifically, TCP itself provides a reliable transmission mechanism because TCP mainly implements reliable transmission through methods such as checksum, sequence number / confirmation response, timeout retransmission, maximum message size, and sliding window control.

[0068] This not only ensures the quality of control data transmission, so that the application can provide users with application experience in a normal and orderly manner; it also ensures the quality of key audio and video data transmission, so that the application can correctly play key audio and video content and ensure user experience.

[0069] In the protocol stack 2, the semi-reliable transmission protocol is a protocol that uses UDP in the transport layer and combines a semi-reliable mechanism to achieve semi-reliable transmission. Specifically, UDP itself does not provide any reliable transmission mechanism, but can be combined with a semi-reliable transmission mechanism to achieve semi-reliable transmission. Semi-reliable transmission mechanisms may include, but are not limited to, serial number / confirmation response, timeout retransmission and other mechanisms. Among them, serial number / confirmation response means that the sending end, such as the server 100, sends data to the receiving end, such as the application at the electronic device 200. After the receiving end receives / does not receive the data, it returns an acknowledgement response (Acknowledge character, ACK) packet to the sending end, and the ACK packet is used to indicate whether the receiving end has received the data. For example. When the sending end sends data containing serial numbers 1-500, if the receiving end only receives data with serial numbers 1-400, the receiving end returns an ACK packet to the sending end, and the ACK packet is used to remind the sending end to resend the data with serial numbers 401-500. Among them, timeout retransmission means that when the sending end has not received the ACK packet for more than the threshold time, the data is resent. In the embodiment of the present application, the protocol stack 2 can pre-set a number of data retransmissions to achieve semi-reliable transmission. That is, when the number of retransmissions of a certain data is less than a specified threshold, it is retransmitted; however, when the number of retransmissions of a certain data exceeds the threshold, it is directly discarded and no longer retransmitted.

[0070] In this way, a semi-reliable transmission protocol with a specified number of retransmissions is used to ensure the transmission quality of non-critical audio and video data to a certain extent, and it can also avoid repeated retransmissions when non-critical data is lost, thereby causing delays in other transmitted data.

[0071] In protocol stack 3, the unreliable transmission protocol is UDP. UDP itself does not provide any reliable transmission mechanism. If the transmitted data is lost or erroneous, the sender does not retransmit and the receiver does not perform error correction, thus achieving unreliable transmission.

[0072] Since the loss of advertising data has almost no impact on the user experience, an unreliable transmission protocol is used to transmit advertising data. There is no situation where other transmitted data is delayed due to the retransmission of advertising data, which can effectively ensure the smoothness of the user experience.

[0073] In some embodiments, the transport layer protocols in each protocol stack correspond to a sending queue respectively, and the sending queue corresponding to the transport layer protocol in a protocol stack may include: one or more sending queues, and the lengths of different sending queues are different. For example, the types of sending queues may include, but are not limited to: low-latency queues, high-throughput queues, and other queues. The low-latency queue has the smallest length, and from small to large are other queues and high-throughput queues. The other queues in the embodiments of the present application are just a noun and have no other meanings. They can also be referred to as non-low-latency and non-high-throughput queues.

[0074] In an embodiment of the present application, the low-latency queue may be referred to as the first queue, and the high-throughput queue and other queues may be referred to as the second queue.

[0075] In the embodiment of the present application, the transmission delay includes: queuing delay and the delay of data transmission in the communication link. The queuing delay refers to the time it takes for data to be sent from the cache in the sending queue. The queuing delay can also be called processing delay. For example, when the bandwidth = 10Mbps, the sending queue can cache a maximum of 4 megabytes (MB) of data. If 4MB of audio and video data has been cached at this time, that is, 32 megabits (Mbit), the queuing delay is: 32Mbit / 10Mbps = 3.2s.

[0076] It can be seen that after implementing the data transmission method provided in the embodiment of the present application, the following technical problems can be solved:

[0077] During data transmission, the sender can use the transport layer protocol in the protocol stack that matches the reliability requirements of the data type to transmit data according to the data type, thereby meeting the reliability requirements of different types of data.

[0078] Furthermore, data can be sent using a sending queue corresponding to the data type / data amount, with different sending queues having different queuing delays, thereby satisfying the real-time nature of data of different data types / data amounts and user experience.

[0079] The data transmission method provided in the embodiments of the present application can be applied to all scenarios involving real-time transmission of various types of data, such as cloud gaming, online live broadcasting, and multi-VR / AR device interaction. The following introduces several typical application scenarios in which the data transmission method can be applied:

[0080] 1. Cloud gaming means that all games are run on the server side, and the server side compresses the rendered game content and transmits it to the electronic device through the network. Specifically, the user first opens the cloud gaming application on the electronic device, selects the game he wants to experience, and then the electronic device can establish a connection with the server that provides game services for the game selected by the user. At this time, a large amount of data will be transmitted between the server and the electronic device, mainly including control data and audio and video data. Among them, the control data is responsible for sending control signals to control the normal and orderly operation of the cloud gaming application, while the audio and video data provides audio and video content for the cloud gaming application. It is worth noting that the above data types not only include audio and video data related to the game, but also include advertising audio and video data placed by advertisers on the cloud gaming platform.

[0081] 2. Live streaming means that after the signal acquisition device collects audio and video data, it sends these audio and video data to the server. The user can open the live streaming platform in the electronic device, select the live content, establish a connection with the server that provides the live content, and then watch the live content. Take the process of users establishing a connection with the server through electronic devices and watching the content of live streaming as an example. This process involves the transmission of multiple types of data, including audio and video data, control data and advertising data. Among them, audio and video data are obtained by the server from the signal acquisition device, and control data can include the normal and orderly operation of the live streaming platform on the electronic device side.

[0082] 3. Interaction of multiple VR / AR devices means that, taking two VR / AR devices as an example, the two VR / AR devices can establish a communication connection through a server that provides VR / AR services for them. Based on the above communication connection, the two VR / AR devices can transmit the operation instructions input by the users of each VR / AR device end, as well as the audio and video data of the VR / AR game content to each other, so as to realize the experience of VR / AR games by multiple people. In this process, various types of data will be involved in the transmission, including audio and video data and control data. Among them, the control data is responsible for sending control signals to control the normal and orderly operation of the VR / AR device application, and the audio and video data is used to provide game content for the VR / AR device.

[0083] Therefore, the following embodiments of the present application only take the process of a server providing cloud games, live broadcasts, or VR / AR games to electronic devices as an example to introduce the data transmission method provided by the embodiments of the present application. However, the data transmission method provided by the embodiments of the present application is not limited to the above-mentioned scenarios, and the data transmission method can be applied to other scenarios including multiple types of data transmission.

[0084] In order to introduce the prompt method provided in the embodiment of the present application more clearly and in detail, the communication system 10 provided in the embodiment of the present application is first introduced below.

[0085] refer to Figure 1 , Figure 1 The communication system 10 provided in an embodiment of the present application is exemplarily shown.

[0086] like Figure 1 As shown, the communication system 10 includes a server 100 and an electronic device 200 .

[0087] The server 100 refers to providing computing or application services for other clients such as the electronic device 200. In an embodiment of the present application, the server 100 may be a cloud gaming server that stores game content and can run the game, and the server 100 may send the running game content to the electronic device 200 in real time. Alternatively, the server 100 may also be a server for storing live content sent by a signal acquisition device, and the server 100 may send the live content in real time to an electronic device that plays the live broadcast. Alternatively, the server 100 may also be a server that provides services for VR / AR devices, and the server 100 may store VR / AR content and send it to the electronic device 200. The above-mentioned server 100 may be provided by game and live broadcast platform developers, suppliers, etc. The number of servers 100 may be one or more.

[0088] The electronic device 200 may be a device equipped with Or electronic devices with other operating systems, such as mobile phones, televisions, tablet computers, desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (UMPC), netbooks, as well as cellular phones, personal digital assistants (PDA), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices, vehicle-mounted devices, smart home devices and / or smart city devices. The embodiments of the present application do not impose any special restrictions on the specific types of the electronic devices.

[0089] In an embodiment of the present application, the server 100 can establish a communication connection with the electronic device 200, and perform data transmission based on the communication connection. Specifically, both the server 100 and the electronic device 200 are deployed with a dynamic configuration module. Taking the electronic device 200 as an example, the dynamic configuration module can be deployed in the user state, independent of the operating system kernel. Taking the server 100 as the sending end and the electronic device 200 as the receiving end as an example, when the server 100 transmits data to the electronic device 200, the dynamic configuration module in the server 100 can determine the protocol stack used in the transmission process according to the data transmission strategy stored therein. At the same time, the electronic device 200 can monitor the protocol stack used by the server 100, and call the dynamic configuration module in the electronic device 200, and use a peer-to-peer protocol stack to receive data.

[0090] Figure 2The hardware structure of the server 100 provided in the embodiment of the present application is shown.

[0091] like Figure 2 As shown, the server 100 may include: one or more processors 301, a memory 302, a communication interface 303, a transmitter 305, a receiver 306, a coupler 307 and an antenna 308. These components may be connected via a bus 304 or other means. Figure 2 Take bus connection as an example.

[0092] The processor 301 can be used to read and execute computer-readable instructions. Specifically, the processor 301 can be used to call a program stored in the memory 302. In an embodiment of the present application, the processor 301 can be used to call a method stored in the memory 302, using a protocol stack that matches the data to be transmitted and sending it to the electronic device 200, and implementing a program on the server 100 side, and executing the instructions contained in the program.

[0093] The memory 302 is coupled to the processor 301 and is used to store various software programs and / or multiple groups of instructions. Specifically, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state storage devices. The memory 302 can store an operating system (hereinafter referred to as the system), such as an embedded operating system such as uCOS, VxWorks, RTLinux, etc. The memory 302 can also store a network communication program, which can be used to communicate with the electronic device 200, etc. In an embodiment of the present application, the memory 302 can store a data transmission strategy for the dynamic configuration module in the server 100 to use a protocol stack matching the data to be transmitted for data transmission according to the data transmission strategy.

[0094] The communication interface 303 can be used for the server 100 to communicate with other communication devices, such as the electronic device 200. Specifically, the communication interface 303 can be a 3G communication interface, a long-term evolution (LTE) (4G) communication interface, a 5G communication interface, a WLAN communication interface, a WAN communication interface, and the like. Not limited to a wireless communication interface, the server 100 can also be configured with a wired communication interface 303 to support wired communication. In an embodiment of the present application, the server 100 can establish a communication connection with the electronic device 200 through the communication interface 203, and perform data transmission based on the communication connection.

[0095] In some embodiments of the present application, the transmitter 305 and the receiver 306 can be regarded as a wireless modem. The transmitter 305 can be used to transmit and process the signal output by the processor 301. The receiver 306 can be used to receive signals. In the server 100, the number of transmitters 305 and receivers 306 can be one or more. The antenna 308 can be used to convert electromagnetic energy in the transmission line into electromagnetic waves in free space, or to convert electromagnetic waves in free space into electromagnetic energy in the transmission line. The coupler 307 can be used to divide the mobile communication signal into multiple paths and distribute it to multiple receivers 306. It can be understood that the antenna 308 of the network device can be implemented as a large-scale antenna array.

[0096] Need to explain, Figure 2 The server 100 shown is only one implementation of the embodiment of the present application. In actual applications, the server 100 may also include more or fewer components, which is not limited here.

[0097] Figure 3 A schematic diagram of the hardware structure of the electronic device 200 is shown.

[0098] The electronic device 200 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0099] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 200. In other embodiments of the present application, the electronic device 200 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0100] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors. The controller may generate an operation control signal according to an instruction opcode and a timing signal to complete the control of fetching and executing instructions.

[0101] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0102] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0103] In the embodiment of the present application, the processor 110 can call the tracking application to obtain the area information that the user has arrived at and the retention time in the area. When the tracking application receives the area information that the target user has arrived at and the retention time in the area sent by the server 300, the processor 110 can compare the received area information and retention time with the area information and retention time that the user has arrived at obtained by itself, and then determine the contact result between the user and the target user, and call the display screen 194 to display the prompt information corresponding to the contact result. For details, please refer to the relevant description of the method embodiment and UI embodiment below, which will not be repeated here.

[0104] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 200.

[0105] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0106] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0107] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0108] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to realize the shooting function of the electronic device 200. The processor 110 and the display screen 194 communicate via the DSI interface to realize the display function of the electronic device 200.

[0109] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0110] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 200, and can also be used to transfer data between the electronic device 200 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0111] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 200. In other embodiments of the present application, the electronic device 200 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0112] In an embodiment of the present application, the processor 110 can call a dynamic configuration module in the electronic device 200 to receive data using a protocol stack that is equivalent to the protocol stack used by the server 100 to send data.

[0113] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 200. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0114] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0115] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.

[0116] Non-volatile memory may include disk storage devices and flash memory.

[0117] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0118] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0119] In the embodiment of the present application, the internal memory 121 can be used to store programs, such as the implementation program of the data transmission method provided in the embodiment of the present application on the electronic device 200 side. The internal memory 121 can also store a data transmission strategy, which can be preset in advance or configured through the cloud and then sent to the dynamic configuration module in the electronic device 200. For a specific description of the data transmission strategy, please refer to the relevant description of the method embodiment below, which will not be repeated here.

[0120] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 200. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0121] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0122] The wireless communication function of the electronic device 200 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0123] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 200 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0124] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 200. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0125] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0126] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 200. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering processing, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0127] In some embodiments, the antenna 1 of the electronic device 200 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 200 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0128] In the embodiment of the present application, the electronic device 200 can establish a communication connection with the server 100 through the mobile communication module 150 or the wireless communication module 160, and negotiate the protocol stack used for data transmission according to the communication connection, and receive data using the protocol stack used when sending data to the server 100. The specific implementation method of data transmission can refer to the detailed description of the method embodiment below, which is not repeated here.

[0129] The electronic device 200 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0130] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 200 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0131] In the embodiment of the present application, the display screen 194 can be used to display a page provided by an application that provides services such as cloud gaming, live webcasting, VR / AR, etc. The page may include some video content.

[0132] The electronic device 200 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0133] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0134] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 200 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0135] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 200 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0136] Video codecs are used to compress or decompress digital videos. The electronic device 200 may support one or more video codecs. Thus, the electronic device 200 may play or record videos in a variety of coding formats, such as moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0137] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the electronic device 200 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0138] The electronic device 200 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0139] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0140] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 200 can listen to music or listen to a hands-free call through the speaker 170A.

[0141] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 200 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0142] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 200 can be provided with at least one microphone 170C. In other embodiments, the electronic device 200 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 200 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0143] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0144] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 200 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 200 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 200 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0145] The gyro sensor 180B can be used to determine the motion posture of the electronic device 200. In some embodiments, the angular velocity of the electronic device 200 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 200 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 200 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0146] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 200 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0147] The magnetic sensor 180D includes a Hall sensor. The electronic device 200 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 200 is a flip phone, the electronic device 200 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover automatic unlocking and other features are set.

[0148] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 200 in all directions (generally three axes). When the electronic device 200 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0149] The distance sensor 180F is used to measure the distance. The electronic device 200 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 200 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0150] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 200 emits infrared light outward through the light emitting diode. The electronic device 200 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 200. When insufficient reflected light is detected, the electronic device 200 can determine that there is no object near the electronic device 200. The electronic device 200 can use the proximity light sensor 180G to detect that the user holds the electronic device 200 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0151] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 200 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 200 is in a pocket to prevent accidental touch.

[0152] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 200 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0153] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 200 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 200 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 200 heats the battery 142 to avoid abnormal shutdown of the electronic device 200 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 200 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0154] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 200, which is different from the position of the display screen 194.

[0155] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0156] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 200 may receive key input and generate key signal input related to user settings and function control of the electronic device 200.

[0157] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0158] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0159] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 200 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 200 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 200 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 200 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 200 and cannot be separated from the electronic device 200.

[0160] The software system of the electronic device 200 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device 200 is exemplified.

[0161] Figure 4 It is a software structure block diagram of the electronic device 200 according to an embodiment of the present invention.

[0162] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. The system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime layer and system libraries, as well as the kernel layer.

[0163] The application layer can include a series of application packages.

[0164] like Figure 4 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, as well as cloud game, live broadcast, VR / AR applications.

[0165] Among them, the functions of cloud games, live streaming, and VR / AR applications can be referred to the detailed description above and will not be elaborated here.

[0166] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0167] like Figure 4 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0168] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0169] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0170] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0171] The phone manager is used to provide communication functions of the electronic device 200, such as management of call status (including connecting, hanging up, etc.).

[0172] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0173] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0174] Runtime includes core libraries and virtual machines. The runtime is responsible for scheduling and management of the Android system.

[0175] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0176] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0177] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0178] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0179] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0180] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0181] A 2D graphics engine is a drawing engine for 2D drawings.

[0182] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0183] The following is an illustrative description of the workflow of the software and hardware of the electronic device 200 in conjunction with capturing a photo scene.

[0184] When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture static images or videos through the camera 193.

[0185] Figure 5 The data transmission method provided by the embodiment of the present application is exemplified. Figure 5 The data transmission method provided in the embodiment of the present application is described in detail.

[0186] It can be understood that in the data transmission process in cloud gaming, live streaming and multi-VR / AR interactive scenarios, a large amount of audio and video data and control data is usually sent from the server to the client to provide application services for the client. Therefore, the following embodiments of this application take the server 100 as the sending end and the electronic device 200 as the receiving end as an example to introduce the data transmission method.

[0187] In some other embodiments, the server 100 can also be used as a receiving end, and the electronic device 200 can also be used as a sending end. For example, in the pre-process of a user watching a live broadcast on the Internet, the signal acquisition device needs to collect audio and video content and transmit the audio and video content to the server. In this process, the signal acquisition device, that is, the electronic device 200, can be regarded as a sending end, and the server 100 can be regarded as a receiving end. At this time, the method flow of the electronic device 200 sending data to the server 100 is the same as the method flow of the server 100 sending data to the electronic device 200. Please refer to the detailed description below for details.

[0188] S101, the electronic device 200 detects a user's operation of launching an application.

[0189] Specifically, the electronic device 200 can detect the user's operation of launching an application, which may be, for example, a cloud game, a live webcast, or a VR / AR application.

[0190] It will be understood that, not limited to the above-mentioned cloud games, live streaming, VR / AR applications, electronic devices can detect that users have started other applications that require real-time transmission of various types of data with other devices such as servers, and the embodiments of the present application do not limit this.

[0191] S102, the server 100 establishes a communication connection with the electronic device 200.

[0192] Specifically, in response to the user starting the application on the electronic device 200, the server 100 may establish a communication connection with the electronic device 200. The communication connection here refers to an upper layer communication connection above the transport layer established between the server 100 and the electronic device 200.

[0193] In the embodiment of the present application, the communication standard for data transmission between the server 100 and the electronic device 200 may be based on the Open System Interconnection (OSI) model, or a five-layer network model or a four-layer network model evolved from OSI, and the embodiment of the present application does not limit this. Taking the five-layer network model or the four-layer network model as an example, the above communication connection may be an application layer connection between the electronic device 200 and the Hypertext Transfer Protocol (HTTP) or the File Transfer Protocol (FTP).

[0194] S103, the electronic device 200 monitors the protocol used by the server 100 to send data.

[0195] Specifically, after the server 100 and the electronic device 200 establish the communication connection described in S102 above, the electronic device 200 can open the corresponding port to continuously listen to the protocol used by the server 100 to send data. The protocol here refers to the transport layer protocol. Then, the electronic device 200 can call the dynamic configuration module on this side to use a peer protocol to receive data. For details, see step S108 below.

[0196] S104, the server 100 determines the protocol used for the data to be sent.

[0197] Specifically, the server 100 can call the API interface of the dynamic configuration module to send the data to be sent to the application on the electronic device 200 to the dynamic configuration module in the server 100. The sent data can be one or more data blocks with different data volumes. Afterwards, the dynamic configuration module can determine the protocol used for the data to be sent according to the data transmission strategy.

[0198] In the embodiment of the present application, the first data and the second data refer to data to be sent, but the reliability requirements of the first data and the second data are different.

[0199] refer to Figure 6 , Figure 6 A schematic diagram of an exemplary data transmission strategy is shown. The above-mentioned data transmission strategy includes: the correspondence between the data type and the protocol stack, and the correspondence between the data type / data volume and the sending queue. Specifically, control data and critical audio and video data use the reliable transmission protocol in protocol stack 1, non-critical audio and video data use the semi-reliable transmission protocol in protocol stack 2, and advertising data uses the unreliable transmission protocol in protocol stack 3. For the correspondence between the data type / data volume and the sending queue, please refer to the detailed description of step S105 below, which will not be repeated here.

[0200] Understandably, Figure 6 The schematic diagram of the data transmission strategy shown is only an example. In the embodiment of the present application, the data transmission strategy may include Figure 6 In addition to the three protocol stacks shown, different protocol stacks can support different reliability of data transmission. When data transmission uses different protocol stacks, the types of sending queues used are not limited to Figure 6 The three types are shown, and the types and quantities of sending queues corresponding to different protocol stacks can be the same or different.

[0201] In an embodiment of the present application, the first protocol may be any one of the above-mentioned reliable transmission protocol, semi-reliable transmission protocol or unreliable transmission protocol, and the first protocol may also be any one of the above-mentioned reliable transmission protocol, semi-reliable transmission protocol or unreliable transmission protocol, but it is worth noting that the first protocol and the second protocol are not the same.

[0202] The dynamic configuration module of the server 100 determines the protocol used by the data to be sent, specifically including the following steps:

[0203] 1. The dynamic configuration module determines the data type.

[0204] The server 100 can directly specify the type of data to be sent. For example, an identifier can be added to the data, which is used to distinguish the data type. When the dynamic configuration module receives the data, the data type can be identified by the identifier. In an embodiment of the present application, the data type may include but is not limited to: control data, key audio and video data, non-key audio and video data, advertising data, etc. Among them, control data is used to determine the operations and order of operations performed by the application, key audio and video data and non-key audio and video data are used to display and play audio and video content in the application, and advertising data is used to display and play advertisements in the application. In some embodiments, the data type may also include more or fewer types, which can be specifically determined based on the application services provided by the server 100, and the embodiment of the present application does not limit this.

[0205] 2. The dynamic configuration module determines the protocol used for the data to be sent based on the data type.

[0206] Specifically, since different types of data have different requirements for the reliability of data transmission, the dynamic configuration module can determine the protocol stack used for data transmission that can support different reliability requirements according to the data type.

[0207] The embodiment of the present application provides three protocol stacks that can support different reliability requirements: a reliable transmission protocol that supports reliable transmission, a semi-reliable transmission protocol that supports semi-reliable transmission, and an unreliable transmission protocol that supports unreliable transmission. Therefore, the dynamic configuration module can determine, based on the data type, that control data and key audio and video data can use a reliable transmission protocol for data transmission; non-key audio and video data can use a semi-reliable transmission protocol for data transmission; and advertising data can use an unreliable transmission protocol for data transmission.

[0208] Among them, the specific principles of the above three protocol stacks supporting different reliability requirements can be referred to the detailed description above, which will not be repeated here.

[0209] Optionally, in some embodiments of the present application, in addition to determining the protocol stack used for data transmission according to the data type, the server 100 can also determine the protocol stack required for different data transmissions in other ways. For example, the server 100 can evaluate the quality of experience (QoE) requirements of the data to be sent, and the server 100 can pre-configure a QoE threshold. When the QoE is greater than the QoE threshold, the dynamic configuration module uses a reliable transmission protocol to transmit the data; when the QoE is less than the QoE threshold, the dynamic configuration module uses protocol stack 3 to transmit the data; when the QoE is equal to the QoE threshold, the dynamic configuration module uses a semi-reliable transmission protocol to transmit the data. Among them, QoE is the user's subjective feeling based on the service provided by the server 100 during the interaction between the application of the electronic device 200 and the server 100. It is a comprehensive evaluation parameter, for example, it can include reliability requirements for data transmission.

[0210] In this way, according to the reliability requirements of different types of data, a protocol stack that meets the reliability requirements can be used for data transmission, thereby meeting user needs and improving user experience.

[0211] S105, the server 100 determines a sending queue corresponding to a protocol used by the data to be sent.

[0212] Specifically, the dynamic configuration module of the server 100 may determine the corresponding sending queue in the protocol adopted by the data to be sent according to the data transmission strategy.

[0213] The corresponding relationship between the data type / data volume and the sending queue in the above data transmission strategy includes: control data corresponds to the low-latency queue in the reliable transmission protocol, and the length of the low-latency queue is the first value; key audio and video data with a data volume less than the second value corresponds to other queues in the reliable transmission protocol, and the length of the other queues is the third value; key audio and video data with a data volume greater than the second value corresponds to the high-throughput queue in the reliable transmission protocol, and the length of the high-throughput queue is the fourth value. The data transmission strategy can be preset in the dynamic configuration module in the server 100 and the electronic device 200, or it can be formulated by the cloud and sent to the electronic device 200.

[0214] The dynamic configuration module of the server 100 determines the sending queue used for data transmission according to the data transmission strategy, which specifically includes the following steps:

[0215] 1. The dynamic configuration module determines the data type and / or data volume of the data to be sent.

[0216] Since the data to be sent may include multiple data blocks, the data volume of different data blocks is different, and the transmission delay is different, so the dynamic configuration module can calculate the data volume of the data to be sent, and then determine the sending queue used in the protocol stack used for data transmission based on the data volume. It is worth noting that when the data to be sent is control data, the dynamic configuration module does not calculate the data volume of the control data.

[0217] 2. The dynamic configuration module determines different sending queues used when using different protocol stacks for data transmission according to the data type and / or data volume.

[0218] First, the dynamic configuration module can set in advance different types of sending queues used when different protocol stacks are used for data transmission. For example, the types of sending queues may include one or more of the following: low-latency queues, high-throughput queues, and other queues. Different types of sending queues have different maximum amounts of data that can be cached. The length of the low-latency queue can be set to a first value, such as 4 kilobytes (KB), the length of other queues can be set to a third value, 400KB, and the length of the high-throughput queue can be set to a fourth value, such as 4MB. Among them, the maximum amount of data that can be cached by different types of sending queues can be, and the dynamic configuration module can set in advance the maximum amount of data that can be cached by different types of sending queues according to the amount of data that often appears during data transmission, and the number of different types of sending queues can be one or more.

[0219] When the dynamic configuration module determines to use a reliable transmission protocol to transmit control data and key audio and video data, the control data uses a low-latency queue for data transmission, the key audio and video data with a data volume greater than or equal to the second value, such as 1MB, uses a high-throughput queue for data transmission, and the key audio and video data with a data volume less than the second value, such as 1MB, uses other queues for data transmission. It should be noted that when the server 100 uses a reliable transmission protocol to transmit data, since both the control data and the key audio and video data use a reliable transmission protocol for data transmission, the dynamic configuration module can directly default to using a low-latency queue for the control data, while the key audio and video data needs to use the corresponding sending queue according to the measured data volume.

[0220] When the dynamic configuration module determines to use a semi-reliable transmission protocol to transmit non-critical audio and video data, data with a data volume less than a second value, such as 1MB, uses a low-latency queue for data transmission, and data with a data volume greater than or equal to the second value, such as 1MB, uses a high-throughput queue for data transmission.

[0221] When the dynamic configuration module determines to use an unreliable transmission protocol to transmit advertising data, data with a data volume less than a second value, such as 1MB, uses a low-latency queue for data transmission, and data with a data volume greater than or equal to the second value, such as 1MB, uses a high-throughput queue for data transmission.

[0222] Optionally, in other embodiments of the present application, the dynamic configuration module can dynamically set the sending queues of different sizes according to the real-time requirement of the data to be sent. For example, when the bandwidth is 20Mbps, if the real-time requirement of the data to be sent is 10ms, then the dynamic configuration module can set the size of the sending queue to 200Kbit, and if the real-time requirement of the data to be sent is 100ms, then the size of the queue is 2Mbit.

[0223] In this way, the dynamic configuration module of the server 100 can determine, based on the transmission delay requirement of the data, that data of different data volumes use the sending queue that best matches its delay requirement, thereby meeting user needs.

[0224] When the network is congested, the sending queue is fully used to cache data. At this time, if there is a request to send audio and video data or control data with a data volume less than the second value, the critical audio and video data or control data with a data volume less than the second value can queue up and wait in the corresponding sending queue, that is, the control data queues for the low-latency queue, and the critical audio and video data with a data volume less than the second value queues for other queues. Compared with the fixed 4MB length sending queue provided by the traditional TCP, the queuing delay of the above control data and the critical audio and video data with a data volume less than the second value is reduced.

[0225] Take a specific scenario as an example: when the bandwidth = 10Mbps, the sending queue of the traditional TCP protocol can cache up to 4MB of data. If 4MB of key audio and video data has been cached at this time, that is, 32Mbit, the queuing delay is: 32Mbit / 10Mbps=3.2s. For users, the queuing delay is too long, which not only causes the transmission delay of the key audio and video data in the sending queue to be too long, but also causes the queuing delay of other control data and audio and video data with smaller data volume waiting to enter the sending queue to be too large. However, by using a plurality of sending queues of different lengths provided in the embodiment of the present application, the queuing delay can be reduced. For example, when the low-latency queue is full of control data, the length of the low-latency queue is 4KB, and the queuing delay of the new control data is: 0.032Mbit / 10Mbps=0.0032s. Obviously, this data transmission method using sending queues of different lengths can effectively ensure the real-time transmission requirements of control data and audio and video data with smaller data volume, and improve the fluency of user experience.

[0226] S106, the server 100 inputs the data to be sent into the sending queue.

[0227] Specifically, when data is input from a sending queue, the dynamic configuration module of the server 100 may cache the data to be sent into the corresponding sending queue in order from high to low data priority.

[0228] In an embodiment of the present application, the priority of the data to be sent can be called the first priority, and the order in which the data to be sent enters the sending queue can be called the first order.

[0229] In an embodiment of the present application, the server 100 may prioritize the data in advance, for example: when a reliable transmission protocol is used to transmit data, control data has the highest priority, followed by critical audio and video data with a data volume less than a second value, and critical audio and video data with a data volume greater than or equal to the second value; when a semi-reliable transmission protocol is used to transmit data, non-critical audio and video data with a data volume less than the second value has a high priority, and non-critical audio and video data with a data volume greater than or equal to the second value has a low priority; when protocol stack 3 is used to transmit data, advertising data with a data volume less than the second value has a high priority, and advertising data with a data volume greater than or equal to the second value has a low priority.

[0230] The principle of data priority division can be determined according to the degree of influence of different types of data on user experience, and the data with greater influence on user experience will be given a higher priority. For example, users are more sensitive to the real-time transmission of control data. If the delay of control data transmission is too long, users can clearly perceive that some operations are not responded to for a long time. Therefore, control data transmission has the greatest impact on user experience, followed by key audio and video data, non-key audio and video data, and advertising data.

[0231] This can ensure that high-priority data is processed first, thereby ensuring the smoothness of user operations and the smoothness of the presentation of key audio and video data. Even if some non-critical audio and video data or advertising data is processed with a delay, it will not affect the user's gaming experience. Overall, it increases data transmission efficiency and improves user experience.

[0232] S107, the server 100 uses the above-determined protocol to send the data in the sending queue to the electronic device 200.

[0233] Specifically, when data is output from a sending queue, the multiplexer of the server 100 may send the data in the corresponding sending queue to the electronic device 200 in order from high to low priority of the sending queue.

[0234] In an embodiment of the present application, the priority of the sending queue can be called the second priority, and the order in which the data to be sent is output from the sending queue can be called the second order.

[0235] Specifically, the priority of the sending queue is: the low-latency queue has the highest priority, and the high-throughput queue has the lowest priority. This is because the data cached in the low-latency queue has a small amount of data, and the corresponding transmission delay is also small, while the data cached in the high-throughput queue has a large amount of data, and the corresponding transmission delay is large, so try to ensure that the data with a small amount of data is sent first to improve the overall data transmission efficiency.

[0236] It is worth noting that when a reliable transmission protocol is used for transmission, in addition to the low-latency queue and the high-throughput queue, there may be other queues. At this time, the queue priorities are in the order of low-latency queue, other queues, and high-throughput queue. This is because, in the process of the server 100 providing game application services for the electronic device 200, control data always needs to be processed first to ensure the normal experience of the user; secondly, key audio and video data with a small amount of data also needs to be transmitted to the electronic device 200 as soon as possible; finally, key audio and video data with a large amount of data can be sent last, because the transmission delay of data with a small amount of data is small, and the transmission delay of data with a large amount of data is large, so try to ensure that data with a small amount of data is sent first to improve transmission efficiency.

[0237] In some other embodiments, special service requests may occur, and the multiplexer may temporarily adjust the priority of the sending queue according to the special service request, thereby changing the data output order in the sending queue. For example, the special service request may be that different audio and video data blocks correspond to different game levels, that is, the display order of audio and video data of different game levels on the electronic device 200 may be different, so the multiplexer may set the priority of the sending queue with a higher cache game level to the highest, ensuring that the data with a higher game level can be output from the sending queue first.

[0238] In this way, by defining the priority of the sending queue, or the multiplexer can, in special business scenarios, give priority to outputting the data cached in the sending queue with a higher priority from the sending queue to the electronic device 200. In this way, it can be further ensured that the data in the high priority queue can be sent to the application in the electronic device 200 first.

[0239] Optionally, in some embodiments, when the data to be sent is output from the sending queue, if each sending queue corresponds to a multiplexer, it means that the data to be sent in all the sending queues can be sent to the electronic device 200 in parallel, that is, the multiplexer does not need to determine the output order of the data in the sending queue according to the priority of the sending queue.

[0240] S108 , the dynamic configuration module of the electronic device 200 uses a protocol stack equivalent to that of the server 100 to parse data.

[0241] Specifically, the server 100 may call the dynamic configuration module in the electronic device 200 to parse the received data according to the protocol stack used by the server 100 to send data monitored by the electronic device 200 as described in step S103.

[0242] It can be seen that in the data transmission method provided in the embodiment of the present application, the data transmission method is personalized and defined through multiple dimensions, such as data reliability requirements and real-time requirements, and multiple priorities, such as the priority of data entering the sending queue and the priority of data outputting the sending queue. After implementing the data transmission method provided in the embodiment of the present application, different data can be personalizedly transmitted using different transmission methods, which can meet both the reliability requirements of different data and the real-time requirements of different data. This improves the user experience.

[0243] It can be understood that when the sending end is the electronic device 200 and the receiving end is the server 100, the steps for the electronic device 200 to send data can refer to the process of the server 100 sending data to the electronic device 200 described in S101-S108 above, which will not be repeated here.

[0244] The various implementation modes of the present application can be combined arbitrarily to achieve different technical effects.

[0245] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in this application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk), etc.

[0246] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0247] In short, the above description is only an embodiment of the technical solution of the present invention, and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data transmission method, the method being applied in a sending end, characterized in that: The method comprises: The transmitting end determines, according to at least one of the data types or quality of experience (QoE) requirements to which the first data and the second data belong, to use a first protocol stack to transmit the first data and the second data, wherein the first protocol stack is one of a protocol supporting reliable transmission, a protocol supporting semi-reliable transmission, or a protocol supporting unreliable transmission; the process of transmitting data using the first protocol stack satisfies a user's reliability requirement for data of the type to which the first data and the second data belong; and the first data and the second data belong to the same data type; The transmitting end determines, according to a first data volume of the first data, a first sending queue for transmitting the first data, and determines, according to a second data volume of the second data, a second sending queue for transmitting the second data, wherein the first sending queue and the second sending queue are sending queues in the first protocol stack, and have different queue lengths; The transmitting end inputs the first data into a first sending queue and inputs the second data into a second sending queue in preference to the second data according to a first priority of the first data and a second priority of the second data, wherein the first priority is higher than the second priority; in, When the first data and the second data are control data or key audio and video data, the first protocol stack is a protocol stack supporting reliable transmission, and the priority of the control data is greater than the priority of the key audio and video data; When the first data and the second data are non-critical audio and video data, the first protocol stack is a protocol stack supporting semi-reliable transmission, and the priority of non-critical audio and video data with a small data volume is higher than the priority of non-critical audio and video data with a large data volume; When the first data and the second data are advertisement data, the first protocol stack is a protocol stack supporting unreliable transmission, and the priority of advertisement data with a small data volume is higher than the priority of advertisement data with a large data volume; When the first protocol stack is a protocol stack supporting reliable transmission, the sending queue in the first protocol stack includes a low-latency queue, other queues, and a high-throughput queue, the lengths of the three queues increase sequentially, and the three queues are respectively used to receive control data, key audio and video data with a data volume less than a second value, and key audio and video data with a data volume greater than or equal to the second value; When the first protocol stack is a protocol stack supporting semi-reliable transmission, the sending queue in the first protocol stack includes a low-latency queue and a high-throughput queue, the lengths of the two queues increase sequentially, and the two queues are respectively used to receive non-critical audio and video data with a data volume less than a second value and non-critical audio and video data with a data volume greater than or equal to the second value; When the first protocol stack is a protocol stack supporting unreliable transmission, the sending queue in the first protocol stack includes a low-latency queue and a high-throughput queue, the lengths of the two queues increase sequentially, and the two queues are respectively used to receive advertisement data with a data volume less than a second value and advertisement data with a data volume greater than or equal to the second value; The length of the sending queue is determined according to the bandwidth of the electronic device and the real-time requirement of the data received by the sending queue. When the real-time requirement is higher, the length of the sending queue is smaller.

2. The method according to claim 1, characterized in that The data type is determined based on the content provided by the data and the user's experience with the content.

3. The method according to claim 1, characterized in that The data type is any one of the following: control data, key audio and video data, non-key audio and video data, and advertisement data.

4. The method according to claim 1, characterized in that: When the first protocol stack is a protocol stack supporting reliable transmission, the sending queues in the first protocol stack include a low-latency queue, other queues and a high-throughput queue, and the priorities of the three queues decrease in sequence; When the first protocol stack is a protocol stack supporting semi-reliable transmission, the sending queue in the first protocol stack includes a low-latency queue and a high-throughput queue, and the priorities of the two queues decrease in sequence; When the first protocol stack is a protocol stack supporting unreliable transmission, the sending queue in the first protocol stack includes a low-latency queue and a high-throughput queue, and the priorities of the two queues decrease in sequence.

5. The method according to claim 1, characterized in that After the transmitting end inputs the first data into a first transmitting queue and inputs the second data into a second transmitting queue, the method further includes: The sending end uses the first protocol stack to send the data in the first sending queue to the receiving end and sends the data in the second sending queue to the receiving end, taking precedence over the second sending queue according to the priorities of the first sending queue and the second sending queue, wherein the priority of the first sending queue is higher than the priority of the second sending queue.

6. The method according to claim 5, characterized in that The length of the first sending queue is smaller than the length of the second sending queue.

7. The method according to any one of claims 1 to 6, characterized in that: The first data is various types of data involved in the real-time audio and video services provided by the sender to the receiver, and the real-time audio and video services include: cloud game services, online live broadcast services or VR / AR services.

8. An electronic device, characterized in that: The electronic device includes one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 7.

Images