UDP data transmission method and related equipment

Abstract

The present application provides a method for UDP data transmission and related equipment, using the characteristics of guaranteed TCP transmission flow and controllable UDP transmission delay, and UDP penetration technology based on TCP proxy. Use raw sockets to establish a TCP channel between two terminals, encapsulate a layer of TCP header on the UDP data packet, encapsulate the UDP data packet to obtain a TCP data packet, and then use the established TCP channel to transmit the TCP data packet. The problem of UDP traffic being limited by QoS or intercepted by a firewall is solved, and based on raw sockets, TCP data packets can directly send and receive data at the IP layer without going through the kernel TCP protocol stack, which solves the problem of large transmission delay of TCP transmission when the network environment is not good, and provides flexibility for the application layer to design a low-latency transmission solution, thereby improving the UDP-based communication experience.

Description

Technical Field

[0001] The present application relates to the field of communications, and more specifically, to a UDP data transmission method and related equipment. Background Art

[0002] Network address translation (NAT) technology is a technology that converts private Internet Protocol (IP) addresses of internal networks into public IP addresses of external networks, which can alleviate the shortage of public IP addresses to a great extent.

[0003] The NAT gateway divides the entire network into two parts: private network and public network. Since the devices on the public network side cannot actively obtain the IP addresses of the devices on the private network, the traffic transmission can only be initiated by the private network side. The devices on the public network side cannot actively access the devices inside the private network. Therefore, NAT technology limits the end-to-end (point-to-point, P2P) communication capabilities.

[0004] The emergence of NAT traversal technology provides an effective solution for P2P communication. At present, there are more and more services with low latency requirements, such as audio and video calls and remote office. For the transmission of these services with low latency requirements, NAT traversal technology is basically used to achieve P2P communication using the user datagram protocol (UDP). However, when using UDP for NAT traversal, the traffic of UDP data faces the risk of being limited by the operator's quality of service (QoS) or blocked by the firewall in the network, and its bandwidth cannot be guaranteed. In particular, when the network traffic is at a peak, the transmission bandwidth is very low, resulting in communication failure, and the user experience of P2P communication based on UDP will be affected. Summary of the invention

[0005] The present application provides a method for UDP data transmission and related equipment, using the characteristics of guaranteed TCP transmission flow and controllable UDP transmission delay, and UDP penetration technology based on TCP proxy. Use raw sockets to establish a TCP channel between two terminals, encapsulate a layer of TCP header on the UDP data packet, encapsulate the UDP data packet to obtain a TCP data packet, and then use the established TCP channel to transmit the TCP data packet. The problem of UDP traffic being limited by QoS or intercepted by a firewall is solved, and based on raw sockets, TCP data packets can directly send and receive data at the IP layer without going through the kernel TCP protocol stack, which solves the problem of large transmission delay of TCP transmission when the network environment is not good, and provides flexibility for the application layer to design a low-latency transmission solution, thereby improving the UDP-based communication experience.

[0006] In a first aspect, a method for UDP data transmission is provided, the method comprising: a first terminal and a second terminal perform a first TCP hole punching; determining that the first TCP hole punching is successful; the first terminal establishes a TCP transmission channel with the second terminal using a raw socket; the first terminal sends a TCP data packet to the second terminal through the TCP transmission channel, the TCP data packet including a TCP header, an IP header and UDP data.

[0007] The UDP data transmission method provided in the first aspect aims at the problem that the UDP data flow is subject to QoS speed limit or firewall interception by the operator when using UDP for NAT penetration communication. The UDP penetration technology based on TCP proxy uses the original socket to establish a TCP channel between two terminals, so that the TCP data packets or TCP messages transmitted between the two terminals can be directly sent and received at the IP layer without passing through the kernel TCP protocol stack. A layer of TCP header is encapsulated on the UDP data packet, and the UDP data packet is encapsulated to obtain a TCP data packet, and then the TCP data packet is transmitted using the established TCP channel. The problem that UDP traffic is limited by QoS or intercepted by the firewall is solved, thereby ensuring that UDP traffic can be treated equally with TCP and improving the P2P communication experience of UDP. In addition, based on the original socket, data can be directly sent and received in the TCP data packet at the IP layer without passing through the kernel TCP protocol stack. In the process of transmitting TCP data packets, the heavy TCP kernel protocol layer is not introduced, which solves the problem of large transmission delay of TCP transmission when the network environment is not good, and facilitates the business to design a low-latency transmission solution that meets business needs based on the UDP protocol at the application layer, greatly improving the design flexibility, thereby improving the communication experience based on UDP.

[0008] Among them, the raw socket (SOCK_RAW) can be used to assemble IP data packets by itself and then send the data packets to other terminals. In other words, the raw socket is based on IP data packets and is a socket that processes raw network messages. The raw socket can send and receive data packets that have not passed through the kernel TCP protocol stack.

[0009] In a possible implementation of the first aspect, before performing the first TCP hole punching, the method also includes: the first terminal obtains the TCP type public network address and port number of the second terminal, and the first port number, and the second terminal receives a UDP data packet through the first port number, and the UDP data packet includes the UDP data; the first terminal sends the TCP type public network address and port number of the first terminal, and the second port number to the second terminal, and the first terminal sends a UDP data packet through the second port number, and the UDP data packet includes the UDP data; the first terminal and the second terminal perform the first TCP hole punching, including: the first terminal and the second terminal perform the first TCP hole punching according to the TCP type public network address and port number of the first terminal and the TCP type public network address and port number of the second terminal.

[0010] Exemplarily, the second port number is the UDP port number of the first terminal, and the UDP port number of the first terminal is the forwarding port of UDP data packets between different modules inside the first terminal. The IP address corresponding to the forwarding port number is generally a loopback address.

[0011] Exemplarily, the first port number is the UDP port number of the second terminal, the UDP port number of the second terminal is the forwarding port number of UDP data packets between different modules inside the second terminal, and the IP address corresponding to the forwarding port number is generally a loopback address.

[0012] Exemplarily, the first terminal obtains the TCP type public network address and port number of the first terminal, and the first port number from the STUN server through the first NAT gateway.

[0013] Exemplarily, the second terminal obtains the TCP type public network address and port number of the second terminal, and the second port number from the STUN server through the second NAT gateway.

[0014] Exemplarily, the first terminal and the second terminal exchange their respective TCP type public network addresses and port numbers, as well as the first port number and the second port number through the NAT gateway device and the signaling server.

[0015] Among them, the NAT gateway device corresponding to the first terminal is a first NAT gateway, and the NAT gateway device corresponding to the second terminal is a second NAT gateway.

[0016] In a possible implementation of the first aspect, the first terminal sends a TCP data packet to the second terminal through the TCP transmission channel, including: the first terminal sends the TCP data packet to a first NAT gateway;

[0017] The source address and port number of the TCP data packet are respectively the private network address and port number of the TCP type of the first terminal; the destination address and port number of the TCP data packet are respectively the public network address and port number of the TCP type of the second terminal.

[0018] In a possible implementation of the first aspect, the TCP data packet also includes a first field; the first field includes: a first port number and a second port number; or, the first field includes: a first port number and a second port number, and a first address corresponding to the first port number and a second address corresponding to the second port number; wherein the second terminal receives a UDP data packet through the first port number and the first address, and the UDP data packet includes the UDP data, and the first terminal sends a UDP data packet through the second port number and the second address, and the UDP data packet includes the UDP data.

[0019] Exemplarily, the UDP data includes a first field.

[0020] Exemplarily, the first field is a UDP overlay field.

[0021] Exemplarily, the first address and the second address are both loopback addresses.

[0022] The loopback address is a special address used by the host to send communications to itself, realizing the functions of client and server on one host. The loopback address provides a shortcut for TCP / IP applications and services running on the same device to communicate with each other. If two services on the same host use the loopback address instead of the assigned host address, they can bypass the lower layers of the TCP / IP protocol stack.

[0023] Exemplarily, the loopback address is 127.0.0.1.

[0024] Exemplarily, the first terminal includes: an application layer (App module) and a TCP proxy module, and the application layer (App module) sends a UDP data packet to the TCP proxy module through the second port number and the loopback address, and the UDP data packet includes the UDP data, UDP header and IP header.

[0025] The source address and port number of the UDP data packet are: the port number and loopback address of the application layer (App module) of the first terminal; the destination address and port number are: the port number and loopback address of the TCP proxy module of the first terminal.

[0026] Among them, the TCP proxy module is used to simulate the TCP handshake process through the original socket and establish a TCP transmission channel with other terminals. In addition, the TCP proxy module is also used to encapsulate the TCP header and IP header on the UDP data packet after receiving the UDP data packet (or it can also be called a UDP message) to form a TCP data packet (or it can also be called a TCP message), so that the TCP data packet is finally sent directly to the opposite device at the IP layer without passing through the TCP transport layer. The TCP proxy module is also used to receive the TCP data packet sent by the opposite device, decapsulate the data packet, remove the TCP header and IP header, so that from this For another example, the application layer (App module) of the first terminal can also send the UDP data packet of the application layer to the TCP proxy module in the first terminal through the virtual network device (tun device). The present application does not limit the method of transmitting data packets between the application layer (App module) of the first terminal and the TCP proxy module in the first terminal.

[0027] In a possible implementation manner of the first aspect, the application layer (App module) of the first terminal may also send the UDP data packet of the application layer to the TCP proxy module in the first terminal through the virtual network device (tun device).

[0028] In a possible implementation of the first aspect, the method further includes: the first terminal and the second terminal perform a second TCP hole punching; determine that the second TCP hole punching is not successful; the first terminal and the second terminal perform a first UDP hole punching; determine that the first UDP hole punching is successful; the first terminal and the second terminal establish a UDP transmission channel; the first terminal sends UDP data to the second terminal through the UDP transmission channel. In this implementation, if the TCP hole punching is unsuccessful and the UDP hole punching is successful, the first terminal and the second terminal transmit UDP data packets through the NAT gateway using the traditional UDP mode, that is, the UDP is used to perform NAT penetration technology to realize the transmission of UDP data packets between the first terminal and the second terminal.

[0029] In a possible implementation of the first aspect, before performing the first UDP hole punching, the method also includes: the first terminal obtains the UDP type public network address and port number of the second terminal; the first terminal sends the UDP type public network address and port number of the first terminal to the second terminal; the first terminal and the second terminal perform the first UDP hole punching, including: the first terminal and the second terminal perform the first UDP hole punching according to the UDP type public network address and port number of the first terminal and the UDP type public network address and port number of the second terminal.

[0030] Exemplarily, the first terminal and the second terminal exchange their respective UDP type public network addresses and port numbers through a NAT gateway device and a signaling server.

[0031] In a possible implementation of the first aspect, the method further includes: the first terminal and the second terminal perform a third TCP hole punching; determine that the third TCP hole punching is not successful; the first terminal and the second terminal perform a second UDP hole punching; determine that the second UDP hole punching is not successful; the first terminal sends UDP data to the second terminal through a relay device. In this implementation, if both TCP hole punching and UDP hole punching fail, the first terminal and the second terminal transmit UDP data packets using forwarding of the relay server.

[0032] For example, the first terminal sends a UDP data packet to the relay server through the first NAT gateway, and after the relay server receives the UDP data packet, it sends it to the second terminal through the second NAT gateway.

[0033] In a second aspect, a method for UDP data transmission is provided, the method comprising: a second terminal and a first terminal perform a first TCP hole punching; determining that the first TCP hole punching is successful; the second terminal establishes a TCP transmission channel with the first terminal using a raw socket; the second terminal receives a TCP data packet from the first terminal through the TCP transmission channel, the TCP data packet comprising a TCP header, an IP header and UDP data.

[0034] The second aspect provides a method for UDP data transmission, which aims at the problem that the traffic of UDP data is subject to QoS speed limit or firewall interception by the operator when using UDP for NAT penetration communication. The UDP penetration technology based on TCP proxy uses the original socket to establish a TCP channel between two terminals, so that the TCP data packets or TCP messages transmitted between the two terminals can be directly sent and received at the IP layer without passing through the kernel TCP protocol stack. A layer of TCP header is encapsulated on the UDP data packet, and the UDP data packet is encapsulated to obtain a TCP data packet, and then the TCP data packet is transmitted using the established TCP channel. The problem of UDP traffic being limited by QoS or intercepted by the firewall is solved, thereby ensuring that UDP traffic can be treated equally with TCP and improving the P2P communication experience of UDP. In addition, based on the original socket, the TCP data packet can be directly sent and received at the IP layer of the sending device and the receiving device, without passing through the kernel TCP protocol stack of the sending and receiving devices, and the heavy TCP kernel protocol layer is not introduced in the process of transmitting the TCP data packet, which solves the problem of large transmission delay of TCP transmission when the network environment is not good, and facilitates the business to design a low-latency transmission solution that meets business needs based on the UDP protocol at the application layer, greatly improving the design flexibility, thereby improving the communication experience based on UDP.

[0035] In a possible implementation manner of the second aspect, the method further includes: the second terminal decapsulates the TCP data packet to obtain UDP data in the TCP data packet.

[0036] Exemplarily, the second terminal includes: an application layer (App module) and a TCP proxy module, the TCP proxy module receives the TCP data packet, decapsulates the TCP data packet, obtains UDP data in the TCP data packet, and encapsulates the UDP data packet. The UDP data packet includes the UDP data, a UDP header, and an IP header.

[0037] The TCP proxy module of the second terminal sends the UDP data packet to the application layer (App module) of the second terminal, wherein the source address of the UDP data packet is: the loopback address and port number of the TCP proxy module of the second terminal, and the destination address is: the loopback address and the first port number of the application layer (App module) of the second terminal.

[0038] Exemplarily, the loopback address is 127.0.0.1.

[0039] In a possible implementation manner of the second aspect, the TCP proxy module of the second terminal may also send the UDP data packet to the application layer (App module) in the second terminal through a virtual network device (tun device).

[0040] In a possible implementation of the second aspect, the TCP data packet also includes a first field; the first field includes: a first port number and a second port number; or, the first field includes: a first port number and a second port number, and a first address corresponding to the first port number and a second address corresponding to the second port number; wherein the second terminal receives a UDP data packet through the first port number and the first address, and the UDP data packet includes the UDP data, and the first terminal sends a UDP data packet through the second port number and the second address, and the UDP data packet includes the UDP data.

[0041] Exemplarily, the UDP data includes a first field.

[0042] Exemplarily, the first field is a UDP overlay field.

[0043] Exemplarily, the first address and the second address are both loopback addresses.

[0044] In a possible implementation of the second aspect, before performing the first TCP hole punching, the method also includes: the second terminal obtains the TCP type public network address and port number of the first terminal, and the second port number, and the first terminal sends a UDP data packet through the second port number, and the UDP data packet includes the UDP data; the second terminal sends the TCP type public network address and port number of the second terminal, and the first port number to the first terminal, and the second terminal receives the UDP data packet through the first port number, and the UDP data packet includes the UDP data; the second terminal and the first terminal perform the first TCP hole punching, including: the second terminal and the first terminal perform the first TCP hole punching according to the TCP type public network address and port number of the second terminal and the TCP type public network address and port number of the first terminal.

[0045] In a possible implementation of the second aspect, the second terminal receives a TCP data packet from the first terminal through the TCP transmission channel, including: the second terminal receives a TCP data packet sent by a second NAT gateway; wherein the destination address and port number of the TCP data packet are respectively a private network address and port number of the TCP type of the second terminal; and the source address and port number of the TCP data packet are respectively a public network address and port number of the TCP type of the first terminal.

[0046] In a possible implementation of the second aspect, the method further includes: the second terminal and the first terminal perform a second TCP hole punching; determine that the second TCP hole punching is not successful; the second terminal and the first terminal perform a first UDP hole punching; determine that the first UDP hole punching is successful; the second terminal and the first terminal establish a UDP transmission channel; the second terminal receives UDP data from the first terminal through the UDP transmission channel. In this implementation, if the TCP hole punching is unsuccessful, but the UDP hole punching is successful, the first terminal and the second terminal transmit UDP data packets through the NAT gateway using the traditional UDP mode, that is, using UDP to perform NAT penetration technology to achieve the transmission of UDP data packets between the first terminal and the second terminal.

[0047] In a possible implementation of the second aspect, before performing the first UDP hole punching, the method also includes: the second terminal obtains the UDP type public network address and port number of the first terminal; the second terminal sends the UDP type public network address and port number of the second terminal to the first terminal; the second terminal and the first terminal perform the first UDP hole punching, including: the second terminal and the first terminal perform the first UDP hole punching according to the UDP type public network address and port number of the second terminal and the UDP type public network address and port number of the first terminal.

[0048] In a possible implementation of the second aspect, the method further includes: the second terminal and the first terminal perform a third TCP hole punching; determine that the third TCP hole punching is not successful; the second terminal and the first terminal perform a second UDP hole punching; determine that the second UDP hole punching is not successful; the second terminal receives UDP data from the first terminal through a relay device. In this implementation, if both TCP hole punching and UDP hole punching fail, the first terminal and the second terminal transmit UDP data packets using the forwarding of the relay server.

[0049] According to a third aspect, a communication device is provided, comprising a unit for executing each step in the above first aspect or any possible implementation of the first aspect.

[0050] In a fourth aspect, a communication device is provided, which includes at least one processor and a memory, the processor and the memory are coupled, the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method in the above first aspect or any possible implementation of the first aspect is executed.

[0051] In a fifth aspect, a communication device is provided, which includes at least one processor and an interface circuit, and the at least one processor is used to execute the method in the above first aspect or any possible implementation of the first aspect.

[0052] In a sixth aspect, a terminal device is provided, which includes any communication device provided in the third aspect, the fourth aspect or the fifth aspect.

[0053] In the seventh aspect, a communication device is provided, which includes at least one processor and a memory, the processor and the memory are coupled, the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method in the above second aspect or any possible implementation of the second aspect is executed.

[0054] In an eighth aspect, a communication device is provided, which includes a unit for executing each step in the above second aspect or any possible implementation of the second aspect.

[0055] In a ninth aspect, a communication device is provided, comprising at least one processor and an interface circuit, wherein the at least one processor is used to execute the method in the above second aspect or any possible implementation of the second aspect.

[0056] In the tenth aspect, a terminal device is provided, comprising any communication device provided in the seventh aspect, the eighth aspect or the ninth aspect.

[0057] In the eleventh aspect, a UDP data transmission system is provided, the system comprising: the terminal device provided in the sixth aspect and the terminal device provided in the tenth aspect.

[0058] Optionally, the system may further include: a NAT gateway device and a server, wherein the server includes a STUN server, a relay server and a signaling server.

[0059] Optionally, the NAT gateway device includes: a first NAT gateway device and a second NAT gateway device.

[0060] In the twelfth aspect, a computer program product is provided, which includes a computer program, which, when executed by a processor, is used to execute the method in the first aspect or any possible implementation of the first aspect, or to execute the method in the second aspect or any possible implementation of the second aspect.

[0061] In the thirteenth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed, it is used to execute the method in the first aspect or any possible implementation of the first aspect, or to execute the method in the second aspect or any possible implementation of the second aspect.

[0062] In the fourteenth aspect, a chip is provided, comprising: a processor, for calling and running a computer program from a memory, so that a communication device equipped with the chip executes the method in the first aspect or any possible implementation of the first aspect, or executes the method in the second aspect or any possible implementation of the second aspect.

[0063] The method and related equipment for UDP data transmission provided by the present application aim at the problem that the flow of UDP data faces the QoS speed limit of the operator or is intercepted by the firewall when using UDP for NAT penetration communication. The UDP penetration technology based on TCP proxy uses the original socket to establish a TCP channel between two terminals, so that the TCP data packets or TCP messages transmitted between the two terminals can be directly sent and received at the IP layer without passing through the kernel TCP protocol stack. A layer of TCP header is encapsulated on the UDP data packet, and the UDP data packet is encapsulated to obtain a TCP data packet, and then the TCP data packet is transmitted using the established TCP channel. The problem that the UDP traffic is limited by QoS or intercepted by the firewall is solved, thereby ensuring that the UDP traffic can be treated equally with TCP and improving the P2P communication experience of UDP. Moreover, based on the original socket, the TCP data packet can be directly sent and received at the IP layer without passing through the kernel TCP protocol stack, and the heavy TCP kernel protocol layer is not introduced in the process of transmitting the TCP data packet, which solves the problem of large transmission delay of TCP transmission when the network environment is not good, and facilitates the business to design a low-latency transmission solution that meets the business needs based on the UDP protocol at the application layer, greatly improving the design flexibility, thereby improving the communication experience based on UDP. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a schematic diagram of a NAT traversal technology (or also known as NAT penetration technology).

[0065] Figure 2 This is a schematic flow chart of a method for implementing P2P communication using UDP for NAT penetration technology (or also known as NAT traversal technology).

[0066] Figure 3 This is a schematic diagram of a communication system suitable for implementing a P2P communication method using UDP for NAT penetration technology.

[0067] Figure 4 This is a schematic diagram of the UDP data packet structure in the process of using UDP to perform NAT penetration technology to achieve P2P communication.

[0068] Figure 5 This is a schematic flow chart of a method for implementing P2P communication using TCP for NAT penetration technology (or also known as NAT traversal technology).

[0069] Figure 6 The present invention is a schematic diagram of a communication system suitable for implementing a P2P communication method using TCP for NAT penetration technology.

[0070] Figure 7This is a schematic diagram of a TCP data packet structure in the process of using TCP to perform NAT penetration technology to achieve P2P communication.

[0071] Figure 8 This is a schematic diagram of an example of a communication system architecture provided by the present application and applicable to an embodiment of the present application.

[0072] Fig. 9 This is a schematic diagram of an example of the software architecture inside a terminal device provided by this application.

[0073] Fig.10 This is a schematic flowchart of a UDP data transmission method provided by the present application.

[0074] Fig.11 This application provides an example applicable to Fig.10 A schematic diagram of a communication system for the described method.

[0075] Fig.12 This is a schematic diagram of an example of the processing process of UDP data within the first terminal provided by the present application.

[0076] Fig.13 This is a schematic diagram of an example of the processing process of UDP data inside the second terminal provided by the present application.

[0077] Fig.14 This is a schematic diagram of an example of a TCP data packet format provided by this application.

[0078] Fig.15 This is a schematic diagram of the format of the TCP header and application data (App Payload) portion of a TCP data packet provided by this application.

[0079] Fig.16 This is a schematic block diagram of an example of a terminal device provided in an embodiment of the present application.

[0080] Fig.17 It is a schematic block diagram of a chip system provided in an embodiment of the present application.

[0081] Fig.18 It is a schematic block diagram of a communication device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0082] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0083] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0084] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0085] In addition, various aspects or features of the present application can be implemented as methods, devices or products using standard programming and / or engineering techniques. The term "product" used in this application covers computer programs that can be accessed from any computer-readable device, carrier or medium. For example, computer-readable media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks or tapes, etc.), optical disks (e.g., compact discs (CDs), digital versatile discs (DVDs), etc.), smart cards and flash memory devices (e.g., erasable programmable read-only memory (EPROM), cards, sticks or key drives, etc.). In addition, the various storage media described herein may represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing and / or carrying instructions and / or data.

[0086] Various aspects or features of the present application may be implemented as methods, devices, or products using standard programming and / or engineering techniques. The term "product" used in the present application covers computer programs that can be accessed from any computer-readable device, carrier, or medium. For example, computer-readable media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or tapes, etc.), optical disks (e.g., compact discs (CDs), digital versatile discs (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memory (EPROM), cards, sticks, or key drives, etc.). In addition, the various storage media described herein may represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0087] NAT technology is a technology that converts the private IP address of an internal network (referred to as the intranet) into the public IP address of an external network (referred to as the extranet). It allows multiple hosts within a certain range to connect to the extranet using only one public IP address, which can alleviate the shortage of public IP addresses to a great extent.

[0088] For example, if a terminal A in a local area network wants to communicate with another terminal B on the Internet, when the data packet sent by A passes through the NAT device at the exit of the local area network, the NAT device will replace the address and port number of device A in the source address field of the data packet with the corresponding public network IP and Port, and then send the data packet to terminal B. The source address seen by terminal B is the converted IP and Port. Terminal B does not know the local area network address of terminal A. When the response information of terminal B reaches the NAT device of terminal A, since the destination address in the response information of terminal B is the public network IP and Port (that is, the external network address of terminal A), the NAT device searches for the internal network address corresponding to this external network address stored in the memory. If it is found, the response information will be forwarded to this address, thus realizing the communication between terminal A and terminal B.

[0089] As can be seen from the above, the NAT gateway divides the entire network into two parts: the private network and the public network. The data traffic between the two parts needs to be converted through the NAT gateway for address mapping. Since the device on the public network side cannot actively obtain the IP address of the device on the private network, the traffic transmission can only be initiated by the private network side first, and the public network side device cannot actively access the internal device of the private network. Therefore, while NAT technology solves the shortage of Internet Protocol version 4 (IPv4) addresses and improves security, it also limits the communication capabilities of P2P.

[0090] The emergence of NAT traversal technology provides an effective solution for P2P communication. On the other hand, with the development and popularization of 4G / 5G technology, more and more low-latency services such as audio and video calls and remote office are requiring low latency, which brings more challenges to application vendors. Data packet transmission based on UDP has the advantages of low resource consumption and fast processing speed, so UDP is usually used more for transmission of services with low latency requirements. Therefore, more and more low-latency services are beginning to design and develop low-latency transmission solutions based on UDP to further improve user experience.

[0091] Figure 1 What is shown is a schematic diagram of an example of NAT traversal technology (or also called NAT penetration technology).

[0092] like Figure 1 As shown, both device A and device B are private network devices located behind their own NAT gateways. Assume that the private network IP address of device A is 192.168.0.8, the port number is 4000, and the private network IP address of device B is 192.168.1.8, the port number is 4000. The IP address of the server is 6.7.8.9, and the port number is 8000.

[0093] When device A accesses the server, the source address and port number of the data packet of device A are: 192.168.0.8:4000. The destination address and port number are: 6.7.8.9:8000. After receiving the data packet, the NAT gateway of device A (called NAT gateway 1) converts the source address of the data packet of device A to: 1.2.3.4, and modifies the source port number to: 62000. In this way, device A can access the server.

[0094] When device B accesses the server, the source address and port number of the data packet of device B are: 192.168.1.8:4000. The destination address and port number are: 6.7.8.9:8000. After receiving the data packet, the NAT gateway of device B (called NAT gateway 2) converts the source address of the data packet of device B to: 1.2.3.5, and modifies the source port number to: 31000. In this way, device B can access the server.

[0095] The server saves the address information (including public IP address and port number) of device B and device A on their respective NAT gateways, where the (public) address information of device A on NAT gateway 1 is: 1.2.3.4:62000. The public address is a TCP type public address of device A or a UDP type public address.

[0096] The (public network) address information of device B in NAT gateway 2 is: 1.2.3.5:31000, and the public network address is a TCP type public network address of device B or a UDP type public network address.

[0097] Furthermore, the server may notify device A and device B of the address information respectively.

[0098] If device A wants to send information to device B, the source address and port number of the information are: 192.168.0.8:4000, and the destination address and port number are: 1.2.3.5:31000. After the information of device A is converted by NAT gateway 1, the source address and port number become: 1.2.3.4:62000. After NAT gateway 2 receives the information, it detects that there is no destination IP or port number related to 1.2.3.4:62000 in its surviving session, so it will discard the data packet, resulting in the failure of communication between device A and device B.

[0099] Therefore, to achieve communication between device A and device B, the server needs to tell device A and device B to perform "TCP hole punching" or "UDP hole punching" in their own NAT gateways respectively to establish a session association table.

[0100] The specific process of "TCP hole punching" or "UDP hole punching" is as follows: Device A sends a message to NAT gateway 1, the source address and port number of the message are: 192.168.0.8:4000, the destination address and port number are: 1.2.3.5:31000. Device B sends a message to NAT gateway 2, the source address and port number of the message are: 192.168.1.8:4000, the destination address and port number are: 1.2.3.4:62000. After the hole punching is successful, NAT gateway 1 maintains the address mapping relationship, which is the corresponding relationship between the private network address of device A and the public network address of device B. NAT gateway 2 also maintains the address mapping relationship, which is the corresponding relationship between the private network address of device B and the public network address of device A. After that, device A sends a data packet to device B. The source address and port number of the data packet are: 192.168.0.8:4000: the destination address and port number are: 1.2.3.5:31000. After the data packet sent by device A is converted by NAT gateway 1, the source address and port number become: 1.2.3.4:62000, and the destination address and port number remain unchanged. After receiving the data packet, NAT gateway 2 determines that the destination IP and port number corresponding to 1.2.3.4:62000 are: 192.168.1.8:4000 based on the maintained address mapping relationship, so that the data packet can be forwarded to device B, realizing the communication between device A and device B.

[0101] On the other hand, with the development and popularization of 4G / 5G technology, more and more audio and video-related services require low latency, which brings more challenges to application manufacturers. Therefore, more and more business applications have begun to design and develop low-latency transmission solutions based on UDP to further improve user experience.

[0102] Figure 2 What is shown is a schematic flow chart of using UDP to perform NAT penetration technology (or also called NAT traversal technology) in the related art to achieve P2P communication. Figure 3 Shown is the execution Figure 2 A schematic diagram of a communication system of the method shown, Figure 3 The curve in is a schematic diagram of UDP data stream transmission.

[0103] like Figure 2 As shown, the method includes:

[0104] S101, device A and device B detect and discover their respective public network addresses and ports through NAT gateway 1 and NAT gateway 2, based on UDP NAT session traversal utilities for NAT (STUN) protocol. Among them, the NAT gateway corresponding to device A is NAT gateway 1, and the NAT gateway corresponding to device B is NAT gateway 2.

[0105] S102, device A and device B exchange their respective UDP public network addresses and port numbers through NAT gateway 1 and NAT gateway 2 via a signaling server.

[0106] S103, based on the exchanged address and port information, device A and device B perform UDP hole punching using UDP sockets through NAT gateway 1 and NAT gateway 2.

[0107] S104, after the hole is successfully punched, device A and device B can establish a UDP channel and perform UDP communication through the UDP channel, that is, device A and device B transmit UDP data packets through NAT gateway 1 and NAT gateway 2.

[0108] Figure 4 The figure shows the structure of a UDP data packet (or UDP datagram) in the process of implementing P2P communication using UDP NAT penetration technology. Figure 4 As shown, the UDP data packet includes: an IP header (IP head), a UDP header (UDP head) and application data (App Payload).

[0109] For example, suppose:

[0110] The UDP type private IP address of device A is: 192.168.0.8, the port number is: 4000, and the UDP type public network address and port number allocated by NAT gateway 1 to device A are: 1.2.3.4:62000.

[0111] The UDP type private IP address of device B is: 192.168.1.8, and the port number is: 4000; the UDP type public network address and port number allocated by NAT gateway 2 to device B are: 1.2.3.5:31000.

[0112] Assumptions Figure 4The UDP data packet shown is a data packet sent by device A to device B. The application (App) module of device A sends the application data (App Payload) to the UDP processing module, and the UDP processing module encapsulates the application data (App Payload) with a UDP header (UDP head). The UDP header carries the private network type port number "4000" of device A and the public network type port number "31000" of device B. The UDP processing module then continues to send the UDP header and application data (App Payload) to the IP processing module. The IP processing module encapsulates the IP header (IP head) before the UDP header, and finally forms a UDP data packet. The structure of the UDP data packet is shown as follows. Figure 4 As shown in the figure, the IP packet header carries: the private network address "192.168.0.8" of device A and the public network address "1.2.3.5:31000" of device B.

[0113] The IP processing module first sends the UDP data packet to NAT gateway 1 through the kernel protocol layer module and the network card module, and NAT gateway 1 receives the UDP data packet. The source address and port number of the UDP data packet are: 192.168.0.8:4000, and the destination address and port number are: 1.2.3.5:31000.

[0114] NAT Gateway 1 replaces the source address and port number in the UDP data packet with 1.2.3.4:62000, and then sends the UDP data packet to NAT Gateway 2. After receiving the UDP data packet, NAT Gateway 2 maintains the correspondence between the private network address of Device B and the public network address of Device A because of the previous UDP hole punching. Based on the correspondence, NAT Gateway 2 can determine that the destination IP and port number corresponding to "1.2.3.4:62000" are 192.168.1.8:4000, and thus replaces the destination address and port number of the UDP data packet with 192.168.1.8:4000, while the source address and port number remain unchanged, which is 1.2.3.4:62000. Then the UDP data packet is sent to Device B.

[0115] After device B receives the UDP data packet, it is transmitted to the IP processing module of device B through the network card module and kernel protocol layer module of device B. The IP processing module decapsulates it, removes the IP header, parses out the UDP header and application data (App Payload) in the UDP data packet, and sends the UDP header and application data (App Payload) to the UDP processing module of device B. The UDP processing module removes the UDP header, parses out the application data (App Payload), and sends the application data (App Payload) to the application (App) module of device B. Finally, the application (App) module parses the application data (App Payload).

[0116] However, when using the above UDP penetration method, the UDP data traffic faces the risk of being limited by the operator's QoS or blocked by the firewall in the network, and its bandwidth cannot be guaranteed. In particular, the transmission bandwidth is very low during peak network traffic, resulting in communication failures, and the user experience of UDP-based P2P communication will be affected.

[0117] Figure 5 The figure shows a schematic flow chart of using TCP to perform NAT penetration technology to achieve P2P communication in the related art. Figure 6 Shown is the execution Figure 5 A schematic diagram of the device architecture of the method shown, Figure 5 The curve in FIG. 1 is a schematic diagram of TCP data stream transmission, wherein the NAT gateway corresponding to device A is NAT gateway 1, and the NAT gateway corresponding to device B is NAT gateway 2.

[0118] like Figure 5 As shown, the method includes:

[0119] S201, device A and device B detect their respective public network addresses and ports through NAT gateway 1 and NAT gateway 2 based on the STUN protocol of TCP.

[0120] S202, device A and device B exchange their respective TCP public network addresses and port numbers through NAT gateway 1 and NAT gateway 2 via a signaling server.

[0121] S203, based on the exchanged address and port information, device A and device B perform TCP hole punching using kernel TCP sockets through NAT gateway 1 and NAT gateway 2.

[0122] S204, after the hole is successfully punched, device A and device B can establish a TCP channel and perform TCP communication through the TCP channel. That is, device A sends a TCP data packet to device B through the NAT gateway 1.

[0123] Figure 7 The figure shows the structure of a TCP data packet (or TCP datagram) in the process of implementing P2P communication using NAT penetration technology using TCP. Figure 7 As shown, the TCP data packet includes: an IP header (IP head), a TCP header (TCP head) and application data (App Payload).

[0124] For example, suppose:

[0125] The TCP type private IP address of device A is: 192.168.2.8, the port number is: 6000, and the TCP type public network address and port number allocated by NAT gateway 1 to device A are: 1.2.3.6:72000.

[0126] The TCP type private IP address of device B is: 192.168.3.8, and the port number is: 6000; the TCP type public network address and port number assigned to device B by NAT gateway 2 are: 1.2.3.7:51000 respectively.

[0127] Assumptions Figure 7 The TCP data packet shown is a data packet sent from device A to device B. The application (App) module of device A sends the application data (App Payload) to the TCP processing module. The TCP processing module encapsulates the application data (App Payload) with a TCP header (TCP head). The TCP header carries the private network port number "6000" of device A and the public network port number "51000" of device B. The TCP processing module then continues to send the TCP header and the application data (App Payload) part to the IP processing module. The IP processing module encapsulates the IP header (IP head) before the TCP header, and finally forms a TCP data packet. The structure of the TCP data packet is shown as follows: Figure 7 As shown in the figure, the IP packet header carries: the private network address "192.168.2.8" of device A and the public network address "1.2.3.7:51000" of device B.

[0128] The IP processing module first sends the TCP data packet to NAT gateway 1 through the kernel protocol layer module and the network card module, and NAT gateway 1 receives the TCP data packet. The source address and port number of the TCP data packet are respectively: 192.168.2.8:6000, and the destination address and port number are respectively: 1.2.3.7:51000.

[0129] NAT Gateway 1 replaces the source address and port number in the TCP data packet with 1.2.3.6; 72000 respectively, and then sends the TCP data packet to NAT Gateway 2. After receiving the TCP data packet, NAT Gateway 2 maintains the correspondence between the private network address of Device B and the public network address of Device A because TCP hole punching was performed before. According to the correspondence, NAT Gateway 2 can determine that the destination IP and port number corresponding to "1.2.3.6:72000" are 192.168.3.8:6000, and thus replaces the destination address and port number of the TCP data packet with 192.168.3.8:6000. The source address and port number of the TCP data packet remain unchanged, which is 1.2.3.6:72000. Then the TCP data packet is sent to Device B, so that the transmission of TCP data packets between Device A and Device B can be realized.

[0130] After device B receives the TCP data packet, it is transmitted to the IP processing module of device B through the network card module and kernel protocol layer module of device B. The IP processing module decapsulates and parses the TCP header and application data (App Payload) in the TCP data packet, and sends the TCP header and application data (App Payload) to the TCP processing module of device B. The TCP processing module removes the TCP header, parses the application data (App Payload), and sends the application data (AppPayload) to the application (App) module of device B. Finally, the application data (AppPayload) is parsed by the application (App) module.

[0131] The method of using TCP for NAT penetration technology. Since the TCP protocol of the Linux kernel is an orderly and reliable transmission protocol, although the bandwidth can be guaranteed, when the network is unstable or the quality is poor, the communication based on the TCP transmission protocol will produce a huge delay, which will have a greater impact on delay-sensitive services (such as audio and video calls). Therefore, it is not suitable for delay-sensitive services (such as audio and video conference calls, remote office, etc.).

[0132] In view of this, the present application provides a method for UDP data transmission, which utilizes the characteristics of guaranteed TCP transmission flow and controllable UDP transmission delay, and is based on UDP proxy UDP penetration technology. A TCP channel between two terminals is established using raw sockets, so that the TCP data packets or TCP messages transmitted between the two terminals can be directly sent and received at the IP layer of the terminal without passing through the kernel TCP protocol stack of the terminal. A layer of TCP header is encapsulated on the UDP data packet, and the UDP data packet is encapsulated to obtain a TCP data packet, and then the TCP data packet is transmitted using the established TCP channel. The problem of UDP traffic being limited by QoS or intercepted by a firewall is solved, and based on raw sockets, data can be directly sent and received in the TCP data packet at the IP layer without passing through the kernel TCP protocol stack. In the process of transmitting TCP data packets, a heavy TCP kernel protocol layer is not introduced, which solves the problem of large transmission delay of TCP transmission when the network environment is not good, and provides flexibility for the application layer to design a low-latency transmission solution, thereby improving the communication experience based on UDP.

[0133] refer to Figure 8 , Figure 8 The figure shows a schematic diagram of a communication system for implementing an embodiment of the present application provided by the embodiment of the present application.

[0134] like Figure 8 As shown, the communication system includes: a first terminal, a second terminal, a NAT gateway device of the first terminal, a NAT gateway device of the second terminal, and a server located in the cloud. The first terminal and the second terminal are respectively located behind their respective NAT gateways and are in a private network. The server includes a STUN server, a relay server, and a signaling server.

[0135] Among them, NAT gateway is a gateway device that supports IP address conversion. It can provide secure and high-performance Internet access services for devices within a private network. It is used to convert between the public network address and port number of the device, and the private network address and port number.

[0136] The STUN server is located in the Internet network and provides a server for client devices (i.e., terminals) to discover public network addresses and ports.

[0137] The relay server is located in the Internet network and is used to provide relay transmission capabilities, providing P2P relay transmission functions for the communication devices of both parties that cannot be directly connected in P2P.

[0138] The signaling server is located in the Internet network and mainly provides signaling interaction capabilities for both communicating parties, such as exchanging address information of the two communicating parties.

[0139] The first terminal and the second terminal include an application (APP) integrated with a P2P software development kit (SDK), such as an audio and video call APP, etc. Among them, the P2P SDK mainly provides P2P penetration capability for the APP, and the APP mainly implements business layer functions, such as audio and video calls.

[0140] In the present application, the first terminal and the second terminal may include: mobile phones, tablet computers, laptop computers, ultra-mobile personal computers (UMPCs), handheld computers, netbooks, personal digital assistants (PDAs), wearable electronic devices, virtual reality devices, and other IOT (internet of things) devices, etc., and the embodiments of the present application are not limited here.

[0141] When transmitting data packets between the first terminal and the second terminal, Figure 8 When the control signaling is transmitted between the first terminal and the second terminal, it can be transmitted according to Figure 8 The signaling transmission channel shown in is used for transmission.

[0142] exist Figure 8In the communication system shown, the devices of the communication system can be interconnected through a communication network. Exemplarily, the communication network can be a wired network or a wireless network. For example, the communication network can be a local area network (LAN) or a wide area network (WAN), such as the Internet. The above-mentioned communication network can be implemented using any known network communication protocol, and the above-mentioned network communication protocol can be various wired or wireless communication protocols, such as Ethernet, universal serial bus (USB), FIREWIRE, global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time division code division multiple access (TD-SCDMA), long term evolution (LTE), Bluetooth, wireless fidelity (Wi-Fi), NFC, voice over Internet protocol (VoIP), communication protocols supporting network slicing architecture or any other suitable communication protocols.

[0143] That is to say, each device in the communication system can form a network (ie, networking) according to certain communication protocols and networking strategies, so that each device in the system can communicate with each other.

[0144] It should be understood that Figure 8 The communication system shown is merely exemplary and should not impose any limitation on the communication system to which the embodiments of the present application are applicable. For example, in other embodiments of the present application, the communication system may further include more terminals, or only include one NAT gateway device, or may further include other servers, etc. The embodiments of the present application are not limited here.

[0145] Fig. 9 The figure shows a schematic diagram of the software architecture inside a terminal provided by the present application. Fig. 9As shown, the terminal includes: an application (APP) module, a TCP proxy (TCP proxy) module, a kernel protocol module and a network card module.

[0146] Among them, the App module (or also called the App layer) can be used to realize the interaction between the mobile phone and the user, such as receiving the operation of the service device providing the service input by the user. The App can be, for example, a system App (such as a smart life App) that comes with the terminal, or it can also be a third-party App, which is not limited in this application.

[0147] The kernel protocol module (or kernel protocol stack module) includes: UDP processing module (or UDP layer), TCP processing module (or TCP layer), IP processing module (or IP layer). Each module included in the kernel protocol module is used to process data packets according to their respective processing methods. For example, the UDP processing module corresponds to the processing method of the UDP protocol layer, the TCP processing module corresponds to the processing method of the TCP protocol layer, and the IP processing module corresponds to the transmission method of the IP protocol layer. The kernel protocol module is usually deployed in an operating system, such as the kernel of a Linux operating system.

[0148] The network card module is used to implement communication with other devices, for example, to communicate with other devices in a wide area network; or to communicate with devices in other external networks via a wide area network, etc. Exemplarily, the network card module may include a wide area network (WAN) communication interface and a local area network (LAN) communication interface, etc.

[0149] The TCP proxy module is used to simulate the TCP handshake process through the original socket and establish a TCP transmission channel with other terminals. In addition, the TCP proxy module is also used to encapsulate the TCP header and IP header on the UDP data packet after receiving the UDP data packet (or UDP message) transmitted by the UDP processing module to form a TCP data packet (or TCP message), so that the TCP data packet is finally sent to the peer device directly at the IP layer without passing through the TCP transport layer. The TCP proxy module is also used to receive the TCP data packet sent by the peer device, decapsulate the data packet, remove the TCP header and IP header, and obtain the UDP data packet from the TCP data packet.

[0150] Among them, the raw socket (SOCK_RAW) can be used to assemble IP data packets by itself and then send the data packets to other terminals. In other words, the raw socket is based on IP data packets and is a socket that processes raw network messages. The raw socket can send and receive data packets that have not passed through the kernel TCP protocol stack.

[0151] It should be understood that Fig. 9 The modules or protocol layers shown are only exemplary and should not impose any restrictions on the internal architecture of the terminal in the present application. Fig. 9 The different structures and modules in the examples shown may also include more or fewer modules, or a combination of multiple different modules, etc. The embodiments of the present application are not limited here.

[0152] The method provided in this application is described below with reference to specific examples.

[0153] Fig.10 The figure shows a schematic flow chart of a UDP data transmission method provided by the present application. Fig.10 In the example shown, the first terminal and the second terminal are both terminal devices located behind their respective NAT gateways in a private network. Among them, the NAT gateway device corresponding to the first terminal is the first NAT gateway, and the NAT gateway device corresponding to the second terminal is the second NAT gateway. Servers located in the cloud The servers located in the cloud include STUN servers, relay servers, and signaling servers. Fig.11 The following table shows the Fig.10 A schematic diagram of a communication system for the described method.

[0154] like Fig.10 As shown, the method includes:

[0155] S301, the first terminal connects to the signaling server through the first NAT gateway and establishes a long connection signaling channel with the signaling server.

[0156] S302, the second terminal connects to the signaling server through the second NAT gateway and establishes a long connection signaling channel with the signaling server.

[0157] S303: The first terminal obtains a TCP type public network address and port number, and a UDP type public network address and port number of the first terminal from a STUN server through a first NAT gateway.

[0158] S304: The second terminal obtains the TCP type public network address and port number, and the UDP type public network address and port number of the second terminal from the STUN server through the second NAT gateway.

[0159] S305: The first terminal and the second terminal exchange their respective TCP and UDP public network addresses and port numbers, as well as the UDP port numbers of the first terminal and the second terminal through the NAT gateway device and the signaling server.

[0160] The UDP port number of the first terminal is the forwarding port of UDP data packets between different modules inside the first terminal, and the IP address corresponding to the forwarding port number is generally a loopback address. The UDP port number of the second terminal is the forwarding port number of UDP data packets between different modules inside the second terminal, and the IP address corresponding to the forwarding port number is generally a loopback address.

[0161] S306: The first terminal and the second terminal perform TCP hole punching and UDP hole punching respectively according to the exchanged TCP type public network address and port number, and UDP type public network address and port number.

[0162] If the TCP hole punching is successful, S307a to S309a are executed after S306 regardless of whether the UDP hole punching is successful.

[0163] If the TCP hole punching fails but the UDP hole punching succeeds, S307b is executed after S306.

[0164] If both TCP hole punching and UDP hole punching are unsuccessful, S307c is executed after S306.

[0165] They will be explained below one by one.

[0166] S307a, the first terminal establishes a TCP transmission channel with the second terminal using a raw socket.

[0167] For example, the TCP proxy module in the first terminal can construct a synchronization (Synchronize, SYN) message according to the TCP protocol through the original socket, and send the SYN message to the second terminal through the first NAT gateway device and the second NAT gateway device. The TCP proxy module in the second terminal receives the SYN message, constructs a synchronization confirmation (Synchronize-Acknowledgement, SYN-ACK) message, and sends the SYN-ACK message to the first terminal through the second NAT gateway device and the first NAT gateway device. After the first terminal receives the SYN-ACK message, it sends an ACK message to the second terminal through the first NAT gateway device and the second NAT gateway device for confirmation. After such a process simulation completes the TCP handshake process, a TCP transmission channel is established.

[0168] S308a, after the TCP transmission channel is successfully established, the first terminal (for example, the TCP proxy module in the first terminal) encapsulates the UDP data packet (or UDP message) of the application layer with a TCP header and an IP header to form a TCP data packet (or TCP message), and then sends the TCP data packet to the second terminal through the TCP transmission channel. The TCP data packet is directly sent to the second terminal at the IP layer of the first terminal. The TCP data packet does not need to be transmitted through the TCP protocol stack of the first terminal kernel during the sending process, for example, Fig.11 The transmission path of the TCP data stream is shown in FIG. The TCP data stream is directly sent at the IP layer of the first terminal, and the TCP data stream does not need to be transmitted through the TCP protocol stack of the first terminal kernel during the sending process.

[0169] For example, the application layer (App module) of the first terminal sends the UDP data packet of the application layer to the TCP proxy module in the first terminal through the loopback address (loopback address), and the TCP proxy module in the first terminal encapsulates the TCP header and the IP header on the UDP data packet to obtain a TCP data packet, and then directly sends data at the IP layer, and the TCP data packet passes through the first NAT gateway device and the second NAT gateway device, and is finally received by the second terminal. The TCP data packet does not need to pass through the kernel TCP protocol stack of the first terminal during the sending process.

[0170] The loopback address is a special address used by the host to send communications to itself, realizing the functions of client and server on one host. The loopback address provides a shortcut for TCP / IP applications and services running on the same device to communicate with each other. If two services on the same host use the loopback address instead of the assigned host address, they can bypass the lower layers of the TCP / IP protocol stack.

[0171] The loopback address is the address set on the loopback interface, which is used to identify the device itself. Generally, devices use 127.0.0.1 as the loopback address of the device by default. When a device sends a data packet to itself, the destination address and source address of the data packet are both the loopback address, but the destination port number and source port number are different.

[0172] For another example, the application layer (App module) of the first terminal can also send the UDP data packet of the application layer to the TCP proxy module in the first terminal through the virtual network device (tun device). This application does not limit the method of transmitting data packets between the application layer (App module) of the first terminal and the TCP proxy module in the first terminal.

[0173] S309a, after receiving the TCP data packet, the second terminal (for example, the TCP proxy module in the second terminal) decapsulates and obtains the UDP data packet. For example, the TCP proxy module in the second terminal removes the TCP header and the IP header in the TCP data packet, thereby obtaining the UDP data packet from the TCP data packet.

[0174] After S309a, the second terminal (eg, the TCP proxy module in the second terminal) sends the UDP data packet to the application layer of the second terminal via the loopback address for processing, thereby finally realizing P2P communication of the UDP data packet between the first terminal and the second terminal.

[0175] The specific process of S308a and S309a is explained below with reference to a specific example.

[0176] Assumptions:

[0177] The App module of the first terminal sends UDP data at the loopback address: 127.0.0.1 and the port number "1000". The port number "1000" is the UDP port number of the first terminal, and the UDP port number of the first terminal can be understood as the port number of the App module of the first terminal that sends the UDP data.

[0178] The TCP proxy module of the first terminal receives UDP data at the loopback address: 127.0.0.1 and the port number "1001".

[0179] The TCP proxy module of the first terminal sends a TCP data packet (TCP message) on the TCP type private network address: 1.2.3.4, port number "1002", and the address 1.2.3.4:1002 is mapped to 100.1.2.3:8080 after the first NAT gateway mapping. "100.1.2.3:8080" is the TCP type public network address and port number of the first terminal.

[0180] For the purpose of distinction, "127.0.0.1:1000" is referred to as the loopback address and port number of the App module of the first terminal;

[0181] "127.0.0.1:1001" is referred to as the loopback address and port number of the TCP proxy module of the first terminal;

[0182] "1.2.3.4:1002" is referred to as the private network address and port number of the first terminal;

[0183] “100.1.2.3:8080” is the public network address and port number of the first terminal;

[0184] The TCP proxy module of the second terminal receives the TCP data packet sent by the first terminal at the TCP type address: 5.6.7.8 and the port number "2002". The address 5.6.7.8:2002 is mapped to 110.5.6.7;9090 by the second NAT gateway. "110.5.6.7:9090" is the TCP type public network address and port number of the second terminal.

[0185] The TCP proxy module of the second terminal sends UDP data to the App module of the second terminal at the loopback address: 127.0.0.1 and the port number "2001".

[0186] The App module of the second terminal receives UDP data at the loopback address: 127.0.0.1 and the port number "2000". The port number "2000" is the UDP port number of the second terminal, and the UDP port number of the second terminal can be understood as the port number of the App module of the second terminal that receives the UDP data.

[0187] In order to distinguish: "127.0.0.1:2000" is called the loopback address and port number of the App module of the second terminal;

[0188] "127.0.0.1:2001" is referred to as the loopback address and port number of the TCP proxy module of the second terminal;

[0189] “5.6.7.8:2002” is the private network address and port number of the second terminal;

[0190] "110.5.6.7:9090" is the public network address and port number of the second terminal;

[0191] Fig.12 The figure shows the process of processing the UDP data inside the first terminal.

[0192] In S308a, if Fig.12 As shown in Figure a, the App module of the first terminal first generates UDP data to be sent, which is the App Payload part of the UDP data packet. The App module of the first terminal starts to send the UDP data through the UDP socket. The UDP data first passes through the UDP processing module, which encapsulates the UDP data with a UDP header (UDP head). Fig.12 As shown in Figure b in the figure. The UDP packet header carries: source port number: 1000, and destination port number: 1001. The source port number is: the port number of the App module of the first terminal; the destination port number is: the port number of the TCP proxy module of the first terminal. The source port number and the destination port number are both forwarding port numbers of UDP data between different modules in the first terminal.

[0193] The UDP processing module transmits the UDP header and App Payload to the IP processing module, which encapsulates the IP header to obtain a UDP data packet. For the sake of distinction, the IP header here is called the first IP header. The structure of the UDP data packet is as follows: Fig.12 As shown in Figure c. The first IP header carries: source address: 127.0.0.1; destination address: 127.0.0.1; thus forming a UDP data packet (or also called a UDP message), which is then processed by the kernel protocol stack and sent to the TCP proxy module of the first terminal. Among them, the source address in the first IP header is: the loopback address of the App module of the first terminal; the destination address is: the loopback address of the TCP proxy module of the first terminal.

[0194] In other words, the source address and port number (port) of the UDP data packet are respectively: 127.0.0.1:1000, and the destination address and port are respectively: 127.0.0.1:1001. The source address and destination address of the UDP data packet are located in the first IP header of the UDP data packet, and the source port number and destination port number of the UDP data packet are located in the UDP header of the UDP data packet.

[0195] The structure of the UDP data packet received by the TCP proxy module of the first terminal is as follows Fig.12 As shown in Figure c. After the TCP proxy module of the first terminal receives the UDP data packet, it starts to encapsulate and forward the App Payload part of the UDP data packet. The TCP proxy module of the first terminal first removes the UDP header (UDP head) of the UDP data packet, adds a UDP overlay header before the App Payload, and then re-encapsulates the TCP header (TCP head) and IP header (IP head) to form a TCP data packet. For the sake of distinction, the IP header of the TCP data packet is called the second IP header, and the TCP header of the TCP data packet is called the second TCP header. The structure of the TCP data packet obtained after the TCP proxy module of the first terminal processes the received UDP data packet is shown as follows: Fig.12 As shown in Figure d. Fig.12 Figure e in FIG. 1 is a schematic diagram of another example of a TCP data packet structure provided by this application.

[0196] Among them, the source port carried in the UDP overlay is: 1000, and the destination port number is: 2000. The source port is: the port number of the App module inside the first terminal that sends the UDP data, and the destination port number is: the port number of the App module inside the second terminal that receives the UDP data. The source address defaults to the loopback address, and the destination address defaults to the loopback address. In other words, the source address is: the loopback address of the App module of the first terminal that sends the UDP data: 127.0.0.1, and the destination address is: the loopback address of the App module of the second terminal that receives the UDP data: 127.0.0.1.

[0197] Since the source address and destination address in the UDP overlay are both loopback addresses: 127.0.0.1 by default, the UDP overlay may not carry the source address and destination address 127.0.0.1. Optionally, the UDP overlay may also carry the source address and destination address, both of which are loopback addresses: 127.0.0.1. In other words, the information carried in the UDP overlay is: the port number of the App module of the first terminal that sends the UDP data and the port number of the App module of the second terminal that receives the UDP data. Optionally, the UDP overlay may also carry: the loopback address of the App module of the first terminal that sends the UDP data and the loopback address of the App module of the second terminal that receives the UDP data, and the loopback addresses are both: 127.0.0.1.

[0198] The source address of the TCP data packet is: 1.2.3.4, and the source port number is: 1002. The destination address of the TCP data packet is: 110.5.6.7, and the destination port number is: 9090. The source address and port number of the TCP data packet are the private network address and port number of the first terminal; the destination address and port number are the public network address and port number of the second terminal. The source address and destination address of the TCP data packet are located in the second IP header of the TCP data packet, and the source port number and destination port number of the TCP data packet are located in the second TCP header of the TCP data packet.

[0199] Fig.12 Figure e in FIG. 1 is a schematic diagram of another example of a TCP data packet structure provided by this application. Fig.12 The App Payload part shown in Figure e is equivalent to Fig.12 The App Payload and UDP overlay parts are shown in Figure d. In other words, the UDP overlay field can also be set in the App Payload part of the TCP data packet.

[0200] It should be understood that Fig.12The TCP data structures shown in Figures d and e are merely exemplary and should not limit the TCP data structures provided by the present application. In other embodiments of the present application, the TCP data packet may also be in other structures, which are not limited by the present application.

[0201] Afterwards, the TCP proxy module of the first terminal directly sends the TCP data packet to the first NAT gateway through the IP processing module and the network card module of the first terminal, without passing through the TCP transport layer of the first terminal.

[0202] After receiving the TCP data packet, the first NAT gateway modifies the source address of the TCP data packet to 100.1.2.3 and the source port number to 8080. The destination address and destination port number in the TCP data packet remain unchanged, and the UDP overlay field also remains unchanged. The first NAT gateway sends the TCP data packet with the modified source address and port number to the second NAT gateway.

[0203] In other words, the source address of the TCP data packet sent by the first NAT gateway to the second NAT gateway is: 100.1.2.3, the source port number is: 8080, the destination address is: 110.5.6.7, and the destination port number is: 9090.

[0204] After the second NAT gateway receives the TCP data packet, due to the previous TCP hole punching, the second NAT gateway maintains the correspondence between the private network address of the second terminal and the public network address of the first terminal. According to the correspondence, the second NAT gateway can determine that the destination IP and port number corresponding to "100.1.2.3:8080" are: 5.6.7.8:2002. Therefore, the second NAT gateway modifies the destination address and port number of the TCP data packet to: 5.6.7.8:2002, and the source address and port number remain unchanged, still: 100.1.2.3:8080, and then sends the TCP data packet to the TCP proxy module in the second terminal.

[0205] Fig.13 The figure shows the process of processing the TCP data packet inside the second terminal.

[0206] In S309a, Fig.13 Figure a in the figure is a schematic diagram of the TCP data packet structure received by the TCP proxy module in the second terminal. Among them, the third TCP packet header carries: source port number: 8080, destination port number: 2002. The third IP packet header carries: source address: 110.1.2.3, destination address: 5.6.7.8.

[0207] After receiving the TCP data packet, the TCP proxy module in the second terminal removes the third TCP header and the third IP header in the TCP data packet, obtains the UDP overlay and App Payload parts in the data packet, and determines that the destination address is the loopback address 127.0.0.1 according to the destination port carried in the UDP overlay: 2000, and then determines that the destination address is the loopback address 127.0.0.1 according to the destination address and source address carried in the UDP overlay, or the default destination address and source address. The UDP data packet is assembled according to the UDP overlay and App Payload parts. The destination port number of the UDP data packet is: the port number of the App module of the second terminal: 2000, the destination address is the loopback address: 127.0.0.1, the source address is also the loopback address: 127.0.0.1, and the source port number is the port number of the TCP proxy module of the second terminal: 2001. Fig.13 FIG. b in FIG. 1 is a schematic diagram of the UDP data packet format obtained after the TCP proxy module in the second terminal processes the TCP data packet. Among them, the fourth IP packet header (fourth IP head) carries: source address: 127.0.0.1. Destination address: 127.0.0.1. The fourth UDP packet header (fourth UDP head) carries: source port: 2001, destination port: 2000.

[0208] The TCP proxy module in the second terminal sends the UDP data packet to the IP processing module of the second terminal. The IP processing module removes the IP header (the fourth IP head) of the UDP data packet, and then sends the remaining UDP header (the fourth UDP head) and the App Payload part to the UDP processing module, such as Fig.13 The UDP processing module removes the UDP header (the fourth UDP head) and then sends the remaining App Payload part to the App module of the second terminal, as shown in FIG. Fig.13 As shown in Figure d, it is finally processed by the application module of the second terminal, and finally the P2P communication of the UDP data packet between the first terminal and the second terminal is realized.

[0209] For another example, the TCP proxy module in one terminal can also send the UDP data packets of the application layer to the application layer (App module) of the second terminal through the virtual network device (tun device). This application does not limit the method of transmitting data between the TCP proxy module of the second terminal and the application layer (App module) in the second terminal.

[0210] Compared with the existing TCP data packets, the TCP header (TCP head) is followed by the application data (App Payload). The format of the TCP data packet provided by this application is as follows Fig.14As shown, a UDP overlay field is also included after the TCP packet header (TCP head).

[0211] Fig.15 The figure shows a schematic diagram of the TCP header and application data (App Payload) format of a TCP data packet provided by the present application. Since the TCP transmission channel between the first terminal and the second terminal in the present application is established by using the original socket method, the TCP header in the TCP data packet is encapsulated by the TCP proxy module of the terminal itself, and will be retransmitted when packet loss occurs, but the same (duplicate) sequence number (SequenceNumber) of the data packet will not appear in the network transmission process, and the sequence numbers in different TCP data packets are monotonically increasing. Among them, Fig.15 The data portion corresponds to the UDP overlay and application data (App Payload) of the TCP data packet provided by this application.

[0212] Traditional TCP is an ordered and reliable transmission protocol. In the prior art, when a TCP data packet is lost, it will be retransmitted. During the network transmission process, different TCP data packets will carry the same (repeated) sequence number, that is, data packets with the same (repeated) sequence number (Sequence Number) will appear during the network transmission process.

[0213] If the TCP hole punching fails but the UDP hole punching succeeds, S307b is executed after S306.

[0214] S307b: The first terminal and the second terminal transmit UDP data packets through the NAT gateway using a traditional UDP mode.

[0215] For example, the first terminal and the second terminal use Figure 2 The method shown uses UDP to perform NAT penetration technology to achieve the transmission of UDP data packets between the first terminal and the second terminal.

[0216] If both TCP hole punching and UDP hole punching fail, S307c is executed after S306.

[0217] S307c, the first terminal and the second terminal transmit UDP data packets by forwarding through the relay server.

[0218] For example, the first terminal sends a UDP data packet to the relay server through the first NAT gateway, and after the relay server receives the UDP data packet, it sends it to the second terminal through the second NAT gateway.

[0219] The UDP data transmission method provided by the present application is aimed at the problem that the flow of UDP data faces the QoS speed limit of the operator or is intercepted by the firewall when using UDP for NAT penetration communication. The UDP penetration technology based on the TCP proxy uses the original socket to establish a TCP channel between the two terminals, so that the TCP data packets or TCP messages transmitted between the two terminals can be directly sent and received at the IP layer without passing through the kernel TCP protocol stack. A layer of TCP header is encapsulated on the UDP data packet, and the UDP data packet is encapsulated to obtain a TCP data packet, and then the TCP data packet is transmitted using the established TCP channel. The problem of UDP traffic being limited by QoS or intercepted by the firewall is solved, thereby ensuring that UDP traffic can be treated equally with TCP and improving the P2P communication experience of UDP. In addition, based on the original socket, the TCP data packet can be directly sent and received at the IP layer without passing through the kernel TCP protocol stack, and the heavy TCP kernel protocol layer is not introduced in the process of transmitting the TCP data packet, which solves the problem of large transmission delay of TCP transmission when the network environment is not good, and facilitates the business to design a low-latency transmission solution that meets business needs based on the UDP protocol at the application layer, greatly improving the design flexibility, thereby improving the communication experience based on UDP.

[0220] Optionally, in the above-mentioned embodiment, as another possible implementation method, in S306, only TCP hole punching may be performed without UDP hole punching. In this case, the first terminal and the second terminal only need to exchange TCP type public network addresses and port numbers, and the UDP port numbers of the first terminal and the second terminal, without exchanging UCP type public network addresses and port numbers. If TCP hole punching is successful, the above-mentioned S307a to S309a are executed after S306. If TCP hole punching is unsuccessful, the above-mentioned S307c is directly executed after S306, that is, transmission is performed by means of a relay server. In this implementation method, only TCP hole punching is performed. If TCP hole punching fails, relay transmission is directly performed. On the basis of solving the problem that UDP traffic is limited by QoS or intercepted by a firewall and the transmission delay of TCP transmission is large when the network environment is not good, it is simpler in design, less complex, and easier to implement.

[0221] It should be understood that the above is only to help those skilled in the art better understand the embodiments of the present application, rather than to limit the scope of the embodiments of the present application. According to the above examples given, those skilled in the art can obviously make various equivalent modifications or changes, for example, some steps in each method of the above method may not be necessary, or some new steps may be added. Or a combination of any two or any multiple embodiments of the above. Such modifications, changes or combined solutions also fall within the scope of the embodiments of the present application.

[0222] It should also be understood that the parameter codes in the above-mentioned embodiments are merely exemplary and should not impose any restrictions on the parameter code format carried by the information in the embodiments of the present application. In other embodiments of the present application, the parameter code format carried by the information in the embodiments of the present application may also be in other formats, etc. The embodiments of the present application are not limited here.

[0223] It should also be understood that the division of the methods, situations, categories and embodiments in the embodiments of the present application is only for the convenience of description and should not constitute a special limitation. The features of various methods, categories, situations and embodiments can be combined without contradiction.

[0224] It should also be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. The size of the sequence number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0225] It should also be understood that the above description of the embodiments of the present application focuses on emphasizing the differences between the various embodiments. The same or similar points that are not mentioned can be referenced to each other. For the sake of brevity, they will not be repeated here.

[0226] The above combination Figure 1-Figure 15 An embodiment of a method for UDP data transmission provided by an embodiment of the present application is described, and the terminal device provided by an embodiment of the present application is described below.

[0227] The present embodiment can divide each device (comprising the above-mentioned first terminal, the second terminal, the NAT gateway device and each server) into functional modules according to the above method. For example, each function can be divided into each functional module, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware. It should be noted that the division of modules in the present embodiment is schematic, which is only a kind of logical function division, and other division methods can be arranged during actual implementation.

[0228] It should be noted that the relevant contents of each step involved in the above method embodiment can all be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0229] The terminal device (including the first terminal and the second terminal) provided in the embodiment of the present application is used to perform any UDP data transmission method provided in the above method embodiment, so that the same effect as the above implementation method can be achieved. In the case of an integrated unit, the terminal device or server may include a processing module, a storage module and a communication module. Among them, the processing module can be used to control and manage the actions of the terminal device or server. For example, it can be used to support the terminal device or server to execute the steps performed by the processing unit. The storage module can be used to support the storage of program codes and data, etc. The communication module can be used to support the communication between the terminal device or server and other devices.

[0230] Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other terminal devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0231] Fig.16 The figure shows a schematic diagram of a terminal device structure provided by the present application. Fig.16 As shown, the terminal device 400 may include a processor 410, an external memory interface 420, an internal memory 421, a universal serial bus (USB) interface 430, an antenna 1, an antenna 2, a mobile communication module 440, a wireless communication module 460, an audio module 470, a speaker 470A, a receiver 470B, a microphone 470C, an earphone interface 470D, a sensor module 480, and the like.

[0232] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 400. In other embodiments of the present application, the terminal device 400 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0233] The processor 410 may include one or more processing units, for example, the processor 410 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0234] The processor 410 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 410 is a cache memory. The memory may store instructions or data that the processor 410 has just used or cyclically used. If the processor 410 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 410, and thus improves the efficiency of the system.

[0235] The wireless communication function of the terminal device 400 can be implemented through antenna 1, antenna 2, mobile communication module 440, wireless communication module 460, modem processor and baseband processor.

[0236] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 400 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0237] The mobile communication module 440 can provide solutions for wireless communications including 2G / 4G / 4G / 4G, etc., applied to the terminal device 400. The mobile communication module 440 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 440 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 440 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 440 can be set in the processor 410. In some embodiments, at least some of the functional modules of the mobile communication module 440 can be set in the same device as at least some of the modules of the processor 410.

[0238] The wireless communication module 460 can provide wireless communication solutions including wireless local area networks (WLAN) applied on the terminal device 400, such as wireless fidelity (Wi-Fi) network, bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 460 can be one or more devices integrating at least one communication processing module. The wireless communication module 460 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signal and filters it, and sends the processed signal to the processor 410. The wireless communication module 460 can also receive the signal to be sent from the processor 410, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0239] In some embodiments, the antenna 1 of the terminal device 400 is coupled to the mobile communication module 440, and the antenna 2 is coupled to the wireless communication module 460, so that the terminal device 400 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time division code division multiple access (TD-SCDMA), LTE, BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS) and / or satellite-based augmentation system (SBAS).

[0240] The terminal device 400 implements the display function through a GPU, a display screen 494, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 494 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 410 may include one or more GPUs, which execute program instructions to generate or change display information.

[0241] The display screen 494 is used to display images, videos, etc. The display screen 494 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 400 may include 1 or N display screens 494, where N is a positive integer greater than 1.

[0242] The terminal device 400 can realize the shooting function through ISP, camera 493, video codec, GPU, display screen 494 and application processor.

[0243] ISP is used to process the data fed back by camera 493. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 493.

[0244] The camera 493 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 400 may include 1 or N cameras 493, where N is a positive integer greater than 1.

[0245] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal device 400 is performing frequency selection, the digital signal processor is used to perform Fourier transform on the frequency energy.

[0246] Video codecs are used to compress or decompress digital videos. The terminal device 400 may support one or more video codecs. Thus, the terminal device 400 may play or record videos in various coding formats, such as moving picture experts group (MPEG) 1, MPEG2, MPEG4, MPEG4, etc.

[0247] The external memory interface 420 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 400. The external memory card communicates with the processor 410 through the external memory interface 420 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0248] The internal memory 421 can be used to store computer executable program codes, which include instructions. The processor 410 executes various functional applications and data processing of the terminal device 400 by running the instructions stored in the internal memory 421. The internal memory 421 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the terminal device 400 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 421 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0249] The terminal device 400 can implement audio functions such as music playing and recording through the audio module 470, the speaker 470A, the receiver 470B, the microphone 470C, the headphone jack 470D, and the application processor.

[0250] The audio module 470 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 470 can also be used to encode and decode audio signals. In some embodiments, the audio module 470 can be arranged in the processor 410, or some functional modules of the audio module 470 can be arranged in the processor 410.

[0251] The speaker 470A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The terminal device 400 can listen to music or listen to a hands-free call through the speaker 470A.

[0252] The receiver 470B, also called a "handset", is used to convert audio electrical signals into sound signals. When the terminal device 400 receives a call or voice message, the voice can be received by placing the receiver 470B close to the human ear.

[0253] Microphone 470C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 470C to input the sound signal into the microphone 470C. The terminal device 400 can be provided with at least one microphone 470C. In other embodiments, the terminal device 400 can be provided with two microphones 470C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the terminal device 400 can also be provided with three, four or more microphones 470C to realize the collection of sound signals, noise reduction, identification of sound sources, realization of directional recording function, etc.

[0254] The earphone interface 470D is used to connect a wired earphone and can be a USB interface 430 or a 4.4 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0255] The sensor module 480 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0256] Of course, the terminal device 400 may also include a charging management module, a power management module, a battery, buttons, indicators, and one or more SIM card interfaces, etc., and the embodiments of the present application do not impose any restrictions on this.

[0257] The USB interface 430 is an interface that complies with USB standard specifications, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 430 may be used to connect a charger to charge the terminal device 400, and may also be used to transmit data between the terminal device 400 and peripheral devices.

[0258] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 400. In other embodiments of the present application, the terminal device 400 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0259] It should be understood that the specific process of the terminal device 400 executing the above corresponding steps can refer to the relevant description of the first terminal or the second terminal executing the steps described in the various embodiments described in the above text. For the sake of brevity, it is not repeated here.

[0260] The present application also provides a chip system, such as Fig.17 As shown, the chip system includes at least one processor 510 and at least one interface circuit 520. The processor 510 and the interface circuit 520 can be interconnected via lines. For example, the interface circuit 520 can be used to receive signals from other devices (such as terminal equipment 400). For another example, the interface circuit 520 can be used to send signals to other devices (such as processor 510). Exemplarily, the interface circuit 520 can read instructions stored in the memory and send the instructions to the processor 510. When the instructions are executed by the processor 510, the chip system can execute the various steps performed by the first terminal or the second terminal in the above-mentioned embodiment. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0261] Fig.18 A schematic block diagram of another example of a communication device 600 provided in an embodiment of the present application is shown. The communication device 600 may correspond to the first terminal or the second terminal described in the above method embodiment. It may also be a chip or component applied to the first terminal or the second terminal, and each module or unit in the communication device 600 is used to execute each action or processing performed by the first terminal or the second terminal described in the above method embodiment, such as Fig.18 As shown, the communication device 600 may include: a processing unit 610 and a communication unit 620. Optionally, the communication device 600 may further include a storage unit 630.

[0262] It should be understood that the specific process of each unit in the communication device 600 performing the above corresponding steps can be referred to in conjunction with the above Figure 5 , Fig.10 , Fig.11 and Fig.12 as well as Fig.13 The relevant descriptions of the steps executed by the first terminal or the second terminal in each embodiment are described in detail and are not repeated here for the sake of brevity.

[0263] Optionally, the communication unit 620 may include a receiving unit (module) and a sending unit (module), which are used to perform the steps of the first terminal or the second terminal receiving information and sending information in the aforementioned various method embodiments. The storage unit 1300 is used to store instructions executed by the processing unit 610 and the communication unit 620. The processing unit 610, the communication unit 620 and the storage unit 630 are connected in communication, the storage unit 630 stores instructions, the processing unit 610 is used to execute the instructions stored in the storage unit, and the communication unit 620 is used to perform specific signal transmission and reception under the drive of the processing unit 610.

[0264] It should be understood that the communication unit 620 may be a transceiver, an input / output interface or an interface circuit, etc., for example, Fig.16 The wireless communication module 460 in the illustrated embodiment is implemented. The storage unit may be a memory, for example, Fig.16 The external memory interface 420 and the internal memory 421 in the illustrated embodiment are implemented. The processing unit 610 may be composed of Fig.16 The processor 410 in the illustrated embodiment may be implemented by the processor 410 , the external memory interface 420 , and the internal memory 421 .

[0265] It should also be understood that Fig.18 The communication device 600 shown may be a first terminal or a second terminal, or the first terminal or the second terminal may include Fig.16 A communication device 600 is shown.

[0266] It should also be understood that the division of units in the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And the units in the device can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some units can also be implemented in the form of software calling through processing elements, and some units can be implemented in the form of hardware. For example, each unit can be a separately established processing element, or it can be integrated in a certain chip of the device. In addition, it can also be stored in the memory in the form of a program, and called and executed by a certain processing element of the device. The processing element here can also be called a processor, which can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each unit above can be implemented by an integrated logic circuit of hardware in the processor element or in the form of software calling through a processing element. In one example, the unit in any of the above devices may be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA), or a combination of at least two of these integrated circuit forms. For another example, when the unit in the device can be implemented in the form of a processing element scheduler, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0267] The embodiment of the present application also provides a device, which is included in the first terminal or the second terminal, and the device has the function of implementing the first terminal or the second terminal in any method of the above embodiment. The function can be implemented by hardware, or the corresponding software can be implemented by hardware. The hardware or software includes at least one module or unit corresponding to the above function. For example, a detection module or unit, a display module or unit, a determination module or unit, and a calculation module or unit, etc.

[0268] The embodiment of the present application also provides a UDP data transmission system, which includes: the first terminal, the second terminal, the first NAT gateway device, the second NAT gateway device and the server provided in the above method embodiment. The server includes a STUN server, a relay server and a signaling server.

[0269] The embodiment of the present application also provides a computer-readable storage medium for storing computer program code, wherein the computer program includes instructions for executing any one of the UDP data transmission methods provided in the above embodiment of the present application. The readable medium may be a read-only memory (ROM) or a random access memory (RAM), which is not limited in the embodiment of the present application.

[0270] The present application also provides a computer program product, which includes instructions. When the instructions are executed, the terminal and the server perform corresponding operations corresponding to the above method.

[0271] The embodiment of the present application also provides a chip located in a communication device, the chip comprising: a processing unit and a communication unit, the processing unit, for example, may be a processor, the communication unit, for example, may be an input / output interface, a pin or a circuit, etc. The processing unit may execute computer instructions to enable the communication device to execute any of the UDP data transmission methods provided in the embodiment of the present application.

[0272] Optionally, the computer instructions are stored in a storage unit.

[0273] Optionally, the storage unit is a storage unit within the chip, such as a register, a cache, etc. The storage unit can also be a storage unit located outside the chip within the terminal, such as a ROM or other types of static storage devices that can store static information and instructions, random RAM, etc. Among them, the processor mentioned in any of the above can be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned feedback information transmission method. The processing unit and the storage unit can be decoupled and respectively set on different physical devices, and connected by wired or wireless means to implement the respective functions of the processing unit and the storage unit to support the system chip to implement the various functions in the above-mentioned embodiments. Alternatively, the processing unit and the memory can also be coupled on the same device.

[0274] Among them, the terminal device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0275] It is understood that the memory in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a ROM, a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasable PROM, EPROM), an electrically erasable programmable read-only memory (electrically EPROM, EEPROM) or a flash memory. The volatile memory can be a RAM, which is used as an external cache. There are many different types of RAM, such as static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (direct rambus RAM, DRRAM).

[0276] Various objects such as various messages / information / equipment / network elements / systems / devices / actions / operations / processes / concepts that may appear in this application are named. It can be understood that these specific names do not constitute a limitation on the relevant objects. The names assigned may change with factors such as scenarios, contexts or usage habits. The understanding of the technical meaning of the technical terms in this application should be mainly determined from the functions and technical effects embodied / executed in the technical scheme.

[0277] In the various embodiments of the present application, unless otherwise specified or provided for in any logical conflict, the terms and / or descriptions between the different embodiments are consistent and may be referenced to each other, and the technical features in the different embodiments may be combined to form new embodiments according to their inherent logical relationships.

[0278] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0279] The method in the embodiment of the present application can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server that integrates one or more available media.

[0280] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0281] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0282] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0283] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0284] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a readable storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned readable storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0285] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for UDP data transmission, characterized in that: The method comprises: The first terminal and the second terminal perform a first TCP hole punching according to the TCP type public network address and port number of the first terminal and the TCP type public network address and port number of the second terminal; Determining that the first TCP hole punching is successful; The first terminal establishes a TCP transmission channel with the second terminal by using a raw socket; The first terminal sends a TCP data packet to the second terminal through the TCP transmission channel, and the TCP data packet includes a TCP packet header, an IP packet header and UDP data.

2. The method according to claim 1, characterized in that Before performing the first TCP hole punching, the method further includes: The first terminal obtains a TCP type public network address and port number of the second terminal, and a first port number, and the second terminal receives a UDP data packet through the first port number, the UDP data packet including the UDP data; The first terminal sends a TCP type public network address and port number of the first terminal and a second port number to the second terminal, and the first terminal sends a UDP data packet through the second port number, and the UDP data packet includes the UDP data.

3. The method according to claim 1 or 2, characterized in that: The first terminal sends a TCP data packet to the second terminal through the TCP transmission channel, including: The first terminal sends the TCP data packet to a first NAT gateway; The source address and port number of the TCP data packet are respectively the private network address and port number of the TCP type of the first terminal; The destination address and port number of the TCP data packet are respectively the public network address and port number of the TCP type of the second terminal.

4. The method according to any one of claims 1 to 3, characterized in that The TCP data packet also includes a first field; The first field includes: a first port number and a second port number; or, The first field includes: a first port number and a second port number, and a first address corresponding to the first port number and a second address corresponding to the second port number; The second terminal receives a UDP data packet through the first port number and the first address, and the UDP data packet includes the UDP data. The first terminal sends a UDP data packet through the second port number and the second address, and the UDP data packet includes the UDP data.

5. The method according to claim 4, characterized in that The first field is a UDP overlay field, and the first address and the second address are both loopback addresses; or, The UDP data includes the first field.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: The first terminal and the second terminal perform a second TCP hole punching; Determining that the second TCP hole punching is unsuccessful; The first terminal and the second terminal perform a first UDP hole punching; Determine that the first UDP hole punching is successful; The first terminal establishes a UDP transmission channel with the second terminal; The first terminal sends UDP data to the second terminal through the UDP transmission channel.

7. The method according to claim 6, characterized in that Before performing the first UDP hole punching, the method further includes: The first terminal obtains the UDP type public network address and port number of the second terminal; The first terminal sends the UDP type public network address and port number of the first terminal to the second terminal; The first terminal and the second terminal perform a first UDP hole punching, including: The first terminal and the second terminal perform the first UDP hole punching according to the UDP type public network address and port number of the first terminal and the UDP type public network address and port number of the second terminal.

8. The method according to claim 6 or 7, characterized in that: The method further comprises: The first terminal and the second terminal perform a third TCP hole punching; Determining that the third TCP hole punching is unsuccessful; The first terminal and the second terminal perform a second UDP hole punching; Determine that the second UDP hole is not successful; The first terminal sends UDP data to the second terminal through a relay device.

9. A method for UDP data transmission, characterized in that: The method comprises: The second terminal and the first terminal perform the first TCP hole punching according to the public network address and port number of the TCP type of the second terminal and the public network address and port number of the TCP type of the first terminal; Determining that the first TCP hole punching is successful; The second terminal establishes a TCP transmission channel with the first terminal by using a raw socket; The second terminal receives a TCP data packet from the first terminal through the TCP transmission channel, wherein the TCP data packet includes a TCP packet header, an IP packet header, and UDP data.

10. The method according to claim 9, characterized in that The method further comprises: The second terminal decapsulates the TCP data packet and obtains the UDP data in the TCP data packet.

11. The method according to claim 9 or 10, characterized in that, before performing the first TCP hole punching, the method further comprises: The second terminal obtains a TCP-type public network address and port number of the first terminal, and a second port number, and the first terminal sends a UDP data packet through the second port number, where the UDP data packet includes the UDP data; The second terminal sends the TCP type public network address and port number of the second terminal and the first port number to the first terminal, and the second terminal receives a UDP data packet through the first port number, and the UDP data packet includes the UDP data.

12. The method according to any one of claims 9 to 11, characterized in that The second terminal receives the TCP data packet from the first terminal through the TCP transmission channel, including: The second terminal receives the TCP data packet sent by the second NAT gateway; The destination address and port number of the TCP data packet are respectively the private network address and port number of the TCP type of the second terminal; The source address and port number of the TCP data packet are respectively the public network address and port number of the TCP type of the first terminal.

13. The method according to any one of claims 9 to 12, characterized in that The TCP data packet also includes a first field; The first field includes: a first port number and a second port number; or, The first field includes: a first port number and a second port number, and a first address corresponding to the first port number and a second address corresponding to the second port number; The second terminal receives a UDP data packet through the first port number and the first address, and the UDP data packet includes the UDP data. The first terminal sends a UDP data packet through the second port number and the second address, and the UDP data packet includes the UDP data.

14. The method according to claim 13, characterized in that The first field is a UDP overlay field, and the first address and the second address are both loopback addresses; or, The UDP data includes the first field.

15. The method according to any one of claims 9 to 14, characterized in that The method further comprises: The second terminal and the first terminal perform a second TCP hole punching; Determining that the second TCP hole punching is unsuccessful; The second terminal and the first terminal perform a first UDP hole punching; Determine that the first UDP hole punching is successful; The second terminal establishes a UDP transmission channel with the first terminal; The second terminal receives the UDP data from the first terminal through the UDP transmission channel.

16. The method according to claim 15, characterized in that Before performing the first UDP hole punching, the method further includes: The second terminal obtains the UDP type public network address and port number of the first terminal; The second terminal sends the UDP type public network address and port number of the second terminal to the first terminal; The second terminal and the first terminal perform a first UDP hole punching, including: The second terminal and the first terminal perform the first UDP hole punching according to the UDP type public network address and port number of the second terminal and the UDP type public network address and port number of the first terminal.

17. The method according to claim 15 or 16, characterized in that The method further comprises: The second terminal and the first terminal perform a third TCP hole punching; Determining that the third TCP hole punching is unsuccessful; The second terminal and the first terminal perform a second UDP hole punching; Determine that the second UDP hole is not successful; The second terminal receives the UDP data from the first terminal through the relay device.

18. A terminal device, characterized in that: include: Processor and memory; The processor is coupled to a memory, and the memory stores program instructions. When the program instructions stored in the memory are executed by the processor, the method according to any one of claims 1 to 8 is executed, or the method according to any one of claims 9 to 17 is executed.

19. A UDP data transmission system, characterized in that: The system comprises a first terminal and a second terminal, wherein the terminal device is used to execute the method according to any one of claims 1 to 8, and the second terminal is used to execute the method according to any one of claims 9 to 17.

20. A computer-readable storage medium, characterized in that: A computer program is stored in the computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the method according to any one of claims 1 to 8, or executes the method according to any one of claims 9 to 17.

21. A chip, characterized in that: include: A processor, configured to call and run a computer program from a memory, so that a communication device equipped with the chip executes a method as claimed in any one of claims 1 to 8, or executes a method as claimed in any one of claims 9 to 17.

Images