Method for controlling audio transmission based on network address translation and related device

Abstract

This application provides an audio transmission control method and related apparatus based on cross-network address translation, applied to an audio transmission control system. It acquires device information and network information of a first terminal and a second terminal, and determines a mapping database between terminal device identifiers and network information based on the device and network information. Upon receiving an audio transmission request from the first terminal, it queries the wide area network information of the second terminal from the mapping database. Then, it generates an audio transmission scheduling instruction based on the network information and the audio transmission request. An audio transmission channel is established between the first and second terminals based on the mapping database and third network information. The audio transmission scheduling instruction is sent to a forwarding server so that the audio data from the first terminal is forwarded to the second terminal via the audio transmission channel through the forwarding server. This improves the success rate of IoT device connections and reduces audio transmission latency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an audio transmission control method and related apparatus based on cross-network address translation. Background Technology

[0002] Currently, most traditional IoT devices (such as speakers and smart panels) operate in private network environments and lack public IP addresses. When these devices engage in bidirectional audio interaction across different carriers and regions, they are easily obstructed by Network Address Translation (NAT) gateways, resulting in a low success rate for point-to-point connections. Furthermore, insufficient compatibility with different NAT types further increases the connection establishment failure rate in heterogeneous network scenarios with symmetric or cone NAT, making it difficult to meet the cross-network communication needs of IoT devices. Simultaneously, relying on a single relay channel for data transmission significantly increases audio data transmission latency, failing to meet the real-time requirements of bidirectional audio interaction. Current methods largely depend on single connection establishment strategies or fixed relay forwarding mechanisms, lacking efficient traversal strategies for different NAT types in complex network environments, and also lacking flexible transmission channel switching and optimization mechanisms, thus limiting the application of traditional IoT devices in bidirectional audio interaction within wide area network scenarios.

[0003] Therefore, how to achieve a high success rate for IoT devices to connect across NAT wide area networks while ensuring low latency and stability of bidirectional audio transmission is an urgent problem to be solved. Summary of the Invention

[0004] This application provides an audio transmission control method and related apparatus based on cross-network address translation. It employs a distributed architecture consisting of a scheduling server, a forwarding server, and a storage server. The scheduling server is responsible for global device management, connection scheduling, and NAT traversal coordination. It constructs a multi-level mapping table structure based on virtual device identifiers and generates a query index through relational identifiers, enabling rapid retrieval and dynamic updating of terminal network information. During connection scheduling, the link cost from the terminal to the candidate relay node, the NAT type influence factor, and the node load are quantified into path costs. Based on these path costs, a target forwarding server is selected to construct the audio transmission channel. During audio data transmission, the audio data is adaptively adjusted based on transmission parameters fed back by the forwarding server and preset thresholds. Relay node switching is performed when necessary, enabling IoT terminals to achieve bidirectional communication across regions and operators without additional public IP configuration. This improves connection success rate while reducing end-to-end transmission latency and enhancing transmission stability.

[0005] In a first aspect, embodiments of this application provide an audio transmission control method based on cross-network address translation, applied to a scheduling server of an audio transmission control system. The audio transmission control system further includes a first terminal, a second terminal, and a forwarding server; the first terminal and the second terminal are respectively communicatively connected to the scheduling server and the forwarding server. The method includes: Obtain first device information and first network information of the first terminal; and obtain second device information and second network information of the second terminal; A mapping relationship library between terminal device identifiers and network information is determined based on the first device information, the first network information, the second device information, and the second network information; Upon receiving the audio transmission request information from the first terminal, the wide area network information of the second terminal is queried from the mapping relationship database to obtain the third network information; An audio transmission scheduling instruction is generated based on the first network information, the third network information, and the audio transmission request information; An audio transmission channel is established between the first terminal and the second terminal based on the mapping relationship library and the third network information, thus obtaining the audio transmission channel; The audio transmission scheduling instruction is sent to the forwarding server so that the audio data of the first terminal is forwarded to the second terminal through the audio transmission channel via the forwarding server.

[0006] Secondly, embodiments of this application provide an audio transmission control device based on cross-network address translation, applied to a scheduling server of an audio transmission control system. The audio transmission control system further includes a first terminal, a second terminal, and a forwarding server; the first terminal and the second terminal are respectively communicatively connected to the scheduling server and the forwarding server. The device includes: The acquisition unit is configured to acquire first device information and first network information of the first terminal; and to acquire second device information and second network information of the second terminal. The determining unit is configured to determine a mapping relationship library between terminal device identifiers and network information based on the first device information, the first network information, the second device information, and the second network information; The processing unit is configured to, upon receiving audio transmission request information from the first terminal, query the wide area network information of the second terminal from the mapping relationship database to obtain third network information; and generate an audio transmission scheduling instruction based on the first network information, the third network information, and the audio transmission request information. The control unit is used to establish a transmission channel between the first terminal and the second terminal based on the mapping relationship library and the third network information, thereby obtaining an audio transmission channel; and to send the audio transmission scheduling instruction to the forwarding server so that the audio data of the first terminal is forwarded to the second terminal through the audio transmission channel via the forwarding server.

[0007] Thirdly, embodiments of this application provide a server, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing steps in any method of the first aspect of this application.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in any method of the first aspect of this application.

[0009] Fifthly, embodiments of this application provide an audio transmission control system based on cross-network address translation, which is operable to cause a computer to perform some or all of the steps described in any method of the first aspect of this application.

[0010] By implementing the embodiments of this application, the following beneficial effects are achieved: Compared to the current method of selecting P2P or TURN relay based solely on NAT type, IoT devices are easily restricted by NAT gateways when transmitting audio across carriers and regions, resulting in a low connection success rate. Even if the connection success rate is improved by introducing relay channels, forwarding by a single relay node will significantly increase audio transmission latency and reduce transmission stability. This application provides an audio transmission control method and related apparatus based on cross-network address translation, applied to a scheduling server of an audio transmission control system. The audio transmission control system further includes a first terminal, a second terminal, and a forwarding server. The first terminal and the second terminal are communicatively connected to the scheduling server and the forwarding server, respectively. The method obtains first device information and first network information of the first terminal; and obtains second device information and second network information of the second terminal; determines a mapping relationship library between terminal device identifiers and network information based on the first device information, first network information, second device information, and second network information; upon receiving audio transmission request information from the first terminal, queries the wide area network information of the second terminal from the mapping relationship library to obtain third network information; generates an audio transmission scheduling instruction based on the first network information, third network information, and audio transmission request information; establishes an audio transmission channel between the first terminal and the second terminal based on the mapping relationship library and the third network information; and sends the audio transmission scheduling instruction to the forwarding server so that the audio data of the first terminal is forwarded to the second terminal through the audio transmission channel via the forwarding server. Thus, on the one hand, a distributed collaborative architecture is constructed through scheduling servers, forwarding servers, and storage servers. The scheduling server uniformly completes device management, connection scheduling, and cross-network address translation coordination, and builds audio transmission channels based on terminal device information and network address information, enabling IoT terminals to achieve bidirectional communication across regions and operators without additional configuration of public IP addresses. On the other hand, compared with solutions that rely solely on single-node forwarding, a joint scheduling strategy that integrates path cost and server load selects the optimal forwarding node, and a multi-channel isolation mechanism improves the stability of concurrent transmission. Furthermore, by constructing a mapping system that includes virtual device identifiers and transmission identifiers, fine-grained link management and isolation control are achieved during audio data transmission, thereby improving connection success rates while effectively reducing transmission latency and enhancing the overall stability of audio transmission. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1This is an architecture diagram of an audio transmission control system provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a server provided in an embodiment of this application; Figure 3 This is a flowchart illustrating an audio transmission control method based on cross-network address translation provided in an embodiment of this application; Figure 4 This is a flowchart illustrating a control method for audio transmission by a forwarding server provided in an embodiment of this application; Figure 5 This is a server architecture diagram of cross-NAT audio transmission control provided in an embodiment of this application; Figure 6 This is a flowchart illustrating another audio transmission control method based on cross-network address translation provided in an embodiment of this application; Figure 7 This is a functional module block diagram of an audio transmission control device based on cross-network address translation provided in an embodiment of this application. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0014] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0015] It should be understood that the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document indicates that the preceding and following related objects are in an "or" relationship. In the embodiments of this application, "multiple" refers to two or more.

[0016] In the embodiments of this application, "at least one item" or its similar expression refers to any combination of these items, including any combination of a single item or a plurality of items. "One or more" means one or more, while "multiple" means two or more. For example, "at least one item" of a, b, or c can represent the following seven cases: a, b, c; a and b; a and c; b and c; a, b, and c. Each of a, b, and c can be an element or a set containing one or more elements.

[0017] In this application, the term "connection" refers to various connection methods, such as direct connection or indirect connection, to achieve communication between devices. This application does not impose any limitations on this.

[0018] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0019] The following is an explanation of the relevant terms used in this application: Network Address Translation (NAT) is a network technology used to modify the source or destination address of IP packets in an IP network. Its main purpose is to enable communication between private networks and public networks and to solve the problem of IPv4 address shortage.

[0020] UDP Hole Punching: UDP Hole Punching is a NAT traversal technology based on the User Datagram Protocol (UDP). It enables terminals on different private networks to actively send UDP packets to a public network server, allowing their respective NAT gateways to establish address and port mapping relationships between the private and public networks, thereby opening up communication links between different NAT gateways and enabling point-to-point direct connections between terminals.

[0021] Traditional IoT devices often operate in private network environments, making them susceptible to NAT gateway obstruction during cross-carrier and cross-regional audio transmission, resulting in low connection success rates. While relay channels have been used to address this issue, relying on a single relay channel significantly increases audio transmission latency and reduces data transmission stability. To resolve these problems, this application provides an audio transmission control method and related apparatus based on cross-network address translation. This method employs a distributed architecture consisting of a scheduling server, a forwarding server, and a storage server. The scheduling server is responsible for global device management, connection scheduling, NAT traversal coordination, and the construction of audio transmission channels. The forwarding server allocates transmission tasks for audio transmission, allowing IoT terminals to access the public network without requiring additional public IP configuration. This supports bidirectional communication across regions and network operators, thereby improving the connection success rate of IoT devices while reducing audio transmission latency and enhancing transmission stability.

[0022] Please see Figure 1 , Figure 1 This is an architecture diagram of an audio transmission control system provided in an embodiment of this application. The audio transmission control system 100 includes an Internet of Things (IoT) device A111, an IoT device B211, a storage server A112, a storage server B212, a forwarding server A113, a forwarding server B213, and a scheduling server 310.

[0023] Among them, IoT devices A111 and B211 are both IoT terminal devices deployed in a private network environment. They can be interactive devices with audio acquisition and playback capabilities, such as smart speakers and smart panels. They are used to initiate or respond to bidirectional audio interaction requests across operators and regions, and to complete the acquisition, encoding, and playback of audio data. They are the terminal interaction nodes of the audio transmission control system 100. IoT devices A111 and B211 are located in independent private network environments and do not have public IP addresses. They need to access the public network through a NAT gateway. In one possible embodiment, after starting up, IoT devices A111 and B211 initiate a registration request to the scheduling server 310 through the public network, submitting device identifiers, product serial numbers, and access network information such as private network IP addresses and port numbers. After verifying the legality of the information, the scheduling server 310 assigns a globally unique virtual ID to each device and obtains and records the terminal's public network exit IP, NAT mapping port, and NAT type through UDP hole punching technology, completing device identity authentication and pre-collection of network information, laying the foundation for subsequent cross-NAT connection establishment.

[0024] Storage servers A112 and B212 are both local storage nodes deployed in conjunction with IoT terminals. They are used to store audio data collected or to be played by the corresponding IoT devices, and also provide data caching for audio transmission. Storage server A112 establishes a connection with IoT device A111 and is responsible for caching the audio data collected by IoT device A111 or the audio data to be sent to IoT device A111 for playback. Storage server B212 establishes a connection with IoT device B211 and undertakes the audio data storage function of the corresponding IoT device B211. Both are local storage carriers for audio data, avoiding the risk of loss when audio data is transmitted directly over the public network, and providing a data foundation for optimization mechanisms such as segmented transmission and breakpoint resume. In one possible embodiment, when IoT device A111 initiates an audio transmission request to IoT device B211, storage server A112 responds to the audio data fragmentation instruction issued by scheduling server 310, splitting the complete audio data to be transmitted into multiple fixed-length audio data fragments to improve the packet loss resistance of public network transmission; at the same time, the audio data received by IoT device B211 is first cached in storage server B212, and then pushed to IoT device B211 for playback after complete reception, ensuring the continuity and smoothness of audio playback.

[0025] Among them, forwarding server A113 and forwarding server B213 are public network forwarding node clusters deployed according to region and operator. They are used to relay audio data and assist in NAT traversal when IoT devices cannot achieve P2P direct connection. They are relay carriers for cross-NAT audio transmission. Forwarding server A113 is in a nearby network environment with IoT device A111 and storage server A112, and can act as a nearby relay node for IoT device A111. Forwarding server B213 is in a nearby network environment with IoT device B211 and storage server B212, and can act as a nearby relay node for IoT device B211. Through unified scheduling by scheduling server 310, both achieve efficient forwarding of audio data in cross-NAT environments. In one possible embodiment, when the NAT type combination of IoT device A111 and IoT device B211 does not meet the P2P direct connection conditions, the scheduling server 310 calculates based on link cost, node load, and NAT adaptability to match the optimal combination of forwarding servers for both parties, guiding the audio data to be transmitted through the relay path of "IoT device A111, storage server A112, forwarding server A113, forwarding server B213, storage server B212, IoT device B211". If the direct connection conditions are met, forwarding servers A113 and B213 are only used for UDP hole punching assistance and channel status monitoring functions, and do not participate in actual data forwarding, so as to achieve efficient use of resources.

[0026] The scheduling server 310 is the control node of the audio transmission control system 100. Deployed in the public network environment, it is used to realize global device management, network information storage, transmission strategy decision-making, and link scheduling control. The scheduling server 310 realizes unified management of IoT device A111, IoT device B211, and their corresponding storage and forwarding nodes by pre-building a mapping relationship library of "virtual ID-device information-network information". At the same time, it generates audio transmission scheduling instructions based on terminal network information and transmission requests to guide the establishment of audio transmission channels, data forwarding, and dynamic optimization. In one possible embodiment, the scheduling server 310 first receives the audio transmission request from IoT device A111, queries the WAN information of IoT device B211 from the mapping relationship database, and then decides on the P2P direct connection or relay forwarding strategy based on the dual-terminal NAT type and network address information, generating the corresponding audio transmission scheduling instruction. During the audio data transmission process, the scheduling server 310 monitors the transmission parameters of forwarding server A113 and forwarding server B213 in real time. When the node load or link quality exceeds the preset threshold, it issues an instruction to adjust the segmented audio data transmission parameters or switches to a backup forwarding node, realizing closed-loop control and dynamic optimization of the transmission process.

[0027] As can be seen, the audio transmission control system 100, through its architecture design of storage servers, forwarding servers, and scheduling servers, decouples terminal interaction, data storage, relay forwarding, and global scheduling functions. The modules work together efficiently, improving the stability of audio transmission across NAT wide area networks. Furthermore, by using IoT device registration and authentication and UDP hole punching mechanisms, it solves the addressing difficulties caused by private network devices lacking public IP addresses. Local caching and fragmentation processing on the storage server enhance the fault tolerance and continuity of audio data transmission. Proximity deployment and intelligent scheduling of the forwarding server cluster maximize the reduction of transmission latency while ensuring a high success rate for cross-NAT connections. Centralized management and dynamic optimization by the scheduling server achieve real-time performance and stability of audio transmission in complex network environments.

[0028] The following is combined Figure 2 The server in the embodiments of this application will be described. Figure 2 This is a schematic diagram of the structure of a server provided in an embodiment of this application, such as... Figure 2 As shown, the server 200 includes a processor 210, a memory 220, a communication interface 230, and one or more programs 221. The processor 210 is communicatively connected to the memory 220 and the communication interface 230 via an internal communication bus.

[0029] The one or more programs 221 are stored in the memory 220 and configured to be executed by the processor 210. The one or more programs 221 include instructions for performing any step in the above method embodiments.

[0030] The processor 210 can be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, units, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The communication unit can be a communication interface, transceiver, transceiver circuit, etc., and the storage unit can be a memory.

[0031] The memory 220 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0032] It is understood that server 200 may include more or fewer structural elements than those shown in the block diagram above, such as a power module, physical buttons, a Wi-Fi module, a speaker, a Bluetooth module, sensors, a display module, etc., without limitation. It is understood that server 200 may be equipped with... Figure 1 The architecture of the audio transmission control system.

[0033] After understanding the software and hardware architecture of this application, the following will be combined with... Figure 3 This application describes an audio transmission control method based on cross-network address translation in an embodiment of the present application. Figure 3 This is a flowchart illustrating an audio transmission control method based on cross-network address translation provided in an embodiment of this application. The method is applied to a scheduling server of an audio transmission control system, which further includes a first terminal, a second terminal, and a forwarding server. The first terminal and the second terminal are communicatively connected to the scheduling server and the forwarding server, respectively. The method specifically includes the following steps: Step S310: Obtain the first device information and the first network information of the first terminal; and obtain the second device information and the second network information of the second terminal.

[0034] Among them, the first device information and the second device information are identity identification information of IoT terminal access system, including: device identifier, product serial number and other unique identification information; the first network information and the second network information are access network attribute information of terminal, including: access operator, private network IP address, port number and other network parameters.

[0035] Specifically, when the first terminal initiates a two-way audio transmission request, the scheduling server first retrieves a locally pre-built mapping library of virtual IDs and actual network information. Based on the first terminal's virtual ID, it searches and extracts the corresponding first device information and first network information, completing the rapid acquisition of data from the initiating end. For the receiving end, the second terminal, the scheduling server first verifies its online status through the mapping library. After confirming that the terminal is in a network-available state, it matches and retrieves the second device information and second network information based on the second terminal's virtual ID, achieving synchronous data collection from both terminals. The acquired device information will be used to verify the legality of the terminal's communication, ensuring that only terminals within the authorized device group can perform two-way audio interaction, avoiding the risk of unauthorized device access. The acquired network information will serve as the basis for the scheduling server to determine the network deployment environment of the two terminals, identify NAT types, and plan communication links. Based on parameters such as the access operator and private network IP, the scheduling server can initially determine whether the two terminals are in the same network or a cross-network environment, providing data support for subsequently selecting the link mode of direct connection penetration or relay forwarding.

[0036] It should be noted that the acquired device and network information is bound to the terminal's unique virtual ID and stored accordingly. The scheduling server dynamically updates the network information. When a terminal experiences changes in its private IP address or port number due to network fluctuations, or switches ISPs, the scheduling server uses a heartbeat detection mechanism to capture these changes and updates the corresponding data in the mapping database in real time. This ensures that the acquired first and second network information always remain consistent with the terminal's actual network status. Furthermore, it supports concurrent information acquisition from multiple terminals. Faced with transmission requests from a large number of device groups, the scheduling server uses a load balancing strategy to allocate independent information retrieval threads to different device groups, enabling parallel acquisition and processing of information from multiple dual-terminal groups and meeting the system's high-concurrency application requirements.

[0037] Step S320: Determine the mapping relationship library between terminal device identifier and network information based on the first device information, the first network information, the second device information, and the second network information.

[0038] The mapping relationship database is the data carrier for the scheduling server to realize global device management and communication scheduling. It uses the terminal virtual ID as the index key to associate and bind device information such as device identifier and product serial number with network information such as public network exit IP, NAT mapping port, NAT type, and access operator, forming a structured terminal information mapping system.

[0039] Specifically, firstly, the scheduling server assigns a globally unique virtual ID to both the first and second terminals. This virtual ID serves as the terminal's unique communication identifier within the system and is bound to the terminal's device information. Next, UDP hole punching technology is used to supplement and verify the terminal's network information, obtaining and verifying the terminal's public IP address, NAT mapping port, and NAT type. Basic network information such as private IP address and port number is then integrated with public network information to form a mapping relationship database indexed by the virtual ID, constructing a "virtual ID-device information-network information" structure. During the construction of this mapping relationship database, the scheduling server categorizes and stores various types of information according to a standardized data structure, setting independent field modules for device information, basic network information, and public network information. Simultaneously, an information retrieval index is established to enable rapid information retrieval based on multiple dimensions such as virtual ID and device identifier. Furthermore, the scheduling server updates the mapping relationship database in real time. When a terminal experiences changes in network status or modifications to device information, a heartbeat packet detection and information retransmission mechanism captures these changes, promptly updating the corresponding data in the mapping relationship database to ensure a high degree of consistency between the database information and the actual terminal status.

[0040] It is evident that by constructing a standardized and structured mapping database between terminal device identifiers and network information, centralized and standardized management of IoT terminal information in cross-NAT WAN environments has been achieved, effectively solving problems such as information dispersion and retrieval difficulties caused by traditional private network deployments of IoT devices. This mapping database not only provides a unified data query basis for the global scheduling of the scheduling server, enabling millisecond-level fast retrieval of terminal information, but also ensures the real-time performance and stability of subsequent NAT traversal and connection establishment.

[0041] In one possible embodiment, determining the mapping relationship library between the terminal device identifier and the network information based on the first device information, the first network information, the second device information, and the second network information specifically includes the following steps: 321. Receive the first detection data and the second detection data from the first terminal and the second terminal in response to the public network mapping detection command; 322. Generate a first virtual device identifier based on the first device information; and generate a second virtual device identifier based on the second device information; 323. Determine the first wide area network address, first network address translation type, and first network translation port mapping information corresponding to the first terminal based on the first detection data; and determine the second wide area network address, second network address translation type, and second network translation port mapping information corresponding to the second terminal based on the second detection data; 324. Based on the first WAN address, the first network address translation type, and the first network translation port mapping information, perform an association mapping on the first virtual device identifier to obtain a first association mapping record; and based on the second WAN address, the second network address translation type, and the second network translation port mapping information, perform an association mapping on the second virtual device identifier to obtain a second association mapping record; 325. Determine the mapping relationship library based on the first association mapping record and the second association mapping record.

[0042] The public network mapping detection command is a network information detection command issued by the scheduling server to IoT terminals within the private network. It triggers the terminal's public network mapping detection behavior based on the UDP protocol. The first and second detection data are the network interaction data fed back by the terminal after responding to the command. The virtual device identifier is a globally unique communication identifier assigned to the terminal by the scheduling server. It is bound to the device's physical information and is non-repeatable, serving as the retrieval index for the mapping relationship database. The WAN address, network address translation type, and network translation port mapping information together constitute the terminal's WAN communication parameters, representing the terminal's public network address, NAT gateway type, and port mapping rules, respectively, and are the network data for enabling cross-NAT WAN communication.

[0043] Specifically, the scheduling server first sends a public network mapping detection command to the first and second terminals that have completed identity authentication. This command triggers the terminals to send probe data packets to the public network scheduling server using UDP hole punching technology. After receiving the terminal's response command, the scheduling server feeds back first and second detection data. This data includes interaction characteristic information between the terminal and the NAT gateway and the public network server, and is the raw data for parsing the terminal's WAN network parameters. Next, based on the device information of the first and second terminals, the scheduling server extracts physical features such as device identifiers and product serial numbers, and generates globally unique first and second virtual device identifiers using a hash algorithm. These identifiers are used to bind the terminal's physical identity to its network communication identity, avoiding the security risks of direct transmission of physical information. Then, the scheduling server parses the detection data, extracting and determining the public network exit IP (i.e., the first WAN address) of the first terminal, the NAT gateway type (i.e., the first network address translation type, including symmetric NAT or cone NAT), and the port mapping rules between the private and public networks (i.e., the first network translation port mapping information) from the first detection data. Simultaneously, it parses the third WAN network information of the second terminal from the second detection data, completing the conversion of the terminal's private network information into WAN communication information. In this way, the first virtual device identifier is multi-dimensionally associated and bound with the first terminal's WAN address, NAT type, and port mapping information, forming a structured first association mapping record. The second association mapping record for the second terminal is constructed in the same way. Both types of records use standardized data formats, including index fields, device information fields, and network information fields. Finally, the scheduling server incorporates the first and second association mapping records into the global data storage system, combining them with the association mapping records of other terminals within the system to construct a unified terminal device identifier and network information mapping relationship database. This mapping relationship database supports rapid retrieval and data updates based on virtual device identifiers, providing accurate data query services for subsequent communication scheduling.

[0044] As can be seen, the constructed mapping database achieves standardized and structured association and binding between terminal device identifiers and WAN network information, effectively solving the problems of scattered network information and difficult addressing in traditional private network IoT devices. Using virtual device identifiers as indexes ensures the uniqueness of terminal identities and improves the efficiency of network information retrieval, providing data support for the real-time establishment of subsequent NAT traversal and link establishment. By parsing the detection data, the WAN communication parameters of the terminal are obtained, laying the foundation for adapting to different NAT types and planning personalized transmission links, and significantly improving the connection success rate and system adaptability of cross-NAT WAN bidirectional audio transmission.

[0045] In one possible embodiment, determining the mapping relationship database based on the first association mapping record and the second association mapping record specifically includes the following steps: 3251. Establish a mapping table structure using the first virtual device identifier and the second virtual device identifier as indexes to obtain the first mapping table; 3252. Update the first mapping table based on the first associated mapping record and the second associated mapping record to obtain the second mapping table; 3253. Determine the association identifier between the first virtual device identifier and the second virtual device identifier in the second mapping table to obtain multiple association relationship identifiers; 3254. Generate a query index based on the multiple association relationship identifiers to query the first virtual device identifier and the second virtual device identifier, thereby obtaining multiple query indexes; 3255. Map the multiple query indexes to the second mapping table to obtain the mapping relationship library.

[0046] The mapping table structure is a standardized data framework for the scheduling server to store terminal association mapping records. Using virtual device identifiers as indexes, it enables the location and rapid retrieval of terminal information and serves as the foundation for building the mapping relationship database. The association relationship identifier is a feature marker representing the communication association attributes between the first and second virtual device identifiers, used to define the device group affiliation, communication permissions, and other associations between terminals. The query index is a retrieval identifier generated based on the association relationship identifier, enabling multi-dimensional and rapid retrieval of terminal information in the mapping table and improving the query efficiency of the mapping relationship database.

[0047] Specifically, a standardized first mapping table structure is first established using the first virtual device identifier and the second virtual device identifier as primary keys. This table structure includes fixed data modules such as preset device identifier fields, WAN address fields, NAT type fields, and port mapping information fields, reserving standardized storage locations for subsequent terminal information input and ensuring that the association mapping records of different terminals can be stored according to unified rules. Next, the various types of information in the first and second association mapping records are respectively entered into the corresponding fields of the first mapping table to complete the information update and data filling of the first mapping table, resulting in a second mapping table that includes complete device and network information for both terminals. This second mapping table serves as the data carrier for the mapping relationship database. Then, according to preset device group authorization rules, the scheduling server assigns association relationship identifiers representing the communication association attributes of the first and second virtual device identifiers in the second mapping table. These identifiers can reflect key information such as whether the terminals belong to the same authorized device group and whether they have two-way audio communication permissions. Multiple association relationship identifiers can achieve a complete representation of multi-dimensional association attributes between terminals. Based on the generated multiple relationship identifiers, the scheduling server constructs a lightweight query index. This index corresponds to the virtual device identifier and the relationship identifier, supporting fast retrieval based on multiple dimensions such as virtual identifier, device group affiliation, and communication permissions, avoiding query latency caused by full table traversal. Finally, the generated multiple query indexes are associated with the second mapping table, making the query index a quick entry point for retrieving terminal information from the second mapping table. This forms a terminal device identifier and network information mapping relationship library that integrates standardized data storage, multi-dimensional relationship labeling, and efficient index retrieval. This library can achieve millisecond-level retrieval and dynamic updating of terminal information.

[0048] It is evident that the constructed mapping relationship library enables structured storage and associated management of terminal data. The table structure indexed by virtual device identifiers ensures the uniqueness and accuracy of terminal information. The construction of multi-dimensional query indexes improves the retrieval efficiency of information within the library, providing key guarantees for the real-time nature of subsequent NAT traversal and link establishment, and significantly improving the overall operating efficiency and adaptability of the cross-NAT wide area network bidirectional audio transmission system.

[0049] Step S330: Upon receiving the audio transmission request information from the first terminal, query the wide area network information of the second terminal from the mapping relationship database to obtain the third network information.

[0050] Among them, the audio transmission request information is the instruction data sent by the first terminal to the scheduling server when it initiates two-way audio interaction, including the virtual ID of the first terminal, the identifier of the target second terminal, and the transmission request instruction; the third network information is the network parameters of the second terminal for adapting to wide area network communication, including the public network exit IP of the second terminal, NAT mapping port, NAT type and the forwarding server node to which it belongs, which is the basis for the scheduling server to plan the communication link and perform NAT traversal.

[0051] Specifically, the scheduling server first parses the transmission request information, extracting the first terminal virtual ID and the target second terminal identifier to identify the communication initiator and receiver. Next, based on the parsed second terminal identifier, the scheduling server initiates a targeted retrieval process in a pre-built mapping database of terminal device identifiers and network information. The retrieval process uses the virtual ID as the index key to quickly match the network information corresponding to the second terminal and filters out third network information suitable for WAN cross-NAT communication. This includes the public network exit IP address capable of public network addressing, the mapping port assigned by the NAT gateway, the NAT type determining the penetration strategy, and information on the nearest deployed forwarding server node. During the retrieval process, the scheduling server verifies the validity of the retrieved third network information by checking the terminal's online status through heartbeat packets, verifying the current availability of the public network exit IP address and mapping port. If invalid information is found, the server triggers a re-collection of the second terminal's network information and an update of the mapping database to ensure that the third network information is consistent with the actual WAN status of the second terminal. Simultaneously, the scheduling server correlates the retrieved third-party network information with the WAN information of the first terminal, providing complete data support for subsequent determination of the dual-terminal network topology and selection of NAT traversal or relay forwarding strategies. Furthermore, for multi-terminal transmission requests in high-concurrency scenarios, the scheduling server allocates independent processing threads to each retrieval task through a load balancing mechanism, enabling parallel retrieval of multiple requests, avoiding retrieval delays caused by request congestion, and ensuring the system's retrieval efficiency under high concurrency conditions.

[0052] It is evident that the retrieval method using virtual IDs as indexes improves information retrieval efficiency; the validity verification and dynamic updating of third-party network information ensures the accuracy and availability of retrieved information, avoids connection failures due to invalid network information, and improves the success rate of link establishment. Meanwhile, the parallel retrieval design adapted for high concurrency allows the system to maintain stable retrieval efficiency even when a large number of terminals simultaneously initiate transmission requests, meeting the application requirements in high-concurrency scenarios.

[0053] Step S340: Generate an audio transmission scheduling instruction based on the first network information, the third network information, and the audio transmission request information.

[0054] Among them, the audio transmission scheduling command is a standardized communication control command issued by the scheduling server to the terminal and the forwarding server cluster, including control parameters such as link establishment method, NAT traversal strategy, forwarding server node selection, and data transmission routing.

[0055] Specifically, the three types of data are first fused and analyzed. The public network exit IP, NAT mapping port, NAT type, access operator and regional node information of the two terminals are obtained from the first network information and the third network information. The network topology relationship of the two terminals is determined, and it is determined whether the two are in the same operator and the same regional network environment, and whether the NAT type is the same type of cone NAT, hybrid NAT or symmetric NAT. The basic request parameters such as transmission priority and audio transmission bitrate requirements are parsed from the audio transmission request information, and the transmission service requirements of the terminals are clarified. Based on the determined parsing results, the scheduling server initiates the transmission strategy decision logic. If both terminals have the same type of cone NAT and are in similar network environments, it generates a P2P direct connection scheduling instruction. This instruction includes public network mapping information for both parties and UDP hole punching trigger parameters, guiding both terminals to directly initiate point-to-point traversal to establish a direct connection channel. If it is a mixed NAT type, it generates an auxiliary traversal scheduling instruction, specifying the nearest forwarding server node for both terminals, which assists in completing NAT traversal. If it is determined to be a high-probability scenario of P2P traversal failure, such as symmetric NAT, it immediately generates a relay transmission scheduling instruction. This instruction plans a relay transmission route of "first terminal - nearest forwarding node - second terminal's nearest forwarding node - second terminal," and configures relevant parameters for data forwarding between nodes. During instruction generation, the scheduling server combines real-time WAN bandwidth monitoring data to match auxiliary control parameters such as dynamic bitrate adjustment and packet loss retransmission for the instruction. If network bandwidth fluctuations are detected, a preset bitrate switching threshold will be included in the instruction to ensure that audio transmission adapts to network changes. In addition, all generated audio transmission scheduling instructions are encapsulated in a standardized data format, including fields such as instruction identifier, terminal virtual ID, execution parameters, and validity period, to ensure that the terminal and the forwarding server can quickly parse and execute them.

[0056] In one possible embodiment, generating an audio transmission scheduling instruction based on the first network information, the third network information, and the audio transmission request information specifically includes the following steps: 341. Determine the first network address information and the first network address translation type in the first network information; 342. Determine the second network address information, the second terminal status information, and the second network address translation type in the third network information; 343. When the second terminal status information is online, a matching analysis is performed based on the first network address translation type and the second network address translation type to obtain the combination result of the network address translation types of the first terminal and the second terminal; 344. If the network address translation type combination result satisfies the preset direct connection conditions between the first terminal and the second terminal, determine the peer access mapping information according to the first network address information and the second network address information; generate a first audio transmission scheduling instruction according to the peer access mapping information and the audio transmission request, and send the first audio transmission scheduling instruction to the first terminal and the second terminal, so that the audio in the storage server connected to the first terminal is sent to the second terminal through the forwarding server; 345. If the network address translation type combination result does not meet the direct connection condition, determine the relay node information of the forwarding server based on the first network address information and the second network address information; determine the scheduling information of the relay channel between the first terminal and the second terminal based on the relay node information, the first network address translation type, and the second network address translation type; generate a second audio transmission scheduling instruction based on the scheduling information and the audio transmission request information, so that the audio in the storage server connected to the first terminal is sent to the second terminal through the relay channel.

[0057] The first and second network address information are addressing data for the terminal's WAN communication, including parameters such as the public network exit IP and NAT mapping port. The network address translation type characterizes the type of NAT gateway to which the terminal belongs, including symmetric NAT and cone NAT. The second terminal status information reflects the terminal's online connectivity status. The direct connection condition is a system-preset NAT type matching rule, which is the standard for determining whether a terminal can achieve a P2P direct connection. It is used to determine the NAT type of whether the first and second terminals can establish a P2P (Peer-to-Peer) direct communication link through UDP hole punching technology. The matching rule peer access mapping information consists of WAN mapping parameters for mutual access between terminals, which is key data guiding the establishment of a P2P direct connection. The relay node information consists of the parameters of the nearest forwarding server node matched by the scheduling server for the terminal, and the relay channel scheduling information is the routing plan and node configuration data for relay transmission.

[0058] Specifically, the scheduling server first extracts the first network address information and the first network address translation type, which characterize the WAN addressing and network attributes of the first terminal, from the first network information, thus completing the extraction of the initiating network parameters. Simultaneously, it parses the second network address information, the real-time online status information of the second terminal, and the second network address translation type from the third network information. After confirming that the second terminal is online and communicable, it performs a combination matching analysis on the first and second network address translation types to obtain the NAT type combination result for both terminals. If the type combination result meets the system's preset direct connection condition, i.e., both terminals are of the same type of cone NAT, it parses the peer access mapping information based on the network address information of both parties, including key data such as the peer's public network exit IP and NAT mapping port. Combined with the service parameters of the audio transmission request information, it generates a first audio transmission scheduling instruction to guide the two terminals to establish a P2P direct connection and sends the instruction to both terminals. The instruction guides the audio data of the first terminal to be lightly forwarded by the forwarding server to achieve direct transmission between the two terminals. If the type combination result does not meet the direct connection condition, such as when there is symmetric NAT or hybrid NAT, the scheduling server will match the nearest forwarding server nodes for both parties based on the network address information of the two terminals, combined with the regional and operator attributes, to obtain the relay node information. Then, based on the relay node information and the NAT type of the two terminals, it will plan the transmission route from the first terminal to the corresponding relay node and then to the corresponding relay node of the second terminal, determine the port configuration, data forwarding rules and other scheduling information of the relay channel, and generate a second audio transmission scheduling instruction in combination with the audio transmission request information. The instruction will guide the system to build a dedicated relay transmission channel to achieve stable transmission of audio data from the first terminal to the second terminal via the relay channel.

[0059] As can be seen, by analyzing the NAT type and network status of the two terminals, the transmission strategy was adapted. When the direct connection condition was met, the P2P direct connection strategy was adopted to significantly reduce the audio transmission latency. When the condition was not met, the relay strategy was used to ensure the connection success rate, effectively balancing the real-time performance and connectivity of audio transmission. Based on the terminal network address information, the nearest relay node was matched to maximize the shortening of the physical path of relay transmission and further reduce the transmission latency in relay mode. At the same time, standardized scheduling instructions were generated by combining audio transmission request information to ensure a high degree of matching between instructions and terminal transmission needs, improving the accuracy and efficiency of instruction execution, and providing scheduling guarantee for stable and efficient bidirectional audio transmission across NAT wide area networks.

[0060] In one possible embodiment, determining the scheduling information of the relay channel between the first terminal and the second terminal based on the relay node information, the first network address translation type, and the second network address translation type specifically includes the following steps: 3451. Determine the first link cost between the first terminal and the relay node for the relay node information; and the second link cost between the second terminal and the relay node; 3452. Determine the relay node address and node load information corresponding to the relay node in the relay node information; 3453. Determine the path cost impact factor of network address translation based on the first network address translation type and the relay node address to obtain a first impact factor; and determine the path cost impact factor of network address translation based on the second network address translation type and the relay node address to obtain a second impact factor; 3454. Determine the relay node link cost based on the first link cost, the second link cost, the first influence factor, and the second influence factor; 3455. Determine the scheduling information of the relay channel based on the relay node link cost and the node load information.

[0061] Link cost is a quantitative indicator characterizing the quality of data transmission links between terminals and relay nodes. It comprehensively reflects transmission characteristics such as transmission distance, network latency, and bandwidth loss, and serves as the basis for selecting the optimal relay link. Node load information consists of real-time operating parameters of each relay node in the forwarding server cluster, including bandwidth utilization, data processing volume, and number of connections, used to measure the node's service capacity and operating pressure. The path cost impact factor is a correction coefficient derived by combining the terminal NAT type and the relay node address matching degree, used to quantify the adaptation loss of different NAT types on the relay transmission path. The relay node path cost is the comprehensive link cost after NAT type adaptation correction.

[0062] Specifically, the scheduling server first extracts the physical addressing parameters (i.e., relay node addresses) of multiple candidate relay nodes from the relay node information. Simultaneously, it collects real-time operational data from each node to obtain node load information, clarifying the basic service capabilities and availability of each relay node. Based on the acquired node load information and the terminal's WAN address information, it calculates the first link cost from the first terminal to each candidate relay node and the second link cost from the second terminal to each candidate relay node through network latency detection and bandwidth loss calculation, thus completing a quantitative assessment of the basic transmission link quality between the terminal and the nodes. Then, it performs an adaptation analysis by combining the first network conversion type of the first terminal, the second network conversion type of the second terminal, and the relay node addresses. For different types of port mapping rules, such as symmetric NAT and cone NAT, it quantitatively calculates the NAT traversal adaptation loss between the terminal and the relay nodes, obtaining the first and second influence factors. The lower the NAT type and node address matching degree, the larger the factor value, and the higher the corresponding path cost correction magnitude. Based on this, the first link cost, the second link cost, and the corresponding influencing factors are weighted and fused to obtain the comprehensive relay node path cost for each candidate relay node, achieving precise correction of link transmission quality. Finally, a secondary screening is performed based on the relay node path cost and multiple node load information, prioritizing the selection of relay node combinations with the lowest path cost and node load within a preset threshold. Simultaneously, data transmission routes, port configurations, and data forwarding rules between terminals and nodes, and between nodes, are planned, ultimately determining relay channel scheduling information that combines high transmission efficiency, node stability, and NAT adaptability.

[0063] It is evident that by constructing a multi-dimensional quantitative evaluation system, precise planning of relay channel scheduling information has been achieved. Link cost, NAT type adaptability, and node load status are incorporated into a unified decision-making framework, effectively avoiding problems such as high transmission latency and node congestion caused by single-indicator decision-making, and significantly reducing audio transmission latency in relay mode, thereby improving the overall stability and reliability of bidirectional audio transmission across NAT wide area networks.

[0064] Step S350: Based on the mapping relationship library and the third network information, establish an audio transmission channel between the first terminal and the second terminal to obtain the audio transmission channel.

[0065] The audio transmission channel is a dedicated communication link for the first and second IoT terminals to achieve bidirectional audio data interaction across NAT wide area networks. It can be adapted to either a P2P direct connection channel or a relay forwarding channel depending on the network environment, and serves as the physical carrier for audio data transmission.

[0066] Specifically, the scheduling server first uses the mapping database to retrieve the WAN information of the first terminal and the network information of the third terminal to reconstruct the network topology, clarifying the NAT type, cross-network attributes, and regional forwarding server node information of both terminals. If the scheduling instruction is a P2P direct connection strategy, the scheduling server pushes the public network exit IP and NAT mapping port information of both terminals to the corresponding terminals, guiding the first and second terminals to simultaneously initiate UDP hole punching operations. Both parties send UDP probe packets to the corresponding public network address based on the obtained WAN information of the other party, enabling their respective NAT gateways to confirm the port mapping relationship and thus establish a point-to-point direct connection channel. After completion, the scheduling server sends millisecond-level test packets to verify the connectivity and link latency of the channel. If the scheduling instruction is an auxiliary penetration strategy, the scheduling server sends the information of the specified nearest forwarding server node to both terminals. This node acts as an intermediary to forward UDP hole punching probe packets, assisting terminals with different NAT types to complete port mapping and link establishment, thereby indirectly establishing a P2P channel. If the scheduling instruction is a relay forwarding strategy, the scheduling server will issue routing configuration instructions to forwarding node A, the address of the first terminal, and forwarding node B, the address of the second terminal, respectively. Based on the third network information, it will complete the addressing and routing binding of node B, constructing a relay transmission channel from the first terminal to node A to node B and then to the second terminal. Simultaneously, it will configure parameters such as the port and protocol for data forwarding between nodes. Regardless of the channel type, after establishment, the scheduling server will monitor the channel status in real time through the mapping relationship database, verify the basic connectivity of data transmission, mark the channel as "available," and synchronize it to the first and second terminals, completing the final establishment of the audio transmission channel. The entire channel establishment process is based on standardized information in the mapping relationship database, requiring no additional public IP configuration for the terminal, and adapting to cross-carrier and cross-regional network environments.

[0067] As can be seen, dynamically adapting P2P direct connection or relay forwarding channels based on the network environment ensures connection success rate while minimizing transmission latency. Using a mapping database as information support, it enables rapid retrieval and matching of terminal information during channel establishment, significantly improving link construction efficiency and shortening connection establishment time. Furthermore, the entire channel establishment process is centrally managed by the scheduling server, ensuring the effectiveness and stability of the transmission channel, and eliminating the need for additional public IP configuration on the terminal, enabling it to adapt to complex WAN environments with different operators and NAT types.

[0068] In one possible embodiment, establishing an audio transmission channel between the first terminal and the second terminal based on the mapping database and the third network information, thereby obtaining the audio transmission channel, specifically includes the following steps: 351. Determine the network address translation type information in the third network information; 352. Filter out the network address and network address translation type corresponding to the second terminal from the mapping relationship library to obtain the target network address information and the target network address translation type; 353. Based on the audio transmission scheduling instruction, the target network address information is sent to the first terminal and the second terminal respectively, so as to control the first terminal to send a connection request to the second terminal based on the audio transmission scheduling instruction; 354. After the second terminal responds to the connection request, a first transmission channel is generated based on the network address translation type information and the target network address translation type; 355. Send an audio test data packet to the second terminal through the first transmission channel for detection, and obtain the detection result; 356. When the detection result meets the preset audio transmission requirements, the audio transmission channel is determined based on the first transmission channel, the first terminal, and the second terminal.

[0069] Among them, the target network address information and the target network address translation type are the second terminal WAN addressing parameters and network translation attributes that are accurately retrieved from the mapping relationship library; the audio test data packet is a millisecond-level dedicated test data used to verify the connectivity of the transmission channel, link delay and data transmission stability. The preset audio transmission requirements are the channel availability thresholds defined by the scheduling server, including indicators such as delay threshold and packet loss rate threshold.

[0070] Specifically, the scheduling server first extracts the network address translation type (NAT) information of the second terminal from the third network information to clarify the type characteristics of its NAT gateway. Next, using the second terminal's virtual device identifier as an index, it filters the corresponding WAN addressing parameters (target network address information) and the target NAT translation type consistent with the actual network state from the mapping database, completing a secondary verification and retrieval of the second terminal's network information to ensure accuracy and timeliness. Based on the previously generated audio transmission scheduling instructions, the scheduling server sends the target network address information to both the first and second terminals. Simultaneously, it triggers the first terminal to initiate a cross-NAT WAN connection request to the second terminal according to the preset communication rules, achieving connection interaction between the two ends. After the second terminal receives and responds to the connection request, the scheduling server combines the extracted NAT translation type information with the retrieved target NAT translation type, matches the corresponding channel building rules based on the NAT type combination characteristics, and generates a first transmission channel. This channel can be adapted to a P2P direct connection channel or a relay forwarding channel depending on the NAT type, and the channel parameters are highly matched with the network attributes of both terminals. After the channel is generated, the scheduling server immediately sends millisecond-level audio test data packets to the second terminal through the first transmission channel. This comprehensively tests key indicators such as link latency, data transmission packet loss rate, and connectivity stability, recording the test results and comparing them with the system's preset audio transmission requirements. If the test results meet the preset thresholds—meaning the channel's latency and packet loss rate both meet the real-time requirements of bidirectional audio transmission—the scheduling server associates and binds the first transmission channel with both the first and second terminals, ultimately determining it as an audio transmission channel suitable for actual audio data transmission, thus completing the construction and validity verification of the entire link.

[0071] As can be seen, by initiating a connection request first and then generating a transmission channel, the communication interaction between the two terminals and the targeted construction of the channel are realized, making the channel parameters highly compatible with the NAT types of the two terminals and improving the success rate of channel establishment. The introduction of an audio test data packet detection step comprehensively verifies the channel transmission performance, ensuring that the established transmission channel meets the real-time and stability requirements of bidirectional audio transmission, and effectively avoiding problems such as audio stuttering and delay caused by the use of invalid channels.

[0072] Step S360: Send the audio transmission scheduling instruction to the forwarding server so that the audio data of the first terminal is forwarded to the second terminal through the audio transmission channel via the forwarding server.

[0073] Among them, the forwarding server is a server cluster deployed on the public network and divided into nodes according to region. It is a relay carrier for cross-NAT wide area network audio transmission and can realize functions such as penetration assistance, data forwarding, and load balancing according to transmission needs.

[0074] Specifically, the scheduling server first matches the nearest forwarding server node between the first and second terminals from the forwarding server cluster based on the geographical information and network affiliation of the first and second terminals in the mapping relationship database. At the same time, it parses the core execution parameters in the audio transmission scheduling instructions, including transmission channel type, data forwarding route, audio bitrate standard, packet loss retransmission rules, etc., and associates and encapsulates these parameters with the terminal virtual ID and transmission channel identifier to form standardized instruction data adapted to the execution of the forwarding server. If the audio transmission channel is a P2P direct connection channel, the scheduling server issues scheduling instructions to the corresponding forwarding server node with penetration assistance and status monitoring as the core. The instructions only require the server to collect channel transmission status data in real time, without participating in actual data forwarding, and only stand by as a backup relay node when the channel fluctuates. If it is a hybrid NAT-assisted penetration channel, the scheduling server issues scheduling instructions to the designated assisting node, requiring the node to forward UDP hole punching probe packets according to preset rules, and switch to status monitoring mode after completing the penetration. If it is a relay forwarding channel, the scheduling server issues precise routing scheduling instructions to forwarding server node A to which the first terminal belongs and forwarding server node B to which the second terminal belongs, respectively. The instructions configure forwarding rules for node A as "receive audio data from the first terminal - forward to node B", and configure processing rules for node B as "receive data forwarded by node A - reassemble and decompress - push to the second terminal". At the same time, the instructions specify the data transmission protocol standard, fragmentation and reassembly requirements and dynamic bitrate adjustment threshold. After receiving the instruction, the node completes the local configuration and starts the audio data processing service. At this time, the audio data of the first terminal is encoded with AAC, fragmented and packaged, and sent to the forwarding server through the established transmission channel. The server strictly follows the scheduling instructions to complete the data verification, forwarding or processing, and finally pushes the compliant audio data to the second terminal. The reverse audio data is returned according to the same rules. Throughout the process, the scheduling server monitors the running status of the forwarding server and the data transmission in real time. If the node bandwidth is too high or a failure occurs, it immediately issues an instruction to switch to the backup node.

[0075] It is evident that by precisely issuing audio transmission scheduling instructions to the forwarding server, efficient linkage between the forwarding server cluster and the audio transmission channel is achieved, effectively solving problems such as high transmission latency and poor stability caused by disordered data forwarding and inaccurate node scheduling in traditional cross-NAT transmission.

[0076] With the above Figure 3 For embodiments that are consistent with those shown, please refer to [link / reference]. Figure 4 , Figure 4 This is a flowchart illustrating a control method for audio transmission by a forwarding server according to an embodiment of this application. As can be seen, the control method for audio transmission by the forwarding server specifically includes the following steps: S410. Send an audio data fragmentation instruction to the forwarding server to control the audio data stored in the storage server connected to the first terminal to be fragmented to obtain multiple fragmented audio data. S420. According to the audio scheduling instruction, the multiple audio data segments are sent to the forwarding server through the audio transmission channel to obtain audio transmission information; S430. Determine the transmission parameters of the forwarding server in the audio transmission information; S440. When the transmission parameters are greater than or equal to the preset transmission parameter threshold of the forwarding server, control the audio transmission parameters of the multiple audio data segments, and / or switch the forwarding server to transmit the multiple audio data segments.

[0077] The audio data fragmentation instruction is a fragmentation processing control instruction issued by the scheduling server for long audio and large-capacity audio data transmission. It is used to split continuous audio data into several small data units of fixed or adaptive length to reduce the probability of single-transmission blocking and improve network jitter tolerance. Audio transmission information includes the real-time operating status and link quality data of the forwarding server during data forwarding. Transmission parameters are used to quantify key performance indicators such as the load level, bandwidth utilization, data forwarding rate, and latency fluctuation of the forwarding server, serving as the core basis for determining whether the current forwarding node is overloaded or in an abnormal state. The preset transmission parameter threshold is a pre-set boundary for safe node operation. When the transmission parameters reach or exceed this threshold, it indicates that the current forwarding node can no longer guarantee low latency and high stability in audio transmission. Preferably, the preset transmission parameter threshold may include: bandwidth utilization ≤ 70%, average forwarding rate per connection ≥ 128kbps, end-to-end transmission latency ≤ 150ms, packet loss rate ≤ 3%, and concurrent connection count ≤ 50. These parameters are set according to actual applications and are not specifically limited here.

[0078] Specifically, the scheduling server first issues an audio data fragmentation instruction to the designated forwarding server. The forwarding server, in conjunction with the storage server bound to the first terminal, performs fragmentation processing on the complete audio data to be transmitted, splitting it into multiple audio data fragments according to the unit size adapted to network transmission. This enhances the data's resilience against packet loss and interference during public-to-private network penetration. Next, based on the generated audio scheduling instruction, the scheduling server sequentially sends the fragmented audio data to the forwarding server through the established audio transmission channel. The forwarding nodes complete data reception, verification, and forwarding, generating audio transmission information including real-time bandwidth usage, forwarding latency, packet loss rate, and concurrent connection count during the forwarding process. The scheduling server continuously parses and extracts the forwarding server's transmission parameters from the audio transmission information, performing real-time perception and quantitative evaluation of the node's operational status. When the transmission parameters are detected to be greater than or equal to the preset threshold, it is determined that the current forwarding node is at risk of congestion, overload, or link degradation. The scheduling server immediately initiates a dynamic control strategy: on the one hand, it can reduce the sending rate of segmented audio data, adjust the segment length, or control the number of concurrent transmissions to alleviate node pressure from the traffic level; on the other hand, it can directly trigger the forwarding server switching mechanism. Based on the mapping relationship library and relay node scheduling information, it reselects a backup forwarding node with a lower load and a better path, and smoothly migrates the segmented audio data that has not been transmitted to the new node to continue forwarding. The entire switching process maintains the continuity and timing integrity of audio data and does not produce obvious stuttering or interruption.

[0079] It is evident that audio data fragmentation effectively improves the fault tolerance and penetration success rate of public network transmission. Real-time monitoring of transmission parameters of forwarding servers enables awareness of the transmission link status. Through a combination of dynamic rate limiting and node switching control mechanisms, it can quickly respond and proactively optimize when nodes are congested or links are degraded, avoiding transmission delays, data loss, or communication interruptions caused by overloaded forwarding nodes, thus ensuring reliable and efficient transmission of audio data.

[0080] For easier understanding, please refer to Figure 5 , Figure 5 This is a server architecture diagram for cross-NAT audio transmission control provided in an embodiment of this application. As can be seen, the overall architecture of the cross-NAT audio transmission control server adopts a layered architecture of distributed deployment and centralized scheduling, including private network A, private network B, a forwarding server cluster in the public network, and a scheduling server.

[0081] Private network A and private network B both include IoT devices and storage servers. The IoT devices are IoT terminals with audio acquisition, encoding, and playback capabilities, used to initiate audio transmission requests, collect local audio data, and receive and play audio data from peers. The storage server establishes a binding connection with the IoT devices to cache and store audio data to be transmitted, and responds to the scheduling server's segmentation instructions, splitting the complete audio data into multiple segments to provide data support for audio transmission. Private network B, including both IoT devices and storage servers, has a structure symmetrical to private network A. The IoT devices in private network B receive audio transmission instructions and receive and play audio data from IoT devices in private network A. The storage server in private network B is bound to the IoT devices in private network B to cache and reassemble segmented audio data forwarded by the forwarding server, ensuring the continuity and integrity of audio playback. Private networks A and B are independent of each other, located in private network environments under different NAT gateways, and do not have independent public IP addresses. The public network forwarding server cluster includes at least two forwarding servers deployed in close proximity. These servers relay audio data, assist with UDP hole punching, and monitor link status when IoT devices in private network A and private network B cannot establish a direct P2P connection. Forwarding servers in the cluster are matched with the nearest server in private network A, and with the nearest server in private network B. Nodes can communicate with each other to achieve data forwarding and routing coordination. The scheduling server, deployed on the public network, serves as the core control hub of the entire system. Internally, it maintains a mapping database, scheduling decisions, and scheduling instructions. It performs terminal registration and authentication, network information collection, NAT type matching analysis, transmission strategy decisions, scheduling instruction generation, transmission status monitoring, and dynamic optimization. The scheduling server establishes communication connections with both IoT devices and storage servers to achieve unified global scheduling and management.

[0082] As can be seen, this architecture, through the collaborative cooperation of private network terminals, storage servers, forwarding server clusters, and scheduling servers, enables bidirectional audio data transmission across NAT, carriers, and regions without changing the existing private network deployment structure or allocating public IP addresses. It effectively solves the problems of communication difficulties, high transmission latency, and poor stability of traditional private network IoT devices, and has high adaptability, high reliability, and high real-time performance.

[0083] For easier understanding, please refer to Figure 6 , Figure 6This is a flowchart illustrating another audio transmission control method based on cross-network address translation provided in this application embodiment. First, the process of "the scheduling server issuing an audio data fragmentation instruction, and the storage server splitting the audio into fragmented data" is executed. The scheduling server sends an audio data fragmentation instruction to a designated forwarding server. This instruction, in conjunction with the storage server connected to the first terminal, performs fragmentation processing on the complete audio data it stores, splitting the continuous audio data stream into several small data units of fixed or adaptive length, resulting in multiple fragmented audio data. Fragmentation processing reduces the probability of blocking in a single transmission, improving the tolerance to network jitter and link fluctuations during public network transmission. Then, the process of "sending the fragmented audio data to the forwarding server through the audio transmission channel according to the scheduling instruction" is executed. The scheduling server, based on the previously generated audio scheduling instruction, sends the multiple fragmented audio data, after fragmentation processing, to the corresponding forwarding server in an orderly manner through a verified and usable audio transmission channel. Next, the process of "the forwarding server generating audio transmission information including real-time load, network latency, and packet loss rate" is executed. This means that during the process of receiving, verifying, and forwarding segmented audio data, the forwarding server collects and generates audio transmission information in real time, including key indicators such as node real-time load, network latency, and packet loss rate. This information comprehensively reflects the operating status and link quality of the forwarding server during the audio data forwarding phase. Then, the process of "the scheduling server extracting transmission data and comparing it with preset thresholds" is executed. The scheduling server extracts transmission parameters that quantify the performance of the forwarding server from the audio transmission information fed back by the forwarding server, including bandwidth utilization, data forwarding rate, end-to-end latency, and concurrent connections. The extracted transmission parameters are compared one by one with the system's preset transmission parameter thresholds to determine whether the current forwarding server is overloaded or the link is degraded.In the "whether the transmission parameters are greater than the threshold" judgment step, if the result is "no," meaning the transmission parameters are less than the preset transmission parameter threshold, it indicates that the current forwarding server's load level, bandwidth resources, and transmission performance all meet the requirements for low latency and high stability in audio transmission. The scheduling server then maintains the current transmission state and continues the orderly transmission of segmented audio data, i.e., executes the "maintain current transmission state and transmit audio data" process. If the result is "yes," meaning the transmission parameters are greater than or equal to the preset transmission parameter threshold, it indicates that the current forwarding server has approached or exceeded the safe operating boundary and cannot guarantee the real-time and smoothness of audio transmission. In this case, the "adjust segmented transmission parameters" process is executed. The process involves two steps: "switching forwarding servers" and "completing stable transmission of segmented audio data." At this point, the scheduling server initiates a dynamic control strategy. On one hand, it can adjust the transmission parameters of the segmented audio data, such as reducing the data transmission rate, decreasing the segment size, or limiting the number of concurrent transmissions, thus alleviating the load pressure on the forwarding server from a traffic perspective. On the other hand, it can directly trigger the forwarding server switching mechanism. Based on the mapping relationship library and relay node scheduling information, it reselects a backup forwarding server with a lower load and a better path, smoothly migrating the incomplete transmission of segmented audio data to the new node for continued forwarding. The entire control process maintains the temporal continuity and integrity of the audio data. After the above control, the stable transmission of segmented audio data to the target terminal is finally completed, achieving efficient and reliable delivery of audio data in a cross-NAT wide area network environment. Thus, this process, through audio data segmentation processing, real-time monitoring of forwarding server status, and dynamic control, effectively avoids audio transmission delays, stuttering, or even interruptions caused by forwarding server overload or link degradation. It significantly improves the system's adaptability and robustness in complex network environments, providing stable and efficient data transmission guarantees for bidirectional audio interaction in cross-NAT IoT networks.

[0084] As can be seen, by adopting the above method and using a distributed architecture of scheduling server, forwarding server and storage server, the scheduling server is responsible for global device management, connection scheduling and NAT traversal coordination and the construction of audio transmission channels. The forwarding server allocates transmission tasks for audio transmission, and the IoT terminal accesses the public network without the need for additional public IP configuration. It supports bidirectional communication across regions and network operators, thereby improving the success rate of IoT device connection, while reducing audio transmission latency and improving transmission stability.

[0085] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the server includes the corresponding hardware structure and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0086] This application embodiment can divide the server into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0087] When dividing each function into modules according to its corresponding function. Figure 7 This is a functional block diagram of an audio transmission control device based on cross-network address translation provided in an embodiment of this application. The audio transmission control device 700 based on cross-network address translation is applied to a scheduling server of an audio transmission control system. The audio transmission control system further includes a first terminal, a second terminal, and a forwarding server. The first terminal and the second terminal are respectively communicatively connected to the scheduling server and the forwarding server. The device includes: The acquisition unit 701 is used to acquire the first device information and the first network information of the first terminal; and to acquire the second device information and the second network information of the second terminal. The determining unit 702 is configured to determine a mapping relationship library between terminal device identifier and network information based on the first device information, the first network information, the second device information, and the second network information; The processing unit 703 is configured to, upon receiving audio transmission request information from the first terminal, query the wide area network information of the second terminal from the mapping relationship database to obtain third network information; and generate an audio transmission scheduling instruction based on the first network information, the third network information, and the audio transmission request information. The control unit 704 is used to establish a transmission channel between the first terminal and the second terminal based on the mapping relationship library and the third network information to obtain an audio transmission channel; and to send the audio transmission scheduling instruction to the forwarding server so that the audio data of the first terminal is forwarded to the second terminal through the audio transmission channel via the forwarding server.

[0088] In one possible embodiment, the processing unit 703, in generating the audio transmission scheduling instruction based on the first network information, the third network information, and the audio transmission request information, is specifically configured to: Determine the first network address information and the first network address translation type in the first network information; Determine the second network address information, the second terminal status information, and the second network address translation type in the third network information; When the second terminal status information is online, a matching analysis is performed based on the first network address translation type and the second network address translation type to obtain the combination result of the network address translation types of the first terminal and the second terminal. If the network address translation type combination result satisfies the preset direct connection conditions between the first terminal and the second terminal, the peer access mapping information is determined according to the first network address information and the second network address information; a first audio transmission scheduling instruction is generated according to the peer access mapping information and the audio transmission request, and the first audio transmission scheduling instruction is sent to the first terminal and the second terminal, so that the audio in the storage server connected to the first terminal is sent to the second terminal through the forwarding server; If the network address translation type combination result does not meet the direct connection condition, the relay node information of the forwarding server is determined according to the first network address information and the second network address information; the scheduling information of the relay channel between the first terminal and the second terminal is determined according to the relay node information, the first network address translation type and the second network address translation type; a second audio transmission scheduling instruction is generated according to the scheduling information and the audio transmission request information, so that the audio in the storage server connected to the first terminal is sent to the second terminal through the relay channel.

[0089] In one possible embodiment, the processing unit 703, in determining the scheduling information of the relay channel between the first terminal and the second terminal based on the relay node information, the first network address translation type, and the second network address translation type, is specifically used for: Obtain the load information of the multiple nodes; Determine the first link cost between the first terminal and the relay node that provides the relay node information; and the second link cost between the second terminal and the relay node; Determine the relay node address and node load information corresponding to the relay node in the relay node information; Based on the first network address translation type and the relay node address, a path cost impact factor for network address translation is determined to obtain a first impact factor; and based on the second network address translation type and the relay node address, a path cost impact factor for network address translation is determined to obtain a second impact factor. The relay node link cost is determined based on the first link cost, the second link cost, the first influence factor, and the second influence factor. The scheduling information of the relay channel is determined based on the relay node link cost and the node load information.

[0090] In one possible embodiment, the control unit 704, in establishing the audio transmission channel between the first terminal and the second terminal based on the mapping database and the third network information, specifically performs the following: Determine the network address translation type information in the third network information; The network address and network address translation type corresponding to the second terminal are filtered out from the mapping relationship database to obtain the target network address information and the target network address translation type; Based on the audio transmission scheduling instruction, the target network address information is sent to the first terminal and the second terminal respectively, so as to control the first terminal to send a connection request to the second terminal based on the audio transmission scheduling instruction; After the second terminal responds to the connection request, a first transmission channel is generated based on the network address translation type information and the target network address translation type; An audio test data packet is sent to the second terminal through the first transmission channel for detection, and the detection result is obtained. When the detection result meets the preset audio transmission requirements, the audio transmission channel is determined based on the first transmission channel, the first terminal, and the second terminal.

[0091] In one possible embodiment, after sending the audio transmission scheduling instruction to the forwarding server, the control unit 704 is further configured to: Send an audio data fragmentation instruction to the forwarding server to control the audio data stored in the storage server connected to the first terminal to be fragmented to obtain multiple fragmented audio data; According to the audio scheduling instruction, the multiple audio data segments are sent to the forwarding server through the audio transmission channel to obtain audio transmission information; Determine the transmission parameters of the forwarding server in the audio transmission information; When the transmission parameters are greater than or equal to the preset transmission parameter threshold of the forwarding server, the audio transmission parameters of the multiple audio data segments are controlled, and / or the forwarding server is switched to transmit the multiple audio data segments.

[0092] In one possible embodiment, the determining unit 702, in determining the mapping relationship library between the terminal device identifier and the network information based on the first device information, the first network information, the second device information, and the second network information, is specifically used for: Receive first and second detection data from the first terminal and the second terminal in response to the public network mapping detection command; A first virtual device identifier is generated based on the first device information; and a second virtual device identifier is generated based on the second device information; Based on the first detection data, determine the first wide area network address, the first network address translation type, and the first network translation port mapping information corresponding to the first terminal; and based on the second detection data, determine the second wide area network address, the second network address translation type, and the second network translation port mapping information corresponding to the second terminal; A first association mapping record is obtained by performing association mapping based on the first WAN address, the first network address translation type, the first network translation port mapping information, and the first virtual device identifier; and a second association mapping record is obtained by performing association mapping based on the second WAN address, the second network address translation type, the second network translation port mapping information, and the second virtual device identifier. The mapping relationship library is determined based on the first association mapping record and the second association mapping record.

[0093] In one possible embodiment, the determining unit 702, in determining the mapping relationship database based on the first associated mapping record and the second associated mapping record, is specifically used for: A mapping table structure is established using the first virtual device identifier and the second virtual device identifier as indexes to obtain the first mapping table; The first mapping table is updated based on the first associated mapping record and the second associated mapping record to obtain the second mapping table; Determine the association identifier between the first virtual device identifier and the second virtual device identifier in the second mapping table to obtain multiple association relationship identifiers; Based on the multiple association identifiers, a query index is generated to query the first virtual device identifier and the second virtual device identifier, resulting in multiple query indexes; The multiple query indexes are mapped to the second mapping table to obtain the mapping relationship library.

[0094] As can be seen, this application provides an audio transmission control device based on cross-network address translation. It acquires first device information and first network information of a first terminal; and second device information and second network information of a second terminal; determines a mapping relationship library between terminal device identifiers and network information based on the first device information, first network information, second device information, and second network information; upon receiving an audio transmission request from the first terminal, it queries the WAN information of the second terminal from the mapping relationship library to obtain third network information; generates an audio transmission scheduling instruction based on the first network information, third network information, and audio transmission request information; establishes an audio transmission channel between the first terminal and the second terminal based on the mapping relationship library and the third network information; and sends the audio transmission scheduling instruction to a forwarding server so that the audio data of the first terminal is forwarded to the second terminal via the audio transmission channel through the forwarding server. Thus, by having the scheduling server responsible for global device management, connection scheduling, and NAT traversal coordination, and constructing the audio transmission channel, and enabling the forwarding server to allocate transmission tasks for audio transmission, the success rate of IoT device connections is improved, and audio transmission latency is reduced; simultaneously, the stability of cross-NAT audio transmission is enhanced.

[0095] This application also provides a computer-readable storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes a server.

[0096] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include a server.

[0097] It should be noted that, for the sake of simplicity, the above embodiments are all described as a series of actions. Those skilled in the art should understand that this application is not limited to the described order of actions, as some steps in the embodiments of this application can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions, steps, modules, or units involved are not necessarily essential to the embodiments of this application.

[0098] In the above embodiments, the descriptions of each embodiment in this application have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0099] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0100] The steps of the methods or algorithms described in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in RAM, flash memory, ROM, EPROM, electrically erasable programmable read-only memory (EEPROM), registers, hard disk, portable hard disk, read-only optical disk (CD-ROM), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium.

[0101] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in the embodiments of this application can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The various modules / units included in the devices and products described in the above embodiments can be software modules / units, hardware modules / units, or a combination of both.

[0102] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above descriptions are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. An audio transmission control method based on cross-network address translation, characterized in that, A scheduling server is used in an audio transmission control system, which further includes a first terminal, a second terminal, and a forwarding server. The first terminal and the second terminal are respectively communicatively connected to the scheduling server and the forwarding server, and the method includes: Obtain first device information and first network information of the first terminal; and obtain second device information and second network information of the second terminal; A mapping relationship library between terminal device identifiers and network information is determined based on the first device information, the first network information, the second device information, and the second network information; Upon receiving the audio transmission request information from the first terminal, the wide area network information of the second terminal is queried from the mapping relationship database to obtain the third network information; An audio transmission scheduling instruction is generated based on the first network information, the third network information, and the audio transmission request information; An audio transmission channel is established between the first terminal and the second terminal based on the mapping relationship library and the third network information, thus obtaining the audio transmission channel; The audio transmission scheduling instruction is sent to the forwarding server so that the audio data of the first terminal is forwarded to the second terminal through the audio transmission channel via the forwarding server.

2. The method as described in claim 1, characterized in that, The step of generating an audio transmission scheduling instruction based on the first network information, the third network information, and the audio transmission request information includes: Determine the first network address information and the first network address translation type in the first network information; Determine the second network address information, the second terminal status information, and the second network address translation type in the third network information; When the second terminal status information is online, a matching analysis is performed based on the first network address translation type and the second network address translation type to obtain the combination result of the network address translation types of the first terminal and the second terminal. If the network address translation type combination result satisfies the preset direct connection conditions between the first terminal and the second terminal, the peer access mapping information is determined according to the first network address information and the second network address information; a first audio transmission scheduling instruction is generated according to the peer access mapping information and the audio transmission request, and the first audio transmission scheduling instruction is sent to the first terminal and the second terminal, so that the audio in the storage server connected to the first terminal is sent to the second terminal through the forwarding server; If the network address translation type combination result does not meet the direct connection condition, the relay node information of the forwarding server is determined according to the first network address information and the second network address information; the scheduling information of the relay channel between the first terminal and the second terminal is determined according to the relay node information, the first network address translation type and the second network address translation type; a second audio transmission scheduling instruction is generated according to the scheduling information and the audio transmission request information, so that the audio in the storage server connected to the first terminal is sent to the second terminal through the relay channel.

3. The method as described in claim 2, characterized in that, The step of determining the scheduling information of the relay channel between the first terminal and the second terminal based on the relay node information, the first network address translation type, and the second network address translation type includes: Determine the first link cost between the first terminal and the relay node that provides the relay node information; and the second link cost between the second terminal and the relay node; Determine the relay node address and node load information corresponding to the relay node in the relay node information; Based on the first network address translation type and the relay node address, a path cost impact factor for network address translation is determined to obtain a first impact factor; and based on the second network address translation type and the relay node address, a path cost impact factor for network address translation is determined to obtain a second impact factor. The relay node link cost is determined based on the first link cost, the second link cost, the first influence factor, and the second influence factor. The scheduling information of the relay channel is determined based on the relay node link cost and the node load information.

4. The method according to any one of claims 1-3, characterized in that, The step of establishing an audio transmission channel between the first terminal and the second terminal based on the mapping relationship database and the third network information, to obtain the audio transmission channel, includes: Determine the network address translation type information in the third network information; The network address and network address translation type corresponding to the second terminal are filtered out from the mapping relationship database to obtain the target network address information and the target network address translation type; Based on the audio transmission scheduling instruction, the target network address information is sent to the first terminal and the second terminal respectively, so as to control the first terminal to send a connection request to the second terminal based on the audio transmission scheduling instruction; After the second terminal responds to the connection request, a first transmission channel is generated based on the network address translation type information and the target network address translation type; An audio test data packet is sent to the second terminal through the first transmission channel for detection, and the detection result is obtained. When the detection result meets the preset audio transmission requirements, the audio transmission channel is determined based on the first transmission channel, the first terminal, and the second terminal.

5. The method according to any one of claims 1-3, characterized in that, After sending the audio transmission scheduling instruction to the forwarding server, the method further includes: Send an audio data fragmentation instruction to the forwarding server to control the audio data stored in the storage server connected to the first terminal to be fragmented to obtain multiple fragmented audio data; According to the audio scheduling instruction, the multiple audio data segments are sent to the forwarding server through the audio transmission channel to obtain audio transmission information; Determine the transmission parameters of the forwarding server in the audio transmission information; When the transmission parameters are greater than or equal to the preset transmission parameter threshold of the forwarding server, the audio transmission parameters of the multiple audio data segments are controlled, and / or the forwarding server is switched to transmit the multiple audio data segments.

6. The method according to any one of claims 1-3, characterized in that, The step of determining the mapping relationship library between terminal device identifiers and network information based on the first device information, the first network information, the second device information, and the second network information includes: Receive first and second detection data from the first terminal and the second terminal in response to the public network mapping detection command; A first virtual device identifier is generated based on the first device information; and a second virtual device identifier is generated based on the second device information; Based on the first detection data, determine the first wide area network address, the first network address translation type, and the first network translation port mapping information corresponding to the first terminal; and based on the second detection data, determine the second wide area network address, the second network address translation type, and the second network translation port mapping information corresponding to the second terminal; The first virtual device identifier is associated with the first wide area network address, the first network address translation type, and the first network translation port mapping information to obtain a first associated mapping record; and the second virtual device identifier is associated with the second wide area network address, the second network address translation type, and the second network translation port mapping information to obtain a second associated mapping record. The mapping relationship library is determined based on the first association mapping record and the second association mapping record.

7. The method as described in claim 6, characterized in that, The step of determining the mapping relationship database based on the first association mapping record and the second association mapping record includes: A mapping table structure is established using the first virtual device identifier and the second virtual device identifier as indexes to obtain the first mapping table; The first mapping table is updated based on the first associated mapping record and the second associated mapping record to obtain the second mapping table; Determine the association identifier between the first virtual device identifier and the second virtual device identifier in the second mapping table to obtain multiple association relationship identifiers; Based on the multiple association relationship identifiers, a query index is generated to query the first virtual device identifier and the second virtual device identifier, resulting in multiple query indexes; The multiple query indexes are mapped to the second mapping table to obtain the mapping relationship library.

8. An audio transmission control device based on cross-network address translation, characterized in that, A scheduling server is used in an audio transmission control system, which further includes a first terminal, a second terminal, and a forwarding server. The first terminal and the second terminal are respectively communicatively connected to the scheduling server and the forwarding server, and the device includes: The acquisition unit is configured to acquire first device information and first network information of the first terminal; and to acquire second device information and second network information of the second terminal. The determining unit is configured to determine a mapping relationship library between terminal device identifiers and network information based on the first device information, the first network information, the second device information, and the second network information; The processing unit is configured to, upon receiving audio transmission request information from the first terminal, query the wide area network information of the second terminal from the mapping relationship database to obtain third network information; and generate an audio transmission scheduling instruction based on the first network information, the third network information, and the audio transmission request information. The control unit is used to establish a transmission channel between the first terminal and the second terminal based on the mapping relationship library and the third network information, thereby obtaining an audio transmission channel; and to send the audio transmission scheduling instruction to the forwarding server so that the audio data of the first terminal is forwarded to the second terminal through the audio transmission channel via the forwarding server.

9. A server, characterized in that, include: Processor, memory, communication interface, and one or more programs; The one or more programs are stored in the memory and configured to be executed by the processor, the programs including instructions for performing the steps of the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-7.

Images