Multi-party audio and video communication method and device based on VoLTE network and VOIP fusion

Abstract

The invention provides a multi-party audio and video communication method and device based on VoLTE network and VOIP fusion, and the method comprises the steps: accessing a VoLTE network if a communication terminal is in a coverage range of the VoLTE network, starting the VOIP network if the communication quality of the VoLTE network does not meet a preset communication quality standard, and connecting the communication terminal to the VOIP network based on the available connection information of the VOIP network; establishing a first audio and video transmission channel and a second audio and video transmission channel between the communication terminal and the VoLTE network and the VOIP network, and determining a main transmission channel and an auxiliary transmission channel based on audio and video transmission states of the first audio and video transmission channel and the second audio and video transmission channel; and transmitting the audio and video data of the communication terminal based on the main transmission channel, and when the main transmission channel is abnormal, switching to the auxiliary transmission channel to transmit the audio and video data. According to the invention, the stability and continuity of multi-party audio and video communication are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to a multi-party audio and video communication method and device based on VoLTE network and VOIP fusion. BACKGROUND

[0002] With the rapid development of Internet technology and communication industry, people's demand for remote communication is growing, and multi-party audio and video communication has become an important way to meet the needs of cross-regional exchange and cooperation.

[0003] At present, in the field of multi-party audio and video communication, the common communication methods mainly include the communication method based on traditional circuit switching network and the communication method based on Voice over Internet Protocol (VOIP). Although the communication method based on traditional circuit switching network has the advantages of high stability and clear sound quality, it has the problems of low network resource utilization and poor expansibility, and it is difficult to quickly adapt to the demand of large-scale users online multi-party audio and video communication. Although the communication method based on VOIP can effectively utilize network resources and meet the demand of user scale growth to a certain extent, it is greatly affected by network environment fluctuation, and in the case of network congestion or instability, it is easy to appear audio and video lag, delay or even interruption, which seriously affects the user experience. In addition, the two types of existing multi-party audio and video communication methods are difficult to realize seamless switching in different network environments, which limits the continuity and reliability of communication, and cannot provide stable multi-party audio and video communication service for users. SUMMARY

[0004] The present application provides a multi-party audio and video communication method and device based on VoLTE network and VOIP fusion, which can improve the stability and continuity of multi-party audio and video communication.

[0005] In a first aspect, the present application provides a multi-party audio and video communication method based on VoLTE network and VOIP fusion, comprising: If the network environment parameter based on the network environment where the communication terminal is located is determined to be in the coverage range of VoLTE network, the communication terminal is connected to VoLTE network, and the communication quality of the VoLTE network is determined based on the communication link state of the VoLTE network whether it meets the preset communication quality standard; If not, start the VOIP network, and connect the communication terminal to the VOIP network based on the available connection information of the VOIP network; A first audio and video transmission channel and a second audio and video transmission channel are established between the communication terminal and the VoLTE network and the VoIP network respectively, and a main transmission channel and an auxiliary transmission channel are determined based on the audio and video transmission status of the first audio and video transmission channel and the second audio and video transmission channel. The communication terminal transmits audio and video data through the main transmission channel. When the main transmission channel malfunctions, the transmission is switched to the auxiliary transmission channel to transmit the audio and video data.

[0006] Secondly, the present invention also provides a multi-party audio and video communication device based on the convergence of VoLTE network and VoIP, applied to the multi-party audio and video communication method based on the convergence of VoLTE network and VoIP as described in the first aspect; the multi-party audio and video communication device is used for: If the network environment parameters of the network environment where the communication terminal is located determine that it is within the coverage area of ​​the VoLTE network, then the communication terminal is connected to the VoLTE network, and the communication quality of the VoLTE network is determined to meet the preset communication quality standard based on the communication link status of the VoLTE network. If the conditions are not met, the VoIP network is activated, and the communication terminal is connected to the VoIP network based on the available connection information of the VoIP network. A first audio and video transmission channel and a second audio and video transmission channel are established between the communication terminal and the VoLTE network and the VoIP network respectively, and a main transmission channel and an auxiliary transmission channel are determined based on the audio and video transmission status of the first audio and video transmission channel and the second audio and video transmission channel. The communication terminal transmits audio and video data through the main transmission channel. When the main transmission channel malfunctions, the transmission is switched to the auxiliary transmission channel to transmit the audio and video data.

[0007] Thirdly, the present invention also provides an electronic device, comprising: a memory for storing computer software programs; and a processor for reading and executing the computer software programs, thereby realizing the multi-party audio and video communication method based on the convergence of VoLTE network and VoIP as described above.

[0008] Fourthly, the present invention also provides a non-transitory computer-readable storage medium storing a computer software program, which, when executed by a processor, implements the multi-party audio and video communication method based on the convergence of VoLTE network and VoIP as described above.

[0009] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the multi-party audio and video communication method based on the convergence of VoLTE network and VoIP as described above.

[0010] The multi-party audio and video communication method based on VoLTE and VoIP integration provided in this invention obtains network environment parameters and determines whether the communication terminal is within VoLTE network coverage by real-time detection of the network environment. It then connects the communication terminal to the VoLTE network, ensuring that high-quality VoLTE is prioritized for communication when conditions are favorable. Based on the communication quality assessment after VoLTE network access, a backup VoIP network is activated if the quality does not meet standards, preventing communication quality degradation due to poor VoLTE network quality. The method simultaneously connects the communication terminal to both VoLTE and VoIP networks, establishing audio and video transmission channels on both networks, enabling dual-channel transmission of audio and video data. By selecting the network channel with better transmission quality as the primary transmission channel and the other as an auxiliary transmission channel, it ensures that audio and video data is preferentially transmitted through the high-quality channel, and finally transmitted primarily through the primary transmission channel. When the primary transmission channel malfunctions, it automatically switches to the auxiliary transmission channel, avoiding audio and video stuttering, delays, or even interruptions caused by network environment fluctuations, as well as the difficulty in seamless switching between different network environments. This improves the stability and continuity of multi-party audio and video communication. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating the multi-party audio and video communication method based on the convergence of VoLTE network and VoIP provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a multi-party audio and video communication device based on the convergence of VoLTE network and VoIP provided in an embodiment of the present invention; Figure 3 An embodiment diagram of the electronic device provided in this invention; Figure 4 An embodiment diagram of a computer-readable storage medium provided in accordance with the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0014] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0015] Optional, see below Figure 1 , Figure 1 This is a flowchart illustrating the multi-party audio and video communication method based on VoLTE network and VoIP integration provided by the present invention. In this embodiment, the executing entity of the multi-party audio and video communication method based on VoLTE network and VoIP integration is an audio and video communication device. Therefore, the multi-party audio and video communication method based on VoLTE network and VoIP integration includes: Step 10: If the network environment parameters of the network environment where the communication terminal is located determine that it is within the coverage area of ​​the VoLTE network, then the communication terminal is connected to the VoLTE network, and the communication quality of the VoLTE network is determined to meet the preset communication quality standard based on the communication link status of the VoLTE network.

[0016] Optionally, the audio and video communication device obtains network environment parameters of the network environment in which the communication terminal is located. These network environment parameters include, but are not limited to, network type identifiers (such as whether it contains the "VoLTE" identifier), network signal strength (such as reference signal received power, signal-to-interference plus noise ratio), and network coverage information (such as the VoLTE service availability area broadcast by the base station). Based on these network environment parameters, the device determines whether the communication terminal is within the coverage area of ​​the Long Term Evolution Voice Bearer (VoLTE) network.

[0017] Furthermore, if it is determined that the device is within the coverage area, the audio / video communication device controls the communication terminal to access the VoLTE network (i.e., establish a connection with the VoLTE network through registration, authentication, and other processes). After access, the audio / video communication device further obtains the communication link status parameters of the VoLTE network, including but not limited to link transmission rate, packet loss rate, latency, jitter, etc., and then compares the communication link status parameters with preset communication quality standards (such as preset packet loss rate ≤1%, latency ≤100 milliseconds, jitter ≤20 milliseconds) to determine whether the communication quality of the VoLTE network meets the standard.

[0018] In one embodiment, the audio / video communication device detects that the network type identifier of the communication terminal includes "VoLTE", the reference signal received power is -85dBm (within a relatively strong signal range), and the base station broadcast information indicates that the current area is a VoLTE service available area. Therefore, it determines that the communication terminal is within the VoLTE network coverage area. Subsequently, the audio / video communication device controls the communication terminal to initiate registration with the VoLTE network, and successfully accesses the network after authentication.

[0019] Step 20: If the conditions are not met, then activate the VoIP network and connect the communication terminal to the VoIP network based on the available connection information of the VoIP network.

[0020] Furthermore, when it is determined that the communication quality of the VoLTE network does not meet the preset communication quality standards, the audio and video communication device activates the Voice over Internet Protocol (VoIP) network function. After activation, the audio and video communication device obtains available connection information of the VoIP network, including but not limited to available VoIP server addresses, server port numbers, network protocol types (such as session initiation protocols), VoIP network signal strength, and account authentication information required for access.

[0021] Furthermore, the audio and video communication device controls the communication terminal to establish a connection with the VoIP network based on available connection information (such as initiating a call request to the VoIP server through the session initiation protocol to complete the session establishment), so that the communication terminal can successfully access the VoIP network, as specifically in steps 201 to 206.

[0022] Continuing with the above embodiment, after access, the audio / video communication device obtains the communication link status of the VoLTE network: transmission rate of 2Mbps, packet loss rate of 3%, latency of 150 milliseconds, and jitter of 30 milliseconds. The preset communication quality standard is packet loss rate ≤1%, latency ≤100 milliseconds, and jitter ≤20 milliseconds. After comparison, the current packet loss rate, latency, and jitter all exceed the standard. Therefore, it is determined that the communication quality of the VoLTE network does not meet the preset communication quality standard. Consequently, the audio / video communication device activates the VoIP network function and connects the communication terminal to the VoIP network based on the available connection information of the VoIP network.

[0023] Step 30: Establish a first audio and video transmission channel and a second audio and video transmission channel between the communication terminal and the VoLTE network and the VoIP network respectively, and determine the main transmission channel and the auxiliary transmission channel based on the audio and video transmission status of the first audio and video transmission channel and the second audio and video transmission channel.

[0024] Furthermore, based on the fact that the communication terminal has been connected to the VoLTE network and the VoIP network, the audio and video communication device establishes audio and video transmission channels for the two networks respectively: a first audio and video transmission channel is established for the VoLTE network (based on the real-time transmission protocol of the VoLTE network or a similar protocol, used to transmit audio and video data), and a second audio and video transmission channel is established for the VoIP network (based on the real-time transmission protocol of the VoIP network or a similar protocol, used to transmit audio and video data), as described in steps 301 to 309.

[0025] Furthermore, the audio and video communication device acquires audio and video transmission status parameters of the first and second audio and video transmission channels, including but not limited to the channel's real-time transmission rate, data packet loss rate, transmission latency, jitter level, and channel stability (such as whether there are frequent interruptions). Based on these parameters, the audio and video communication device determines the main transmission channel and the auxiliary transmission channel through preset rules (such as the channel with higher transmission rate, lower packet loss rate, and lower latency being the main transmission channel, and the other channel being the auxiliary transmission channel).

[0026] In one embodiment, the transmission status of the first audio / video transmission channel was detected as follows: transmission rate 1.5 Mbps, packet loss rate 3%, latency 150 ms, jitter 30 ms, and two brief interruptions within the past 10 seconds; the transmission status of the second audio / video transmission channel was as follows: transmission rate 2 Mbps, packet loss rate 0.5%, latency 80 ms, jitter 10 ms, and no interruptions within the past 10 seconds. The preset rule is "prioritize the channel with high transmission rate, low packet loss rate, low latency, and high stability as the main transmission channel." Therefore, the second audio / video transmission channel (VoIP network) is determined to be the main transmission channel, and the first audio / video transmission channel (VoLTE network) is determined to be the auxiliary transmission channel.

[0027] Step 40: Transmit audio and video data of the communication terminal based on the main transmission channel. If the main transmission channel malfunctions, switch to the auxiliary transmission channel to transmit audio and video data.

[0028] Furthermore, the audio and video communication device controls the communication terminal to prioritize the transmission of audio and video data (such as voice signals and video image data) through the main transmission channel. During transmission, the audio and video communication device continuously monitors the status of the main transmission channel, and the monitored parameters include, but are not limited to, whether transmission interruption occurs, whether packet loss rate suddenly increases (e.g., exceeding a preset threshold), whether latency increases sharply (e.g., exceeding a preset threshold), and whether transmission rate drops significantly (e.g., below a preset threshold).

[0029] Furthermore, when any of the above-mentioned abnormalities are detected in the main transmission channel, the audio and video communication device immediately triggers the channel switching mechanism, stops transmitting data through the main transmission channel, and switches the audio and video data to the auxiliary transmission channel for transmission to ensure the continuity of audio and video communication.

[0030] Continuing with the above embodiment, the first audio / video transmission channel (VoLTE network) serves as an auxiliary transmission channel. The audio / video communication device controls the transmission of audio and video data from the communication terminal through the second audio / video transmission channel. During transmission, the audio / video communication device monitors in real time that the packet loss rate of the second audio / video transmission channel suddenly increases from 0.5% to 10% (the preset abnormal threshold is 5%), and the latency increases from 80 milliseconds to 300 milliseconds (the preset abnormal threshold is 200 milliseconds), determining that the main transmission channel has malfunctioned. At this time, the audio / video communication device immediately triggers a switchover, stopping data transmission through the second audio / video transmission channel and switching the audio / video data to the first audio / video transmission channel (VoLTE network) for transmission, ensuring uninterrupted audio / video communication.

[0031] This invention, through real-time detection of the network environment of the communication terminal, obtains network environment parameters and determines whether it is within the VoLTE network coverage area. It then connects the communication terminal to the VoLTE network, ensuring that high-quality VoLTE is prioritized for communication when conditions are favorable. Based on the communication quality assessment after connecting to the VoLTE network, a backup VoIP network is activated if the quality does not meet standards, preventing communication quality degradation due to poor VoLTE network quality. By simultaneously connecting the communication terminal to both the VoLTE and VoIP networks and establishing audio and video transmission channels on both networks, dual-channel transmission of audio and video data is possible. By selecting the network channel with better transmission quality as the primary transmission channel and the other as the auxiliary transmission channel, audio and video data is prioritized for transmission through the high-quality channel, and finally transmitted primarily through the primary transmission channel. When the primary transmission channel malfunctions, it automatically switches to the auxiliary transmission channel, avoiding audio and video stuttering, delays, or even interruptions caused by network environment fluctuations, as well as the difficulty in seamless switching between different network environments. This improves the stability and continuity of multi-party audio and video communication.

[0032] In one embodiment, steps 201 to 206 include: Step 201: Based on available connection information, candidate access nodes that meet the connection stability requirements in the VoIP network are selected, and based on the network operator information of each candidate access node, target access nodes that are compatible with the network operator of the VoLTE network are determined.

[0033] Optionally, the audio / video communication device filters candidate access nodes that meet connection stability requirements based on available connection information (including connection stability parameters and network operator information for each access node). Connection stability requirements include, but are not limited to, the historical connection interruption frequency of the access node (e.g., number of interruptions per unit time ≤ preset number) and signal fluctuation amplitude (e.g., signal strength variation range ≤ preset amplitude). After filtering candidate access nodes, the audio / video communication device extracts the network operator information (e.g., operator name, network identification code) of each candidate access node and compares it with the network operator information of the VoLTE network to determine the nodes that are compatible with both as target access nodes (the compatibility criterion is that there is a network interoperability agreement or roaming cooperation relationship between the operators).

[0034] In one embodiment, the available connection information of the VoIP network includes three access nodes: Node A (operator "telecom operator X", with 1 interruption in the past hour and a signal fluctuation of 5dB), Node B (operator "telecom operator Y", with 5 interruptions in the past hour and a signal fluctuation of 15dB), and Node C (operator "telecom operator X", with 0 interruptions in the past hour and a signal fluctuation of 3dB). The connection stability requirement is "the number of interruptions in the past hour ≤ 2 and the signal fluctuation ≤ 10dB", therefore Node A and Node C are selected as candidate access nodes. In step 10, the VoLTE network operator is "telecom operator X", which is fully compatible with "telecom operator X" but has no compatible protocol with "telecom operator Y", therefore Node A and Node C are determined as target access nodes.

[0035] Step 202: Calculate the load rate based on the real-time network load data of each target access node, and select priority access nodes from the target access nodes whose load rates are lower than the preset load threshold.

[0036] Further, the audio / video communication device acquires real-time network load data for each target access node. This real-time network load data includes the number of currently connected users, the current data transmission rate, and the maximum supported transmission rate. Based on these parameters, the device calculates the load rate of each target access node, as described in steps 2021 to 2023. Then, the audio / video communication device compares the load rate of each target access node with a preset load threshold (e.g., 70%), selecting nodes with load rates below this threshold as priority access nodes.

[0037] In one embodiment, the target access nodes are node A and node C. The load rate of node A is 80% > 70%, and the load rate of node C is 50% < 70%. Therefore, node C is selected as the preferred access node.

[0038] Step 203: Sort the priority access nodes based on the network delay parameters of each priority access node to obtain the sorted priority access nodes, and select the first-ranked priority access node as the final access node.

[0039] Furthermore, the audio / video communication device acquires the network latency parameters of each priority access node. The network latency parameters are the round-trip time (RTT) between the communication terminal and the priority access node (i.e., the time interval from when the communication terminal sends data to when the receiving node responds). All priority access nodes are sorted based on the RTT, with the node ranking higher for smaller RTTs. If multiple priority access nodes exist, the node ranked first (i.e., with the smallest RTT) is selected as the final access node; if only one priority access node exists, that node is directly selected as the final access node.

[0040] Continuing with the above embodiment, node C is the preferred access node (assuming node D is also a preferred access node to complete the example). Node C's network latency (round-trip time) is 40 milliseconds, and node D's network latency (round-trip time) is 60 milliseconds. Following the rule of "lower round-trip time, higher ranking," node C ranks first, and node D ranks second; therefore, node C is the final access node.

[0041] Step 204: Generate a connection request between the communication terminal and the VoIP network based on the final access node.

[0042] Furthermore, the audio and video communication device generates a connection request between the communication terminal and the VoIP network based on the final access node. This connection request includes, but is not limited to: the communication terminal's device identifier (e.g., International Mobile Equipment Identity), user account information (e.g., VoIP account), the final access node's address information (e.g., server IP address, port number), the requested network protocol type (e.g., Session Initiation Protocol), and the audio and video encoding formats supported by the communication terminal. This ensures that the final access node can identify the request source and access requirements. Continuing with the above embodiment, the final access node is node C (server IP address "192.168.1.100", port number 5060). The generated connection request includes: the communication terminal's International Mobile Equipment Identity "351234567890123", the VoIP account "user123@voipserver.com", node C's IP address and port number "192.168.1.100:5060", the protocol type "Session Initiation Protocol", and the supported audio and video encoding formats "G.711 (voice), H.264 (video)".

[0043] Step 205: Send an access request to the final access node based on the connection request, and determine whether to allow the communication terminal to access the VoIP network based on the feedback information from the final access node.

[0044] Further, the audio / video communication device sends a connection request to the final access node according to the transmission method supported by the final access node (such as based on User Datagram Protocol or Transmission Control Protocol), initiating an access request. Afterwards, the audio / video communication device waits for and receives feedback information returned by the final access node. This feedback information includes allowing access (e.g., including an "authentication successful" identifier and session parameters) or denying access (e.g., including reasons for denial such as "invalid account" or "node busy"). Based on the feedback information, the audio / video communication device determines whether the final access node allows the communication terminal to access the VoIP network. Continuing with the above embodiment, the audio / video communication device sends a connection request to node C ("192.168.1.100:5060") via User Datagram Protocol. After verification, node C returns feedback information: "authentication successful, access allowed, session identifier: S123456". Based on this information, the audio / video communication device determines that the final access node allows the communication terminal to access the VoIP network.

[0045] Step 206: If access is allowed, the connection between the communication terminal and the VoIP network is completed based on the access configuration parameters of the final access node.

[0046] Furthermore, once the final access node determines that the communication terminal is allowed to access, the audio / video communication device obtains the access configuration parameters returned by the node, including but not limited to session description protocol information (such as audio / video transmission port, encoding format negotiation results), encryption key (used for data transmission encryption), and heartbeat detection time interval (used to maintain the connection). Based on these parameters, the audio / video communication device configures the corresponding network parameters in the communication terminal (such as setting the transmission port, loading the encryption key, and starting the heartbeat detection mechanism) to complete the connection with the VoIP network, enabling the communication terminal to transmit audio and video data through the node.

[0047] Continuing with the above embodiment, the final access node returns the following access configuration parameters: audio / video transmission port "5004-5005", encoding format negotiation result "voice G.711, video H.264", encryption key "KeyABC123", and heartbeat detection interval of 30 seconds. Configure these parameters in the communication terminal, set the transmission port to 5004-5005, enable G.711 and H.264 encoding, load the encryption key "KeyABC123", and start a heartbeat detection every 30 seconds to complete the connection between the communication terminal and the VoIP network.

[0048] The embodiments of the present invention can select nodes from the available access nodes of the VoIP network that are compatible with VoLTE network operators, have stable connections, low load, and minimal latency. Through a standardized request-authentication-configuration process, the communication terminal can efficiently and reliably access the VoIP network, ensuring that the VoIP network can serve as an effective supplement to achieve stable communication when the quality of the VoLTE network is poor.

[0049] In one embodiment, steps 2021 to 2023 include: Step 2021: Calculate the user load percentage based on the number of currently connected users and the maximum number of users that each target access node can support.

[0050] Optionally, the audio / video communication device calculates the user load percentage of each target access node based on the current number of connected users (i.e., the total number of users currently accessing the VoIP network through that node) and the maximum number of users that can be supported (i.e., the maximum number of users that the node's hardware and software configuration can handle) in the real-time network load data of each target access node. The calculation formula is: User load percentage = (Current number of connected users / Maximum number of users supported) * 100%. This percentage reflects the node's load level in terms of user access; the higher the value, the closer the node is to saturation in terms of user capacity.

[0051] Continuing with the target access nodes A and C, node A currently has 800 connected users and can support a maximum of 1000 users. Therefore, node A's user load percentage is (800 / 1000) * 100% = 80%. Node C currently has 500 connected users and can support a maximum of 1000 users. Therefore, node C's user load percentage is (500 / 1000) * 100% = 50%.

[0052] Step 2022: Calculate the data load percentage based on the current data transmission rate of each target access node and the maximum supported transmission rate of each target access node.

[0053] Furthermore, the audio and video communication device calculates the data load percentage of each target access node based on the current data transmission rate (i.e., the total rate of audio and video data currently being transmitted by the node) and the maximum supported transmission rate (i.e., the maximum data transmission rate that the node's hardware and network link can support) in the real-time network load data of each target access node. The calculation formula is: Data load percentage = (Current data transmission rate / Maximum supported transmission rate) * 100%. This percentage reflects the load level of the node in terms of data transmission; the higher the value, the closer the node's transmission bandwidth is to its upper limit.

[0054] Continuing with the above, the current data transmission rate of the target access node A is 8Mbps, and the maximum supported transmission rate is 10Mbps. Therefore, the data load percentage of node A is (8 / 10) * 100% = 80%. The current data transmission rate of node C is 4Mbps, and the maximum supported transmission rate is 10Mbps. Therefore, the data load percentage of node C is (4 / 10) * 100% = 40%.

[0055] Step 2023: Calculate the load rate of each target access node by performing weight calculation based on the user load ratio and data load ratio of each target access node and the corresponding calculation weight.

[0056] Furthermore, the audio and video communication device sets corresponding calculation weights for user load ratio and data load ratio respectively. Based on the user load ratio, data load ratio and the set calculation weights, the load rate of each target access node is calculated. The calculation formula is: load rate = (user load ratio * user load weight) + (data load ratio * data load weight). The load rate comprehensively reflects the overall load of the node in terms of both user carrying and data transmission.

[0057] Continuing with the above embodiment, the weight for calculating the user load percentage is set to 0.6, and the weight for calculating the data load percentage is set to 0.4. For node A, the load rate = (80% * 0.6) + (80% * 0.4) = 48% + 32% = 80%. For node C, the load rate = (50% * 0.6) + (40% * 0.4) = 30% + 16% = 46%.

[0058] The embodiments of the present invention can comprehensively consider the load of the target access node in terms of both user carrying capacity and data transmission capacity, calculate the load rate that reflects the overall load level of the node, and ensure that the selected priority access nodes (load rate below a preset threshold) are in a better state in terms of both user carrying capacity and data transmission capacity, thus providing a guarantee for the stability of data transmission after the communication terminal accesses the VoIP network.

[0059] In one embodiment, steps 301 to 305 include: Step 301: Based on the connection status between the communication terminal and the VoLTE network, obtain the address of the Session Management Function (SMF) node of the VoLTE network, and send a channel establishment request to the corresponding SMF node based on the SMF node address.

[0060] Optionally, the audio / video communication device first confirms the connection status between the communication terminal and the VoLTE network (e.g., whether registration and authentication have been completed and the connection is normal). If the connection is normal, the audio / video communication device obtains the address information (including IP address and port number) of the Session Management Function (SMF) node from the core network control plane of the VoLTE network. This node is responsible for the management of session establishment, modification, and release.

[0061] Furthermore, based on the obtained SMF node address, the audio and video communication device sends a request to establish the first audio and video transmission channel to the corresponding SMF node. The request includes the identification information of the communication terminal (such as the International Mobile Subscriber Identity), the type of audio and video service to be transmitted (such as real-time voice, real-time video), etc., to trigger the SMF node to start the channel establishment process.

[0062] In one embodiment, after detecting that the communication terminal has successfully accessed the VoLTE network and the connection status is normal, the address of the SMF node is obtained from the core network control plane as "10.0.0.1:8080", and a channel establishment request is sent to the SMF node, including the communication terminal's International Mobile Subscriber Identity "460011234567890", service type "real-time audio and video call", etc., requesting the establishment of a first audio and video transmission channel.

[0063] Step 302: Based on the response information of the SMF node to the channel establishment request, obtain the QoS parameters required for audio and video transmission, and select candidate UPF nodes that meet the QoS parameters from the user plane function UPF nodes available in the VoLTE network.

[0064] Furthermore, the audio / video communication device receives the response information from the SMF node to the channel establishment request. This response information includes the Quality of Service (QoS) parameters required for audio / video transmission, including but not limited to QoS level identifiers (such as guaranteed bit rate and maximum bit rate), latency requirements (such as one-way transmission latency limit), and packet loss rate requirements (such as the maximum allowed packet loss percentage). Further, the audio / video communication device filters available User Plane Function (UPF) nodes in the VoLTE network based on the QoS parameters: it compares the performance parameters (such as maximum supported transmission rate, historical average latency, and historical average packet loss rate) of each available UPF node with the QoS parameters, and identifies all UPF nodes whose performance parameters meet the QoS parameter requirements as candidate UPF nodes.

[0065] Continuing with the above embodiment, the QoS parameters in the response information returned by the SMF node are: guaranteed bit rate of 2Mbps, maximum bit rate of 5Mbps, one-way transmission latency ≤100ms, and packet loss rate ≤1%. Available UPF nodes in the VoLTE network include node X, node Y, and node Z. Comparison shows that: node X supports a maximum bit rate of 6Mbps, an average latency of 80ms, and an average packet loss rate of 0.5% (all meeting the QoS parameters); node Y supports a maximum bit rate of 1Mbps (not meeting the guaranteed bit rate of 2Mbps); and node Z has an average latency of 120ms (not meeting the latency ≤100ms requirement). Therefore, node X is selected as a candidate UPF node.

[0066] Step 303: Send a path probing request to each candidate node based on the candidate UPF node to obtain the transmission path information of each transmission path between each candidate node and the communication terminal.

[0067] Furthermore, the audio / video communication device sends a transmission path probe request to each candidate UPF node. This request is transmitted through the potential transmission path between the communication terminal and the candidate UPF node to probe specific path information. Transmission path information includes, but is not limited to, the number of network nodes (e.g., base stations, routers) traversed by the path (i.e., hop count), the link bandwidth of the path, the historical outage frequency of the link (to reflect stability), and the real-time transmission latency of the path. The audio / video communication device receives the probe response returned by each candidate UPF node, extracts and records the transmission path information for all transmission paths between each candidate node and the communication terminal.

[0068] Continuing with the above embodiment, the candidate UPF node is node X. A transmission path probe request is sent to node X, requesting transmission through two potential paths: Path 1 via base station A and router B; Path 2 via base station A and router C. The probe response returned by node X shows: Path 1 has a hop count of 2 (base station A → router B → node X), a link bandwidth of 5 Mbps, and no interruptions in the past 10 minutes; Path 2 has a hop count of 3 (base station A → router C → switch D → node X), a link bandwidth of 5 Mbps, and one interruption in the past 10 minutes. The transmission path information for these two paths is recorded.

[0069] Step 304: Analyze the hop count and link stability of each transmission path based on the transmission path information, and prioritize the candidate UPF nodes based on the hop count and link stability of each transmission path.

[0070] Furthermore, the audio and video communication device analyzes the hop count (i.e., the number of network nodes traversed by the path) and link stability (assessed through historical link outage frequency; lower outage frequency indicates higher stability) of each transmission path between the candidate UPF node and the communication terminal based on transmission path information. For each candidate UPF node, the transmission path with the fewest hops is prioritized; if multiple paths with the same hop count exist, the path with the highest link stability (i.e., the lowest historical outage frequency) is selected as the optimal transmission path for that candidate node. All candidate UPF nodes are prioritized based on the hop count and link stability of the optimal transmission path: candidate UPF nodes with fewer hops and higher link stability have higher priority.

[0071] Continuing with the above embodiment, the candidate UPF node is node X (assuming another node W is also a candidate node to complete the example). Node X has an optimal transmission path hop count of 2 and has had 0 interruptions in the last 10 minutes; node W has an optimal transmission path hop count of 3 and has also had 0 interruptions in the last 10 minutes. According to the rule of "lower hop count, higher priority", node X has a higher priority than node W. Therefore, the priority ranking result is that node X ranks first and node W ranks second.

[0072] Step 305: Establish the first audio and video transmission channel between the communication terminal and the VoLTE network based on the UPF node priority ranking result.

[0073] Furthermore, the audio and video communication device establishes the first audio and video transmission channel between the communication terminal and the VoLTE network according to the priority ranking result of the UPF nodes, as specifically in steps 3051 to 3054.

[0074] The embodiments of the present invention can select the optimal user plane transmission nodes and paths based on the core network functional nodes (SMF, UPF) of the VoLTE network, combined with QoS parameters and transmission path characteristics, and finally establish a first audio and video transmission channel that meets the audio and video transmission quality requirements. This can ensure that audio and video data are transmitted in the VoLTE network with a preset quality of service, and improve the stability of multi-party audio and video communication.

[0075] In one embodiment, steps 3051 to 3054 include: Step 3051: Determine the optimal UPF node based on the UPF node priority sorting result, and generate the path configuration information of the first audio and video transmission channel based on the optimal UPF node.

[0076] Optionally, the audio / video communication device refers to the UPF node priority ranking result and selects the candidate UPF node ranked first as the optimal UPF node. Further, the audio / video communication device generates path configuration information for the first audio / video transmission channel based on this optimal UPF node. The path configuration information includes the node sequence of the transmission link (i.e., the sequential order of all network nodes that data passes through from the communication terminal to the optimal UPF node, such as the communication terminal, base station, router, and optimal UPF node) and port information (i.e., the port numbers used for data transmission between the communication terminal and each intermediate node, between intermediate nodes, and between intermediate nodes and the optimal UPF node, ensuring accurate data routing in the link).

[0077] Continuing with the above embodiment, the candidate UPF node ranked first in the UPF node priority sorting result is node X (address "10.0.1.2:5000"). The audio and video communication device determines node X as the optimal UPF node and generates path configuration information: the node sequence of the transmission link is "communication terminal → base station A → router B → node X"; the port information is "transmission port 5001 between communication terminal and base station A, transmission port 5002 between base station A and router B, and transmission port 5003 between router B and node X".

[0078] Step 3052: Send channel configuration instructions to the communication terminal and the optimal UPF node respectively based on the path configuration information, so as to complete the parameter configuration of the communication terminal and the optimal UPF node based on the channel configuration instructions.

[0079] Furthermore, the audio and video communication device sends channel configuration instructions to the communication terminal and the optimal UPF node based on the path configuration information. The instructions sent to the communication terminal include the address and corresponding transmission port (e.g., 5001) of the next-hop node to be connected (e.g., base station A), the data encapsulation format (e.g., real-time transmission protocol), and QoS parameters (e.g., guaranteed bit rate). The instructions sent to the optimal UPF node include the address and corresponding transmission port (e.g., 5003) of the previous-hop node to be connected (e.g., router B), the data parsing format, and the QoS guarantee mechanism (e.g., bandwidth reservation).

[0080] Furthermore, after receiving the instruction, the communication terminal and the optimal UPF node complete their own network parameter configuration according to the instruction content to prepare for the establishment of the transmission link, as detailed in steps 30521 to 30525.

[0081] Continuing with the above embodiment, a channel configuration instruction is sent to the communication terminal: Next-hop node: Base station A (address 10.0.2.1), transmission port 5001, data encapsulation format: Real-time transmission protocol, guaranteed bit rate 2Mbps; A channel configuration instruction is sent to the optimal UPF node (node ​​X): Previous-hop node: Router B (address 10.0.3.1), transmission port 5003, data parsing format: Real-time transmission protocol, enable bandwidth reservation (reserved 2Mbps), and the communication terminal and node X complete the parameter configuration according to the instructions.

[0082] Step 3053: Establish a transmission link between the communication terminal and the optimal UPF node based on the configured parameters, and verify whether the transmission of audio and video data is normal based on the transmission link.

[0083] Furthermore, after the communication terminal and the optimal UPF node complete parameter configuration, the audio / video communication device triggers the establishment of a transmission link between them based on the configured parameters: the communication terminal sends a connection request through the configured next-hop node address and port, which is routed through intermediate nodes (such as base stations and routers) according to the node sequence, and finally reaches the optimal UPF node. The optimal UPF node receives and responds to the request based on the configured previous-hop node address and port, forming a complete bidirectional transmission link. After the link is established, the audio / video communication device sends test audio and video data (such as a preset voice clip and video frame) to the communication terminal, which is transmitted to the optimal UPF node through the transmission link. The optimal UPF node then returns confirmation information to verify whether the transmission of audio and video data is normal (such as whether the data is received completely, without out-of-order delivery, and without significant distortion).

[0084] Continuing with the above embodiment, the communication terminal establishes a transmission link with the optimal UPF node (Node X) based on configuration parameters: the communication terminal sends a connection request to Node X (10.0.1.2:5003) through base station A (10.0.2.1:5001) and router B (10.0.3.1:5002). After Node X responds, the link is established. Test data (10 seconds of voice clip and 5 frames of video image) is sent to the communication terminal. The data is transmitted to Node X via the link. After receiving the data, Node X returns a confirmation message of "data complete and without distortion," verifying that the transmission is normal.

[0085] Step 3054: If normal, then the first audio and video transmission channel is confirmed to be established.

[0086] Furthermore, when the transmission of audio and video data is verified to be normal (i.e., the test data is received completely without obvious errors or distortion, and the transmission delay and packet loss rate meet expectations), the audio and video communication device determines that the first audio and video transmission channel between the communication terminal and the VoLTE network has been successfully established.

[0087] Continuing with the above embodiment, in step 3053, the test data transmission is normal, and the confirmation information returned by node X shows "the voice segment is complete, the video frame is not lost, the transmission delay is 80 milliseconds, and the packet loss rate is 0%", which meets the expected requirements. Therefore, it is determined that the first audio and video transmission channel has been established successfully.

[0088] The embodiments of the present invention can complete path configuration, parameter synchronization, link establishment and transmission verification based on the optimal UPF node, and finally form a stable first audio and video transmission channel. The first audio and video transmission channel strictly follows the preset path and parameters, ensuring that audio and video data are transmitted in the VoLTE network according to the expected quality standards, thereby improving the stability of multi-party audio and video communication.

[0089] In one embodiment, steps 30521 to 30525 include: Step 30521: Based on the channel configuration command, parameter setting requests and encoding parameters are sent to the modem and audio / video processing module of the communication terminal, respectively, so as to configure the transmission power and signal modulation method of the communication terminal based on the parameter setting request, and to set the compression format and frame rate of the audio / video data based on the encoding parameters.

[0090] Optionally, the audio / video communication device extracts parameter setting information for the communication terminal from the channel configuration command, including transmit power parameters, signal modulation method parameters, and audio / video encoding parameters. First, a parameter setting request is sent to the communication terminal's modem. This request includes specific transmit power values ​​(such as a reasonable power value determined based on network signal strength) and signal modulation methods (such as quadrature amplitude modulation, phase shift keying, etc.). After receiving the request, the modem adjusts the transmit power to the specified value and enables the specified signal modulation method to ensure the stability and efficiency of signal transmission. Simultaneously, encoding parameters are sent to the communication terminal's audio / video processing module. These parameters include the compression format of the audio and video data (such as G.711 format for voice and H.264 format for video) and frame rate (such as 30 frames per second for video). The audio / video processing module completes the encoding configuration based on these parameters, ensuring that the output audio and video data meets transmission requirements.

[0091] Continuing with the above embodiment, the parameter setting request for the communication terminal in the channel configuration instruction is "transmit power 15dBm, signal modulation mode is quadrature amplitude modulation (16QAM)"; the encoding parameters are "voice compression format G.711, video compression format H.264, video frame rate 30 frames / second". The audio and video communication device sends the above parameter setting request to the modem, the modem adjusts the transmit power to 15dBm and enables 16QAM modulation mode; it sends the encoding parameters to the audio and video processing module, which sets the voice encoding to G.711, the video encoding to H.264, and the video frame rate to 30 frames / second.

[0092] Step 30522: Send session establishment parameters to the optimal UPF node based on the channel configuration command, so as to configure the packet filtering rules of the optimal UPF node based on the session establishment parameters.

[0093] Furthermore, the audio / video communication device extracts session establishment parameters from the channel configuration command. These parameters include the communication terminal's identification information (such as the International Mobile Equipment Identity), the protocol type of the audio / video data (such as Real-Time Transmission Protocol), and the data port range (such as the previously determined transmission port). The audio / video communication device then sends these session establishment parameters to the optimal UPF node. Based on these parameters, the optimal UPF node configures packet filtering rules: only packets from the communication terminal that conform to the specified protocol type and are transmitted through the specified port are allowed to pass, while packets that do not conform to the rules are rejected, thus ensuring the security of the transmission link and the accuracy of the data.

[0094] Continuing with the above embodiment, the session establishment parameters in the channel configuration command are "Communication terminal International Mobile Equipment Identity 351234567890123, Protocol type Real-Time Transport Protocol, Port range 5001-5003". The audio and video communication device sends these parameters to the optimal UPF node (Node X). Node X configures data packet filtering rules based on the parameters: only data packets with source identifier 351234567890123, protocol Real-Time Transport Protocol, and port in the range of 5001-5003 are accepted; all other data packets are filtered.

[0095] Step 30523: Set the QoS queue of the optimal UPF node based on the packet filtering rules, and configure the data forwarding priority based on the QoS queue.

[0096] Furthermore, the audio and video communication device, based on the packet filtering rules configured by the optimal UPF node, further controls the setting of Quality of Service (QoS) queues by the optimal UPF node. The QoS queues are set according to the priority requirements of audio and video transmission (e.g., real-time voice data has higher priority than video data, or priority is determined based on QoS level identifiers), and filtered packets are classified and placed into different queues according to priority. The optimal UPF node configures data forwarding priority based on the QoS queues: packets in high-priority queues are forwarded first, while packets in low-priority queues are forwarded later, ensuring that high-priority audio and video data (such as voice) achieves lower latency and higher stability during transmission.

[0097] Continuing with the above embodiment, the packet filtering rules of the optimal UPF node (Node X) allow voice and video packets of the real-time transmission protocol to pass through. The audio and video communication device controls Node X to set QoS queues: queue 1 (high priority) is used to transmit voice packets (G.711 format), and queue 2 (medium priority) is used to transmit video packets (H.264 format). Node X is configured with a data forwarding priority of "packets from queue 1 are forwarded before packets from queue 2" to ensure that voice data is transmitted first.

[0098] Step 30524: Based on the encryption parameters in the channel configuration instruction, establish an encrypted transmission channel between the communication terminal and the optimal UPF node, and complete the key negotiation between the communication terminal and the optimal UPF node based on the encrypted transmission channel.

[0099] Furthermore, the audio / video communication device extracts encryption parameters from the channel configuration instructions. These parameters include encryption algorithms (such as Advanced Encryption Standard) and key lengths (such as 256 bits). Further, based on these encryption parameters, the device triggers the communication terminal to establish an encrypted transmission channel with the optimal UPF node: both the communication terminal and the optimal UPF node enable the specified encryption algorithm and key length. Subsequently, key negotiation is performed through this encrypted transmission channel. The negotiation process is as follows: the communication terminal generates a random key, encrypts it using the optimal UPF node's public key, and sends it to the node. The node decrypts the key using its own private key. After both parties confirm the key consistency, they use this key as the encryption key for subsequent data transmission, ensuring that audio / video data is encrypted during transmission and preventing information leakage.

[0100] Continuing with the above embodiment, the encryption parameters in the channel configuration command are "Advanced Encryption Standard (AES-256) encryption algorithm, 256-bit key length". The audio / video communication device controls the communication terminal to enable the AES-256 encryption algorithm and establish an encrypted transmission channel with the optimal UPF node (node ​​X). The communication terminal generates a random 256-bit key "K1234567890ABCDEF", encrypts it using node X's public key, and sends it. Node X decrypts the key using its private key, and both parties confirm their agreement, completing the key negotiation.

[0101] Step 30525: Complete the parameter configuration of the communication terminal and the optimal UPF node based on the feedback information of the parameter configuration.

[0102] Furthermore, after the communication terminal completes parameter configuration (modem and audio / video processing module settings), and the optimal UPF node completes packet filtering rule configuration, QoS queue and forwarding priority configuration, encrypted transmission channel establishment, and key negotiation, both return parameter configuration completion feedback information (such as "modem configuration complete," "UPF queue configuration complete," etc.) to the audio / video communication device. After the audio / video communication device receives and confirms that all feedback information indicates successful configuration, it determines that the parameter configuration of the communication terminal and the optimal UPF node is complete. Continuing with the above embodiment, the communication terminal returns feedback information: "Modem transmit power and modulation method configuration complete; audio / video processing module encoding parameter configuration complete"; the optimal UPF node (node ​​X) returns feedback information: "Packet filtering rule configuration complete; QoS queue and forwarding priority configuration complete; encrypted channel establishment and key negotiation complete." The audio / video communication device confirms that all feedback is successful and determines that the parameter configuration is complete.

[0103] The embodiments of the present invention can comprehensively complete the parameter configuration of the communication terminal and the optimal UPF node, including signal transmission parameters, audio and video encoding parameters, data filtering rules, QoS priority and encryption transmission mechanism, etc., ensuring that the first audio and video transmission channel meets the preset requirements in terms of signal quality, data format, transmission efficiency and security, providing comprehensive parameter protection for the establishment of subsequent transmission links and stable transmission of audio and video data, and improving the stability of multi-party audio and video communication.

[0104] In one embodiment, steps 306 to 309 include: Step 306: Based on the connection node between the communication terminal and the VoIP network, obtain the Session Initiation Protocol (SIP) server address of the VoIP network, and send an audio / video channel establishment request to the corresponding SIP server based on the SIP server address.

[0105] Optionally, the audio / video communication device confirms that the communication terminal is connected to the VoIP network and obtains the connection node information corresponding to the connection (such as the access gateway address of the VoIP network). Further, based on the connection node information, the audio / video communication device obtains the address information (including IP address and port number) of the Session Initiation Protocol (SIP) server from the control plane of the VoIP network. This server is responsible for establishing, modifying, and terminating VoIP sessions.

[0106] Furthermore, the audio and video communication device sends a request to establish a second audio and video transmission channel to the corresponding SIP server based on the SIP server address. The request includes the VoIP account of the communication terminal, the target communication object information (such as the other party's VoIP account), and the type of audio and video service to be established (such as two-way real-time audio and video), so as to trigger the SIP server to start the channel establishment process.

[0107] In one embodiment, the communication terminal is connected to a VoIP network connection node (access gateway address "172.16.0.1"), and obtains the SIP server address "172.16.0.10:5060" from this node. The audio / video communication device sends an audio / video channel establishment request to the SIP server. The request includes information such as the communication terminal's VoIP account "user123@voipserver.com", the target account "user456@voipserver.com", and the service type "two-way real-time audio / video call", requesting the establishment of a second audio / video transmission channel.

[0108] Step 307: Based on the feedback information from the SIP server regarding the audio / video channel establishment request, obtain the media negotiation parameters required for channel establishment, and determine the audio / video encoding formats and transmission protocol types supported by the audio / video transmission channel based on the media negotiation parameters.

[0109] Furthermore, the audio and video communication device receives feedback information from the SIP server regarding the channel establishment request. This feedback information includes media negotiation parameters required for channel establishment, such as the audio and video encoding formats supported by the SIP server (e.g., G.711 and G.729 for voice, and H.264 and H.265 for video), transmission protocol types (e.g., Real-time Transport Protocol, Real-time Transport Control Protocol), and media port range.

[0110] Furthermore, the audio and video communication device compares these media negotiation parameters with the audio and video encoding formats and transmission protocol types supported by the communication terminal, determines the common intersection of the two, and uses it as the final audio and video encoding formats and transmission protocol types supported by the second audio and video transmission channel to ensure that the media data between the communication terminal and the VoIP network can be transmitted compatiblely.

[0111] Continuing with the above embodiment, the media negotiation parameters returned by the SIP server are: supported voice encoding formats G.711 and G.729, video encoding formats H.264 and H.265, transmission protocol types Real-Time Transport Protocol (RTP) and Real-Time Transport Control Protocol (RTCP), and media port range 5000-6000. The communication terminal supports the encoding formats G.711 for voice and H.264 for video, and the transmission protocol is Real-Time Transport Protocol. Through comparison, it is determined that the second audio and video transmission channel supports the audio and video encoding formats "G.711 for voice and H.264 for video" and the transmission protocol type "Real-Time Transport Protocol".

[0112] Step 308: Select candidate media servers that support audio and video encoding formats and transmission protocol types from the available media servers in the VoIP network, and send a media capability detection request to each candidate server to obtain the real-time media processing capability of each candidate server based on the media capability detection request.

[0113] Furthermore, the audio and video communication device obtains a list of all available media servers (the media servers are responsible for forwarding and processing audio and video data) from the VoIP network, and filters them based on the audio and video encoding format and transmission protocol type: it checks the support capabilities of each media server one by one, and determines the media servers that support both the encoding format and transmission protocol type as candidate media servers.

[0114] Furthermore, the audio and video communication device sends a media capability detection request to each candidate media server. The request contains preset test media data (such as short audio clips or video frames) to detect the server's real-time media processing capabilities, including but not limited to the server's media data forwarding rate, encoding format conversion capability (if required), and processing latency.

[0115] Furthermore, the audio and video communication device receives the probe responses returned by the candidate media servers, and extracts and records the real-time media processing capability parameters of each server.

[0116] Continuing with the above embodiment, the available media servers in the VoIP network include server M, server N, and server P. Step 307 determines that "voice G.711, video H.264, real-time transmission protocol" must be supported. After screening, server M and server N support this requirement, while server P does not support H.264 video encoding. Therefore, the candidate media servers are M and N. The audio and video communication device sends a media capability detection request (containing a 1-second G.711 voice segment and 2 frames of H.264 video frames) to M and N. Server M returns a processing capability of "forwarding rate 2Mbps, processing latency 30ms"; server N returns a processing capability of "forwarding rate 1.5Mbps, processing latency 50ms".

[0117] Step 309: Based on the real-time media processing capabilities of each candidate server, establish a second audio and video transmission channel between the communication terminal and the VoIP network.

[0118] Furthermore, the audio and video communication device establishes a second audio and video transmission channel between the communication terminal and the VoIP network based on the real-time media processing capability of each candidate server, as described in steps 3091 to 3095.

[0119] This invention enables the establishment of a second audio and video transmission channel based on the core control node (SIP server) and media processing node (media server) of the VoIP network, combined with media negotiation and server capability screening. This channel satisfies audio and video encoding compatibility, transmission protocol matching, and processing performance standards. The second audio and video transmission channel provides a reliable link for communication terminals to transmit audio and video data in the VoIP network. Together with the first audio and video transmission channel, it forms a primary-secondary-backup transmission system, laying the foundation for ensuring communication continuity and improving the stability and continuity of multi-party audio and video communication.

[0120] In one embodiment, steps 3091 to 3095 include: Step 3091: Analyze the number of concurrent processing and the media data forwarding rate based on the real-time media processing capability of each candidate server, and rank the candidate media servers by performance based on the number of concurrent processing and the forwarding rate of each candidate server to obtain the media server performance ranking result.

[0121] Optionally, the audio / video communication device extracts the concurrent processing count (i.e., the number of audio / video sessions currently being processed simultaneously by the server) and the media data forwarding rate (i.e., the amount of audio / video data forwarded by the server per unit time) from the real-time media processing capabilities of each candidate media server. Based on these two metrics, the candidate media servers are ranked according to the following rules: Concurrent processing count is compared first; a lower count indicates a lighter server load and better performance. If the concurrent processing counts are the same, the media data forwarding rate is compared; a higher rate indicates stronger data transmission capabilities and better performance. This ranking of media server performance is then used to obtain the final media server performance ranking.

[0122] In one embodiment, the candidate media servers are server M and server N. Server M has a real-time media processing capability with a concurrent processing capacity of 50 sessions and a media data forwarding rate of 2 Mbps; server N has a concurrent processing capacity of 80 sessions and a media data forwarding rate of 1.5 Mbps. According to the rule of "lower concurrent processing capacity takes precedence, and if the number of concurrent processing sessions is the same, higher forwarding rate takes precedence," server M performs better than server N. Therefore, the media server performance ranking is: server M ranks first, and server N ranks second.

[0123] Step 3092: Determine the optimal media server based on the media server performance ranking results, and generate media connection configuration information for the second audio and video transmission channel based on the optimal media server.

[0124] Furthermore, the audio-visual communication device selects the top-ranked candidate media server as the optimal media server based on the media server performance ranking results. Further, the audio-visual communication device generates media connection configuration information for the second audio-visual transmission channel based on the optimal media server. This media connection configuration information includes the address (IP address and port number) of the optimal media server, the media transmission path between the communication terminal and the server (such as the sequence of intermediate nodes like routers and switches), and the port allocation for audio and video data transmission (voice port and video port), ensuring that the connection path and port information between the communication terminal and the optimal media server are clearly defined. Continuing with the above embodiment, the top-ranked media server in the performance ranking results is server M (IP address "172.16.1.10", port number 5060). The audio-visual communication device determines server M as the optimal media server and generates the following media connection configuration information: optimal media server address "172.16.1.10:5060", transmission path "communication terminal → router A → server M", voice transmission port 5001, and video transmission port 5002.

[0125] Step 3093: Based on the media connection configuration information, send media parameter configuration instructions to the communication terminal and the optimal media server respectively, so as to complete the media parameter settings of the communication terminal and the optimal media server based on the media parameter configuration instructions.

[0126] Furthermore, the audio and video communication device sends media parameter configuration instructions to the communication terminal and the optimal media server respectively based on the media connection configuration information. The instructions sent to the communication terminal include the address of the optimal media server to be connected to, the corresponding voice and video transmission ports, the audio and video encoding formats used (such as voice G.711 and video H.264 determined in step 307), and the transmission protocol (such as real-time transmission protocol). The instructions sent to the optimal media server include the address of the communication terminal to receive the instructions, the corresponding voice and video transmission ports, the encoding format decoding rules, and the transmission protocol adaptation parameters.

[0127] Furthermore, after receiving the instruction, the communication terminal and the optimal media server complete their own media parameter settings according to the instruction content, as detailed in steps 30931 to 30934.

[0128] Continuing with the above embodiment, the audio and video communication device sends a media parameter configuration instruction to the communication terminal: "Connection address 172.16.1.10:5060, voice port 5001, video port 5002, encoding format: voice G.711, video H.264, transmission protocol: Real-time transmission protocol"; and sends an instruction to the optimal media server (server M): "Receiving address (communication terminal IP), voice port 5001, video port 5002, decoding format: voice G.711, video H.264, adapted to Real-time transmission protocol". The communication terminal and server M respectively complete the media parameter settings according to the instructions.

[0129] Step 3094: Establish a media transmission link between the communication terminal and the optimal media server based on the configured media parameters, and test whether the bidirectional transmission of audio and video data is normal based on the media transmission link.

[0130] Furthermore, after the communication terminal and the optimal media server complete the media parameter settings, the audio and video communication device triggers the two to establish a media transmission link based on the set parameters: the communication terminal sends a media connection request through the configured optimal media server address and port, which is routed through intermediate nodes in the transmission path to reach the optimal media server. The server receives and responds to the request, forming a bidirectional media data transmission link.

[0131] Furthermore, after the link is established, the audio and video communication device controls the communication terminal to send test audio and video data (such as a voice and video clip) to the optimal media server, and simultaneously controls the optimal media server to return a test audio and video data to the communication terminal, thereby testing whether the bidirectional transmission is normal (such as whether the data is complete, distortion-free, and the transmission delay is within a reasonable range). Continuing the above embodiment, the communication terminal and the optimal media server (server M) establish a media transmission link based on configuration parameters: the communication terminal sends a connection request to server M (172.16.1.10:5060) through router A, and the link is established after server M responds. The audio and video communication device controls the communication terminal to send 10 seconds of G.711 voice and 5 frames of H.264 video to server M. After receiving the data, server M returns the same test data. The communication terminal successfully receives the data, and the data is complete and distortion-free, with a transmission delay of 40 milliseconds. The test result is that the bidirectional transmission is normal.

[0132] Step 3095: If normal, then the second audio / video transmission channel has been successfully established.

[0133] Furthermore, when the bidirectional transmission of the test audio and video data is normal (i.e., the test data is transmitted completely bidirectionally between the communication terminal and the optimal media server without obvious errors or distortion, and the transmission delay and packet loss rate meet expectations), the audio and video communication device determines that the second audio and video transmission channel between the communication terminal and the VoIP network has been successfully established, and this channel can be used for the subsequent formal bidirectional transmission of audio and video data.

[0134] Continuing with the above embodiment, the bidirectional transmission test in step 3094 shows that both voice and video data between the communication terminal and server M are received completely without distortion, with a transmission latency of 40 milliseconds and a packet loss rate of 0%, meeting the expected requirements. Therefore, the audio and video communication device determines that the second audio and video transmission channel has been successfully established.

[0135] The embodiments of the present invention can select the optimal node based on the performance of candidate media servers, complete media parameter configuration, link establishment and bidirectional transmission verification, and finally establish a stable second audio and video transmission channel, ensuring smooth bidirectional transmission of audio and video data between the communication terminal and the VoIP network. It complements the first audio and video transmission channel, provides reliable link support for subsequent switching between primary and secondary channels and ensuring the continuity of audio and video communication, and improves the stability and continuity of multi-party audio and video communication.

[0136] In one embodiment, steps 30931 to 30934 include: Step 30931: Based on the media parameter configuration command, send parameter setting requests and protocol parameters to the media acquisition module and network transmission module of the communication terminal respectively, so as to configure the audio sampling rate and video resolution based on the parameter setting request, and set the reporting period of the real-time transmission control protocol RTCP based on the protocol parameters.

[0137] Optionally, the audio / video communication device extracts parameter setting information for the communication terminal from the media parameter configuration command, including the audio sampling rate, video resolution, and the Real-Time Transmission Control Protocol (RTCP) reporting period. First, a parameter setting request is sent to the media acquisition module of the communication terminal. This request includes specific audio sampling rates (e.g., 8kHz, 16kHz, used to determine the audio signal acquisition accuracy) and video resolutions (e.g., 720*480 pixels, 1920*1080 pixels, used to determine the video image clarity). After receiving the request, the media acquisition module adjusts the audio sampling rate and video resolution to the specified values ​​to ensure that the acquired audio and video data meets transmission requirements. Simultaneously, protocol parameters are sent to the network transmission module of the communication terminal. These parameters include the RTCP reporting period (e.g., 5 seconds, 10 seconds, used to specify the interval for sending transmission quality reports). The network transmission module sets the RTCP reporting period based on these parameters.

[0138] In one embodiment, the media parameter configuration instruction requests the following parameter settings for the communication terminal: "audio sampling rate 16kHz, video resolution 1280*720 pixels"; the protocol parameter is "RTCP reporting period 5 seconds". The audio and video communication device sends the above parameter setting request to the media acquisition module, which adjusts the audio sampling rate to 16kHz and the video resolution to 1280*720 pixels; it also sends the protocol parameters to the network transmission module, which sets RTCP to send a transmission quality report every 5 seconds.

[0139] Step 30932: Send media receiving parameters to the optimal media server based on the media parameter configuration instruction, so as to configure the media stream caching strategy of the optimal media server based on the receiving parameters.

[0140] Furthermore, the audio and video communication device extracts media reception parameters from the media parameter configuration instructions. These parameters include the type of media stream (such as voice stream, video stream), data encapsulation format (such as encapsulation format based on real-time transmission protocol), maximum allowable data packet delay time, etc.

[0141] Furthermore, the audio and video communication device sends these media reception parameters to the optimal media server. The optimal media server configures a media stream caching strategy based on these parameters: different caching processing methods are adopted for different types of media streams (such as a shorter caching time for audio streams to reduce latency and a slightly longer caching time for video streams to cope with jitter), and the maximum storage time of cached data is limited according to the maximum allowable latency time, so as to ensure that the received media data can be played smoothly without the real-time performance degraded due to excessive caching.

[0142] Continuing with the above embodiment, the media receiving parameters in the media parameter configuration instruction are: "Media stream type: audio stream (G.711), video stream (H.264); data encapsulation format: real-time transport protocol; maximum allowable data packet delay time: 30 milliseconds for audio stream, 100 milliseconds for video stream". These parameters are sent to the optimal media server (server M). Server M configures a caching strategy based on these parameters: audio stream caching time ≤ 30 milliseconds, video stream caching time ≤ 100 milliseconds, performing targeted caching processing for the two types of media streams respectively.

[0143] Step 30933: Set the buffer size of the optimal media server based on the media stream caching strategy, and establish a clock synchronization mechanism between the communication terminal and the optimal media server based on the media clock synchronization parameters in the media parameter configuration instruction. Configure the data buffer duration based on the buffer size, and calibrate the media timestamps of the communication terminal and the optimal media server based on the clock synchronization mechanism.

[0144] Furthermore, based on the media stream caching strategy configured by the optimal media server, the audio and video communication device further controls the buffer size of the optimal media server. The buffer size must meet the maximum buffered data volume specified in the caching strategy (e.g., calculated based on the maximum allowable delay time and media data rate; voice stream buffer size = voice data rate * 30 milliseconds, video stream buffer size = video data rate * 100 milliseconds), ensuring that the buffer can accommodate media data within the specified duration. Simultaneously, media clock synchronization parameters (such as timestamp reference and synchronization accuracy requirements) are extracted from the media parameter configuration instructions. Based on these parameters, a clock synchronization mechanism is established between the communication terminal and the optimal media server: by sending and receiving time synchronization messages (such as Network Time Protocol messages), the system clocks of both are kept consistent. The corresponding data buffering duration (i.e., the maximum delay time specified in the caching strategy) is configured based on the buffer size. The timestamp of the media data sent by the communication terminal is calibrated with the local timestamp received by the optimal media server based on the clock synchronization mechanism, ensuring the time consistency of audio and video data and avoiding audio-visual asynchrony during playback.

[0145] Continuing with the above embodiment, the optimal media server (server M)'s media stream caching strategy requires a maximum audio stream buffer of 30 milliseconds and a maximum video stream buffer of 100 milliseconds. The audio data rate is 64kbps (8KB / s), and the video data rate is 2Mbps (250KB / s). Therefore, the audio stream buffer size = 8KB / s * 0.03s = 0.24KB, and the video stream buffer size = 250KB / s * 0.1s = 25KB. Server M sets the buffer size accordingly. The media clock synchronization parameter is "synchronization accuracy ±10 milliseconds". The audio and video communication device controls the communication terminal to synchronize the clock with server M through Network Time Protocol (NTP) messages. After calibration, the time difference between the two is ≤10 milliseconds, ensuring that the media data timestamps are consistent.

[0146] Step 30934: Based on the parameter configuration feedback information, complete the bidirectional media parameter synchronization between the communication terminal and the optimal media server.

[0147] Furthermore, once the communication terminal completes parameter settings (audio sampling rate and video resolution of the media acquisition module, RTCP reporting period of the network transmission module), and the optimal media server completes media stream caching strategy configuration, buffer size setting, and clock synchronization mechanism establishment, both return parameter configuration completion feedback information to the audio-visual communication device (such as "media acquisition parameter configuration complete" or "server buffer setting complete"). After receiving and confirming that all feedback information indicates successful configuration, the audio-visual communication device determines that the bidirectional media parameters between the communication terminal and the optimal media server are synchronized (i.e., the encoding format, transmission protocol, caching strategy, clock, and other parameters of both parties are consistent), thus completing the media parameter setting.

[0148] Continuing with the above embodiment, the communication terminal returns feedback information: "Audio sampling rate of 16kHz and video resolution of 1280*720 pixels configured; RTCP reporting period of 5 seconds set." The optimal media server (server M) returns feedback information: "Media stream caching strategy configured; buffer size set; clock synchronization mechanism established, time difference 5 milliseconds." The audio and video communication device confirms that all feedback is successful and determines that the two-way media parameter synchronization between the two parties is complete.

[0149] The embodiments of this invention can comprehensively complete the media parameter settings and synchronization between the communication terminal and the optimal media server, including key aspects such as audio and video acquisition parameters, transmission protocol parameters, caching strategies, and clock synchronization. This ensures that the acquisition, transmission, reception, and processing of media data in the second audio and video transmission channel are mutually matched, effectively reducing the risks of data distortion, excessive latency, or audio-visual asynchrony. It provides core parameter guarantees for the stable operation of subsequent media transmission links and the high-quality transmission of audio and video data, and improves the stability and continuity of multi-party audio and video communication.

[0150] Furthermore, the multi-party audio and video communication device based on the convergence of VoLTE network and VoIP provided by the present invention will be described below. The multi-party audio and video communication device based on the convergence of VoLTE network and VoIP described below can be referred to in correspondence with the multi-party audio and video communication method based on the convergence of VoLTE network and VoIP described above.

[0151] Optional, refer to Figure 2 , Figure 2 This is a schematic diagram of the structure of the multi-party audio and video communication device based on VoLTE network and VoIP integration provided by the present invention. The multi-party audio and video communication device includes: The single network access module 210 is used to connect the communication terminal to the VoLTE network if the network environment parameters of the network environment where the communication terminal is located determine that it is within the coverage area of ​​the VoLTE network, and to determine whether the communication quality of the VoLTE network meets the preset communication quality standard based on the communication link status of the VoLTE network. The dual network access module 220 is used to activate the VoIP network if the conditions are not met, and connect the communication terminal to the VoIP network based on the available connection information of the VoIP network. The transmission channel construction module 230 is used to establish a first audio and video transmission channel and a second audio and video transmission channel between the communication terminal and the VoLTE network and the VoIP network respectively, and to determine the main transmission channel and the auxiliary transmission channel based on the audio and video transmission status of the first audio and video transmission channel and the second audio and video transmission channel. The transmission channel switching module 240 is used to transmit audio and video data of the communication terminal based on the main transmission channel, and to switch to the auxiliary transmission channel to transmit audio and video data when the main transmission channel is abnormal.

[0152] The embodiments of the present invention improve the stability and continuity of multi-party audio and video communication.

[0153] Please see Figure 3 , Figure 3 An embodiment diagram of an electronic device provided in accordance with the present invention. For example... Figure 3 As shown, this embodiment of the invention provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, it implements the processes of steps 10 to 40: If the network environment parameters of the network environment where the communication terminal is located determine that it is within the coverage area of ​​the VoLTE network, then the communication terminal is connected to the VoLTE network, and the communication quality of the VoLTE network is determined to meet the preset communication quality standard based on the communication link status of the VoLTE network. If the conditions are not met, the VoIP network is activated, and the communication terminal is connected to the VoIP network based on the available connection information of the VoIP network. A first audio and video transmission channel and a second audio and video transmission channel are established between the communication terminal and the VoLTE network and the VoIP network respectively, and a main transmission channel and an auxiliary transmission channel are determined based on the audio and video transmission status of the first audio and video transmission channel and the second audio and video transmission channel. The communication terminal transmits audio and video data through the main transmission channel. When the main transmission channel malfunctions, the transmission is switched to the auxiliary transmission channel to transmit the audio and video data.

[0154] Please see Figure 4 , Figure 4 An embodiment diagram of a computer-readable storage medium provided in accordance with an embodiment of the present invention is shown. Figure 4 As shown, this embodiment provides a computer-readable storage medium 400 on which a computer program 311 is stored. When the computer program 311 is executed by a processor, it implements the processes of steps 10 to 40. If the network environment parameters of the network environment where the communication terminal is located determine that it is within the coverage area of ​​the VoLTE network, then the communication terminal is connected to the VoLTE network, and the communication quality of the VoLTE network is determined to meet the preset communication quality standard based on the communication link status of the VoLTE network. If the conditions are not met, the VoIP network is activated, and the communication terminal is connected to the VoIP network based on the available connection information of the VoIP network. A first audio and video transmission channel and a second audio and video transmission channel are established between the communication terminal and the VoLTE network and the VoIP network respectively, and a main transmission channel and an auxiliary transmission channel are determined based on the audio and video transmission status of the first audio and video transmission channel and the second audio and video transmission channel. The communication terminal transmits audio and video data through the main transmission channel. When the main transmission channel malfunctions, the transmission is switched to the auxiliary transmission channel to transmit the audio and video data.

[0155] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the multi-party audio and video communication method based on VoLTE network and VoIP convergence provided by the above methods, which includes steps 10 to 40: If the network environment parameters of the network environment where the communication terminal is located determine that it is within the coverage area of ​​the VoLTE network, then the communication terminal is connected to the VoLTE network, and the communication quality of the VoLTE network is determined to meet the preset communication quality standard based on the communication link status of the VoLTE network. If the conditions are not met, the VoIP network is activated, and the communication terminal is connected to the VoIP network based on the available connection information of the VoIP network. A first audio and video transmission channel and a second audio and video transmission channel are established between the communication terminal and the VoLTE network and the VoIP network respectively, and a main transmission channel and an auxiliary transmission channel are determined based on the audio and video transmission status of the first audio and video transmission channel and the second audio and video transmission channel. The communication terminal transmits audio and video data through the main transmission channel. When the main transmission channel malfunctions, the transmission is switched to the auxiliary transmission channel to transmit the audio and video data.

[0156] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0157] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-party audio and video communication method based on VoLTE network and VoIP integration, characterized in that, include: If the network environment parameters of the network environment where the communication terminal is located determine that it is within the coverage area of ​​the VoLTE network, then the communication terminal is connected to the VoLTE network, and the communication quality of the VoLTE network is determined to meet the preset communication quality standard based on the communication link status of the VoLTE network. If the conditions are not met, the VoIP network is activated, and the communication terminal is connected to the VoIP network based on the available connection information of the VoIP network. A first audio and video transmission channel and a second audio and video transmission channel are established between the communication terminal and the VoLTE network and the VoIP network, respectively. Based on the audio and video transmission status of the first audio and video transmission channel and the second audio and video transmission channel, a main transmission channel and an auxiliary transmission channel are determined. The communication terminal transmits audio and video data through the main transmission channel. When the main transmission channel malfunctions, the transmission is switched to the auxiliary transmission channel to transmit the audio and video data.

2. The multi-party audio and video communication method based on VoLTE network and VoIP integration according to claim 1, characterized in that, The steps for establishing the first audio and video transmission channel between the communication terminal and the VoLTE network include: Based on the connection status between the communication terminal and the VoLTE network, the session management function (SMF) node address of the VoLTE network is obtained, and a channel establishment request is sent to the corresponding SMF node based on the SMF node address. Based on the response information of the SMF node to the channel establishment request, the Quality of Service (QoS) parameters required for audio and video transmission are obtained, and candidate UPF nodes that meet the QoS parameters are selected from the user plane function (UPF) nodes available in the VoLTE network. Based on the candidate UPF node, a path probing request is sent to each candidate node to obtain the transmission path information of each transmission path between each candidate node and the communication terminal; Based on the transmission path information, the hop count and link stability of each transmission path are analyzed, and the candidate UPF nodes are prioritized based on the hop count and link stability of each transmission path. The first audio and video transmission channel between the communication terminal and the VoLTE network is established based on the UPF node priority ranking result.

3. The multi-party audio and video communication method based on VoLTE network and VoIP integration according to claim 2, characterized in that, The establishment of the first audio and video transmission channel between the communication terminal and the VoLTE network based on the UPF node priority ranking result includes: The optimal UPF node is determined based on the UPF node priority ranking result, and the path configuration information of the first audio and video transmission channel is generated based on the optimal UPF node; the path configuration information includes the node sequence and port information of the transmission link; Based on the path configuration information, channel configuration instructions are sent to the communication terminal and the optimal UPF node respectively, so as to complete the parameter configuration of the communication terminal and the optimal UPF node based on the channel configuration instructions; Based on the configured parameters, a transmission link is established between the communication terminal and the optimal UPF node, and the transmission link is used to verify whether the transmission of audio and video data is normal. If everything is normal, then the first audio / video transmission channel has been successfully established.

4. The multi-party audio and video communication method based on VoLTE network and VoIP integration according to claim 3, characterized in that, The parameter configuration of the communication terminal and the optimal UPF node based on the channel configuration instructions includes: Based on the channel configuration command, parameter setting requests and encoding parameters are sent to the modem and audio / video processing module of the communication terminal, respectively, so as to configure the transmission power and signal modulation method of the communication terminal based on the parameter setting request, and to set the compression format and frame rate of audio and video data based on the encoding parameters. Based on the channel configuration command, session establishment parameters are sent to the optimal UPF node to configure the packet filtering rules of the optimal UPF node based on the session establishment parameters; Based on the packet filtering rules, the QoS queue of the optimal UPF node is set, and the data forwarding priority is configured based on the QoS queue; Based on the encryption parameters in the channel configuration instruction, an encrypted transmission channel is established between the communication terminal and the optimal UPF node, and key negotiation between the communication terminal and the optimal UPF node is completed based on the encrypted transmission channel. Based on the feedback information after parameter configuration, the parameter configuration of the communication terminal and the optimal UPF node is completed.

5. The multi-party audio and video communication method based on VoLTE network and VoIP integration according to claim 1, characterized in that, The steps for establishing a second audio / video transmission channel between the communication terminal and the VoIP network include: Based on the connection node between the communication terminal and the VoIP network, the Session Initiation Protocol (SIP) server address of the VoIP network is obtained, and an audio / video channel establishment request is sent to the corresponding SIP server based on the SIP server address. Based on the feedback information from the SIP server regarding the request to establish the audio and video channel, the media negotiation parameters required for channel establishment are obtained, and the audio and video encoding formats and transmission protocol types supported by the audio and video transmission channel are determined based on the media negotiation parameters. Candidate media servers that support the audio and video encoding format and the transmission protocol type are selected from the media servers available in the VoIP network. Media capability detection requests are sent to each candidate server based on the candidate media servers to obtain the real-time media processing capability of each candidate server based on the media capability detection requests. Based on the real-time media processing capabilities of each candidate server, a second audio and video transmission channel is established between the communication terminal and the VoIP network.

6. The multi-party audio and video communication method based on VoLTE network and VoIP integration according to claim 5, characterized in that, The establishment of a second audio and video transmission channel between the communication terminal and the VoIP network based on the real-time media processing capabilities of each candidate server includes: The concurrent processing capacity and media data forwarding rate of each candidate server are analyzed based on their real-time media processing capabilities. The candidate media servers are then ranked by performance based on their concurrent processing capacity and forwarding rate to obtain the media server performance ranking results. The optimal media server is determined based on the media server performance ranking results, and media connection configuration information for the second audio and video transmission channel is generated based on the optimal media server. Based on the media connection configuration information, media parameter configuration instructions are sent to the communication terminal and the optimal media server respectively, so as to complete the media parameter settings of the communication terminal and the optimal media server based on the media parameter configuration instructions; Based on the configured media parameters, a media transmission link is established between the communication terminal and the optimal media server, and the bidirectional transmission of audio and video data is tested based on the media transmission link to determine if it is normal. If everything is normal, then the second audio / video transmission channel has been successfully established.

7. The multi-party audio and video communication method based on VoLTE network and VoIP integration according to claim 6, characterized in that, The process of setting media parameters for the communication terminal and the optimal media server based on the media parameter configuration instructions includes: Based on the media parameter configuration command, parameter setting requests and protocol parameters are sent to the media acquisition module and network transmission module of the communication terminal, respectively, so as to configure the audio sampling rate and video resolution based on the parameter setting request, and to set the reporting period of the real-time transmission control protocol RTCP based on the protocol parameters. Based on the media parameter configuration instruction, media receiving parameters are sent to the optimal media server to configure the media stream caching strategy of the optimal media server based on the receiving parameters; Based on the media stream caching strategy, the buffer size of the optimal media server is set, and based on the media clock synchronization parameter in the media parameter configuration instruction, a clock synchronization mechanism is established between the communication terminal and the optimal media server to configure the data buffer duration based on the buffer size and to calibrate the media timestamps of the communication terminal and the optimal media server based on the clock synchronization mechanism. Based on the parameter configuration feedback information, the bidirectional media parameter synchronization between the communication terminal and the optimal media server is completed.

8. The multi-party audio and video communication method based on VoLTE network and VoIP convergence according to any one of claims 1 to 7, characterized in that, Connecting the communication terminal to the VoIP network based on the available connection information of the VoIP network includes: Based on the available connection information, candidate access nodes that meet the connection stability requirements in the VoIP network are selected, and target access nodes that are compatible with the network operator of the VoLTE network are determined based on the network operator information of each candidate access node. The load rate is calculated based on the real-time network load data of each target access node, and priority access nodes with load rates lower than the preset load threshold are selected from the target access nodes. The priority access nodes are sorted based on the network latency parameters of each priority access node to obtain the sorted priority access nodes, and the first-ranked priority access node is selected as the final access node. Based on the final access node, a connection request is generated between the communication terminal and the VoIP network; Based on the connection request, an access application is sent to the final access node, and based on the feedback information from the final access node, it is determined whether to allow the communication terminal to access the VoIP network; If access is permitted, the connection between the communication terminal and the VoIP network is completed based on the access configuration parameters of the final access node.

9. The multi-party audio and video communication method based on VoLTE network and VoIP integration according to claim 8, characterized in that, The real-time network load data includes the number of currently connected users, the current data transmission rate, and the maximum supported transmission rate. The calculation of load rate based on real-time network load data for each target access node includes: Calculate the user load percentage based on the current number of connected users and the maximum number of users that each target access node can support; Calculate the data load percentage based on the current data transmission rate and the maximum supported transmission rate of each target access node. The load rate of each target access node is obtained by calculating the load ratio based on the user load ratio and data load ratio of each target access node and the corresponding calculation weight.

10. A multi-party audio and video communication device based on VoLTE network and VoIP convergence, characterized in that, The multi-party audio and video communication device is used for: If the network environment parameters of the network environment where the communication terminal is located determine that it is within the coverage area of ​​the VoLTE network, then the communication terminal is connected to the VoLTE network, and the communication quality of the VoLTE network is determined to meet the preset communication quality standard based on the communication link status of the VoLTE network. If the conditions are not met, the VoIP network is activated, and the communication terminal is connected to the VoIP network based on the available connection information of the VoIP network. A first audio and video transmission channel and a second audio and video transmission channel are established between the communication terminal and the VoLTE network and the VoIP network, respectively. Based on the audio and video transmission status of the first audio and video transmission channel and the second audio and video transmission channel, a main transmission channel and an auxiliary transmission channel are determined. The communication terminal transmits audio and video data through the main transmission channel. When the main transmission channel malfunctions, the transmission is switched to the auxiliary transmission channel to transmit the audio and video data.

Images