Converged communication gateway for weak network processing, audio and video communication method and system

Abstract

The invention discloses a converged communication gateway for weak network processing, an audio and video communication method and an audio and video communication system, and provides the converged communication gateway for weak network processing, which realizes dynamic processing of weak network conditions in a multi-network aggregation environment by combining an RTC (Real-time Communication) protocol, an SIP (Session Initiation Protocol) signaling mechanism and a QoS (Quality of Service) optimization strategy. According to the method and the device, the request of packet loss retransmission or frame rate / resolution reduction can be automatically initiated according to the real-time monitoring result of the network quality, and the audio and video streams are recoded and transmitted, so that the real-time performance and the stability of communication are effectively improved in a weak network environment. In addition, the weak network processing gateway can dynamically adjust the transmission parameters according to the recovery of the network condition, thereby achieving the intelligent optimization and management of the communication quality.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a fusion communication gateway for weak network processing, an audio and video communication method and a system. Background Art

[0002] With the rapid development of network technology, the demand for data communication in various application scenarios is growing. Especially in multi-network aggregation environments, data transmission rate and stability have become important criteria for user experience. Multi-network aggregation technology refers to the superposition of the bandwidth of multiple network links to achieve higher transmission rates and more stable network connections. In application scenarios such as video conferencing, real-time voice calls, and large-scale data transmission, higher requirements are placed on the real-time, stability, and high quality of network transmission. However, in complex network environments, network quality instability often occurs. Especially in weak network environments, network packet loss, delay, and jitter are more prominent, resulting in a significant decrease in communication quality, which seriously affects the actual application effect.

[0003] At present, the weak network problem that occurs during network transmission mainly relies on traditional retransmission mechanisms and bandwidth optimization strategies. For example, existing video conferencing systems and real-time communication systems use the SIP (Session Initialization Protocol) protocol for call connections and the RTP (Real-time Transport Protocol) protocol for audio and video stream transmission. However, in a weak network environment, due to high packet loss rate, large transmission delay and severe jitter, the traditional network retransmission mechanism often cannot meet the high timeliness requirements of real-time communication. In addition, when facing a sudden decrease in network quality, the existing technology lacks effective dynamic adaptation means and cannot adjust the transmission parameters in time, resulting in a further decrease in communication quality. Therefore, how to improve the real-time and stability of audio and video data transmission in a weak network environment has become a problem that needs to be solved urgently in the current technical field.

[0004] Therefore, in audio and video communications in a multi-network aggregation environment, the network quality changes significantly dynamically, the traditional retransmission mechanism is not timely enough, and the transmission parameter adjustment response is slow, which have become problems that need to be solved urgently. Summary of the invention

[0005] The present application provides a fusion communication gateway, audio and video communication method and system for weak network processing, aiming to solve the problems in the above-mentioned background technology.

[0006] In a first aspect, a converged communication gateway for weak network processing is provided, wherein the gateway comprises:

[0007] The signaling module is used to provide SIP and RTC standard protocols, where SIP is used for call connection between external devices and RTC is used for call connection between gateways;

[0008] Media transmission module, used for transmission and control of audio and video media data through RTP and RTCP;

[0009] The service quality module is used to monitor the network transmission quality and process according to the packet loss rate and jitter in a weak network environment. The processing includes:

[0010] When the packet loss rate is less than or equal to 30% and the jitter is less than or equal to 200ms, a packet loss retransmission request is initiated;

[0011] When the packet loss rate is greater than 30% or the jitter is greater than 200ms, the frame rate and resolution of the transmission are reduced and the RTP stream is re-encoded.

[0012] In the above solution, optionally, the service quality module includes:

[0013] A packet loss detection unit, used to detect the packet loss rate in network transmission;

[0014] A jitter detection unit, used to detect jitter in network transmission;

[0015] The bandwidth adjustment unit is used to dynamically adjust the transmission bandwidth according to the detected network transmission status.

[0016] In the above solution, optionally, the signaling module establishes a call connection with an external conference system through SIPTrunk, and establishes a communication connection with other converged communication gateways for weak network processing through RTC protocol.

[0017] In the above solution, optionally, the media transmission module is used to receive audio and video RTP streams, and transmit and forward the RTP streams between multiple converged communication gateways for weak network processing.

[0018] In the above solution, optionally, when the service quality module detects that the network transmission condition returns to normal, it initiates a request to increase the frame rate and resolution, and re-encodes the RTP stream.

[0019] In a second aspect, an audio and video communication method is provided, wherein the method comprises the following steps:

[0020] Through the signaling module, use the SIP protocol to connect calls with external devices, and use the RTC protocol to establish calls between converged communication gateways that handle weak networks;

[0021] Through the media transmission module, audio and video data are transmitted based on the RTP protocol;

[0022] Monitor network transmission quality through the service quality module, including detecting network packet loss rate and jitter;

[0023] According to the monitoring results:

[0024] When the packet loss rate is less than or equal to 30% and the jitter is less than or equal to 200ms, a packet loss retransmission request is initiated;

[0025] When the packet loss rate is greater than 30% or the jitter is greater than 200ms, the frame rate and resolution of the transmission are reduced and the RTP stream is re-encoded.

[0026] In the above solution, optionally, when it is detected that the network transmission condition returns to normal, a request to increase the frame rate and resolution is initiated, and the RTP stream is re-encoded.

[0027] In the above solution, optionally, the step of monitoring the network transmission quality includes:

[0028] Detect the packet loss rate of network transmission;

[0029] Detect the jitter of network transmission.

[0030] In a third aspect, an audio and video communication system is provided, wherein the system comprises:

[0031] Video conferencing system;

[0032] At least two converged communication gateways for weak network processing according to claim 1;

[0033] The terminal device transmits audio and video data between the video conferencing system and the terminal device through the fusion communication gateway processed by the weak network.

[0034] In the above solution, optionally, the converged communication gateways for weak network processing perform call connection through RTC protocol.

[0035] Compared with the prior art, this application has at least the following beneficial effects:

[0036] This application is based on further analysis and research of the existing technical problems. This application provides a fusion communication gateway for weak network processing. By combining the RTC (real-time communication) protocol, SIP signaling mechanism and QoS (quality of service) optimization strategy, dynamic processing of weak network conditions is achieved in a multi-network aggregation environment. This application can automatically initiate packet retransmission or frame rate / resolution reduction requests based on the real-time monitoring results of network quality, and re-encode and transmit audio and video streams, thereby effectively improving the real-time and stability of communication in a weak network environment. In addition, the weak network processing gateway of this application can also dynamically adjust transmission parameters according to the recovery of network conditions, and realize intelligent optimization and management of communication quality.

[0037] The fusion communication gateway for weak network processing of the present invention solves the problems of obvious dynamic changes in network quality, insufficient timeliness of retransmission mechanism and influence of video quality on transmission parameter adjustment in the background technology through the collaborative work of signaling module, media transmission module and quality of service (QoS) module. The signaling module realizes efficient call connection between external devices and gateways and between gateways through SIP and RTC protocols, reduces the amount of signaling data, reduces delay and ensures rapid establishment of calls. The media transmission module adopts RTP and RTCP protocols to transmit and control audio and video streams to ensure the real-time and controllability of data transmission. The quality of service module adopts dynamic processing strategies for different network states by real-time monitoring of network packet loss rate and jitter: when the network is slightly weakened, a packet loss retransmission request is initiated to supplement the lost data; when the network deteriorates seriously, the frame rate and resolution are reduced and the RTP stream is re-encoded to reduce bandwidth occupancy and ensure smooth transmission of audio and video data. In addition, when the network returns to normal, the system automatically increases the frame rate and resolution to restore the transmission quality. The present invention can effectively adapt to dynamic changes in a weak network environment, improve the real-time and stability of audio and video communications, and solve the problems of unstable transmission, severe delay and low retransmission efficiency in a weak network environment in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of a converged communication gateway module for weak network processing provided by one embodiment of the present application;

[0039] Figure 2 A diagram of a weak network processing gateway usage provided by an embodiment of the present application;

[0040] Figure 3 A conference system invitation initiation flow chart provided for one embodiment of the present application;

[0041] Figure 4 A schematic diagram of a call flow through a weak network processing gateway provided for one embodiment of the present application;

[0042] Figure 5 A flowchart of gateway processing in a weak network environment is provided for one embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] In the description of this application: unless otherwise specified, the meaning of "plurality" is two or more. The terms "first", "second", "third", etc. in this application are intended to distinguish the objects referred to, and do not have special meanings in terms of technical connotations (for example, they should not be understood as emphasizing the importance or order, etc.). Expressions such as "including", "comprising", "having", etc. also mean "not limited to" (certain units, components, materials, steps, etc.).

[0045] In one embodiment, Figure 1 As shown, a fusion communication gateway for weak network processing is provided, characterized in that the gateway includes:

[0046] The signaling module is used to provide SIP and RTC standard protocols, where SIP is used for call connection between external devices and RTC is used for call connection between gateways;

[0047] Media transmission module, used for transmission and control of audio and video media data through RTP and RTCP;

[0048] The service quality module is used to monitor the network transmission quality and process according to the packet loss rate and jitter in a weak network environment. The processing includes:

[0049] When the packet loss rate is less than or equal to 30% and the jitter is less than or equal to 200ms, a packet loss retransmission request is initiated;

[0050] When the packet loss rate is greater than 30% or the jitter is greater than 200ms, the frame rate and resolution of the transmission are reduced and the RTP stream is re-encoded.

[0051] This embodiment provides a converged communication gateway for weak network processing, aiming to solve the problems of obvious dynamic changes in network quality in a multi-network aggregation environment, insufficient timeliness of traditional retransmission mechanisms, and slow response to transmission parameter adjustment proposed in the background technology. By adopting the collaborative work of the signaling module, the media transmission module, and the quality of service (QoS) module, the transmission stability and real-time performance of audio and video communications in a weak network environment can be effectively improved.

[0052] Specific implementation method:

[0053] The signaling module provides SIP protocol and RTC protocol. Among them, SIP protocol is used for call connection between external devices, meeting the standard communication requirements of traditional conference systems and terminal devices, and ensuring that audio and video data can be called and connected smoothly.

[0054] At the same time, by establishing call connections between multiple converged communication gateways for weak network processing through the RTC protocol, RTC can reduce the amount of data for signaling interaction in a weak network environment, reduce network transmission delays, and speed up call establishment.

[0055] The signaling module connects the video conferencing system and the terminal device through SIPTrunk to ensure that the call request can be accurately forwarded to the target device, further improving the stability of the call connection.

[0056] The media transmission module uses RTP (Real-time Transport Protocol) and RTCP (Real-time Control Protocol) to transmit and control audio and video data.

[0057] RTP is used to transmit audio and video streams to ensure real-time communication, while RTCP is used to monitor the quality of transmission, such as bandwidth utilization, delay, and jitter, to provide data support for network optimization.

[0058] When audio and video streams are transmitted between multiple converged communication gateways that handle weak networks, the media transmission module ensures the forwarding and relaying of data streams, and optimizes the transmission path through an intelligent management mechanism to reduce packet loss and delay.

[0059] Network quality monitoring: The QoS module monitors the transmission quality of the network in real time, including detecting the packet loss rate and jitter of network transmission.

[0060] Weak network processing mechanism: According to the monitored network quality, the QoS module adopts dynamic adjustment strategies:

[0061] When the packet loss rate is less than or equal to 30% and the jitter is less than or equal to 200ms, a packet loss retransmission request is initiated, that is, a request is made to the sender to retransmit the lost data packet to reduce communication interruption caused by data loss.

[0062] When the packet loss rate is greater than 30% or the jitter is greater than 200ms, the QoS module reduces the frame rate and resolution of the transmission, and re-encodes the transmitted RTP stream to reduce the transmission bandwidth, ensuring that audio and video data can be stably transmitted when network resources are limited.

[0063] Network recovery detection: When it is detected that the network condition has returned to normal, the QoS module initiates a request to increase the frame rate and resolution, adjusts the transmission parameters back to normal, and restores the best transmission quality.

[0064] Through real-time network quality monitoring and dynamic parameter adjustment, the converged communication gateway in this embodiment can adapt to changes in network transmission quality, ensure the transmission stability of audio and video data in a weak network environment, and avoid image freezes and audio interruptions caused by problems such as packet loss and jitter.

[0065] When the packet loss rate is low, the system quickly replenishes the lost data packets through the packet loss retransmission mechanism, reducing the problem of incomplete information caused by data loss and ensuring the integrity of the audio and video streams.

[0066] By optimizing the initiation timing and processing mechanism of retransmission requests, the timeliness of the retransmission mechanism is improved to avoid data accumulation and increased delay caused by retransmission lag.

[0067] When network conditions deteriorate, the system can quickly reduce the frame rate and resolution and re-encode the RTP stream to reduce the transmission bandwidth, thereby ensuring smooth transmission of audio and video data when bandwidth resources are insufficient.

[0068] This dynamic adjustment mechanism has a fast response speed and can optimize parameters in real time according to network status, effectively reducing the degradation of communication quality caused by network congestion.

[0069] After the network returns to normal, the QoS module can detect changes in network status in real time, initiate requests to increase frame rate and resolution, and re-encode the RTP stream to adjust the transmission quality back to normal levels. This process can intelligently adapt to network changes and ensure that communication quality can quickly reach the best state after the weak network is restored.

[0070] Through the collaborative work of the signaling module, media transmission module and QoS module, the converged communication gateway realizes the intelligent management and dynamic optimization of audio and video data, effectively solving the communication quality problem in a weak network environment.

[0071] The system can maintain the real-time and stability of communications under complex network conditions with high packet loss rate and severe jitter, thus improving user experience.

[0072] Through the above specific implementation, the converged communication gateway of this embodiment can effectively solve the problems of unstable network quality, insufficient timeliness of retransmission mechanism and slow response speed of transmission parameters proposed in the background technology. The signaling module of the gateway ensures stable call connection, the media transmission module ensures efficient transmission of audio and video streams, and the service quality module optimizes transmission bandwidth and parameters through real-time monitoring and dynamic adjustment mechanism, ultimately improving the stability and real-time performance of audio and video communication in weak network environment.

[0073] In this embodiment, the service quality module includes:

[0074] A packet loss detection unit, used to detect the packet loss rate in network transmission;

[0075] A jitter detection unit, used to detect jitter in network transmission;

[0076] The bandwidth adjustment unit is used to dynamically adjust the transmission bandwidth according to the detected network transmission status.

[0077] The Quality of Service (QoS) module consists of a packet loss detection unit, a jitter detection unit, and a bandwidth adjustment unit. Each unit works together to achieve dynamic monitoring and real-time adjustment of network transmission quality.

[0078] The packet loss detection unit monitors the transmission status of the audio and video data streams and calculates the difference between the total amount of data packets sent and received, thereby obtaining the packet loss rate in real time.

[0079] The packet loss rate is detected using statistical methods. By sampling and analyzing data packets within a specific time window, the lost data packets are accurately located and the loss ratio is quantified.

[0080] The unit can transmit the detected packet loss rate information to the bandwidth adjustment unit in real time as an important basis for bandwidth optimization.

[0081] The jitter detection unit monitors the arrival time interval of audio and video data packets in real time, calculates the time deviation between data packets, and thus obtains the jitter of network transmission.

[0082] The unit uses continuous sampling technology to analyze the data flow over a period of time, obtain the average and peak values ​​of network jitter, and transmit the jitter value to the bandwidth adjustment unit.

[0083] Jitter detection can effectively identify abnormal audio and video playback problems caused by network delay fluctuations, such as image freezes or intermittent sound.

[0084] The bandwidth adjustment unit dynamically adjusts the transmission parameters of the audio and video data streams, including frame rate, resolution and data stream encoding mode, according to the real-time data provided by the packet loss detection unit and the jitter detection unit.

[0085] When it is detected that the network packet loss rate is low and the jitter is small, the bandwidth adjustment unit maintains the existing transmission bandwidth and parameter configuration to ensure high-quality audio and video transmission; when the packet loss rate increases or the jitter is significant, the bandwidth adjustment unit will automatically reduce the frame rate and resolution, and re-encode the RTP stream to reduce bandwidth occupancy and ensure continuous transmission of the data stream.

[0086] The bandwidth adjustment unit supports dynamic recovery of transmission parameters. When the network conditions improve, the system automatically increases the frame rate and resolution to restore transmission quality.

[0087] Through the packet loss detection unit and jitter detection unit, the service quality module can grasp the packet loss rate and jitter of network transmission in real time. The packet loss detection unit calculates the difference between the sent and received packets to promptly detect data loss problems; the jitter detection unit analyzes the arrival time interval of data packets to quickly capture the impact of network delay fluctuations on audio and video transmission. This precise network monitoring mechanism provides data support for subsequent bandwidth adjustments.

[0088] The bandwidth adjustment unit dynamically adjusts the audio and video transmission parameters based on the real-time network monitoring results. When the network quality is poor, the bandwidth adjustment unit reduces the frame rate and resolution to reduce the transmission bandwidth occupancy, ensure the continuous transmission of audio and video streams, and avoid communication interruptions due to network congestion.

[0089] After the network returns to normal, the bandwidth adjustment unit can automatically restore the transmission parameters to improve the quality of the audio and video streams. This dynamic adjustment mechanism ensures the stability and adaptability of communication.

[0090] The collaboration between the packet loss detection unit and the bandwidth adjustment unit enables the system to quickly respond to changes in network quality. When the packet loss rate is low, the packet loss retransmission mechanism quickly replenishes the lost data packets; when the packet loss rate is high, the bandwidth adjustment unit immediately reduces the bandwidth demand, thereby reducing the network burden. This optimization strategy significantly improves the transmission efficiency in a weak network environment.

[0091] The real-time analysis of the jitter detection unit can effectively reduce audio and video playback problems caused by network delay fluctuations, such as image freezes and intermittent sound. Through the fast response of the bandwidth adjustment unit, the system can balance the network transmission load and ensure smooth audio and video playback.

[0092] The various units in the service quality module cooperate with each other, making the converged communication gateway have strong intelligent adaptability. The system can dynamically adjust the transmission strategy according to the current network conditions and always maintain the best transmission efficiency and quality under different network conditions.

[0093] In summary, this embodiment realizes real-time monitoring of network quality and dynamic optimization of transmission parameters through the organic combination of the packet loss detection unit, the jitter detection unit and the bandwidth adjustment unit, effectively solves the audio and video communication delay, packet loss and freeze problems caused by unstable network quality, and greatly improves the audio and video transmission efficiency and stability in weak network environments.

[0094] In this embodiment, the signaling module establishes a call connection with an external conference system through SIPTrunk, and establishes a communication connection with other converged communication gateways for weak network processing through the RTC protocol.

[0095] The core function of the signaling module is to establish a call connection with the external conference system through SIPTrunk, and to establish a communication connection with other converged communication gateways for weak network processing through the RTC protocol. The specific implementation is as follows:

[0096] The signaling module in the system connects to the external conference system through SIPTrunk (SIP trunk), ensuring that the external device and the converged communication gateway for weak network processing can successfully establish a call connection.

[0097] SIPTrunk is a call control technology based on the SIP protocol. Through this technology, the signaling module can transmit signaling messages between different devices to achieve functions such as call establishment, maintenance and termination.

[0098] The signaling module forwards call requests between the external conference system and the terminal device, accurately routes the call signaling of the conference system to the target gateway or terminal device, thereby completing the initial connection of audio and video data.

[0099] Between multiple converged communication gateways that handle weak networks, the signaling module uses the RTC protocol for communication connection. The RTC protocol is based on a real-time communication mechanism, which can reduce the amount of data transmitted by signaling in a weak network environment, reduce latency, and ensure efficient connection between gateways.

[0100] The signaling module uses the RTC protocol to achieve fast signaling interaction in a weak network environment, establish communication paths between multiple gateways, and provide a stable channel for subsequent audio and video stream transmission.

[0101] The RTC protocol avoids the connection establishment failure or timeout problems that may occur in traditional protocols in weak network environments. Its lightweight signaling transmission mechanism significantly improves the success rate and response speed of call establishment.

[0102] After receiving a call request from an external conference system, the signaling module can route the call signaling to the correct target device through the SIP protocol, and maintain the call connection between gateways through the RTC protocol.

[0103] During call maintenance, the signaling module monitors the signaling interaction status in real time, handles possible packet loss or delay, and ensures complete transmission of signaling data.

[0104] The signaling module also has a certain fault tolerance capability and can automatically re-initiate a connection request when signaling transmission fails, thereby improving the robustness of the system.

[0105] Through the connection between SIPTrunk and the external conference system, the signaling module can quickly establish a call connection between the conference system and the target terminal device. The introduction of SIPTrunk technology simplifies the signaling interaction process between devices, ensures smooth communication between external devices and the converged communication gateway, and realizes the rapid response of the conference system.

[0106] The standardized design of the SIP protocol enables the converged communication gateway of the present invention to be compatible with mainstream conference systems and communication devices, and to meet the needs of various application scenarios.

[0107] By establishing communication connections between multiple converged communication gateways through the RTC protocol, the signaling module can effectively reduce the amount of signaling data, reduce network latency, and improve the success rate of connection establishment in a weak network environment.

[0108] The lightweight mechanism of the RTC protocol avoids the timeout or connection failure problems that may occur in traditional protocols under weak network conditions, ensuring that a stable signaling channel can be quickly established between gateways.

[0109] The signaling module transmits signaling data through the RTC protocol, which can significantly reduce the delay in the call establishment process, especially in a weak network environment. The low-latency signaling interaction mechanism ensures that the conference system and terminal devices can quickly complete the call connection and improve the user experience.

[0110] The application of SIPTrunk technology makes the signaling module have good compatibility, which can be seamlessly connected with mainstream conference systems and adapt to different network environments and equipment requirements.

[0111] In the communication between gateways, the RTC protocol further enhances the stability of the system. By dynamically adjusting the signaling transmission mechanism, it can effectively respond to changes in network conditions and avoid signaling interruptions or call drops.

[0112] The signaling module can establish communication connections between multiple converged communication gateways, realize the collaborative work between gateways, and provide basic guarantee for subsequent audio and video data transmission. This multi-gateway collaborative signaling mechanism is suitable for large-scale, multi-node complex network environments.

[0113] In summary, the signaling module described in this embodiment realizes efficient and low-latency call connection and signaling interaction through the combination of SIPTrunk and RTC protocols, effectively solves problems such as high call establishment delay, high connection failure rate and signaling interruption in weak network environments, and provides stable and reliable basic support for audio and video communication systems.

[0114] In this embodiment, the media transmission module is used to receive audio and video RTP streams, and transmit and forward the RTP streams between multiple converged communication gateways for weak network processing.

[0115] The core function of the media transmission module is to receive audio and video RTP streams, and transmit and forward the RTP streams between multiple converged communication gateways with weak network processing to ensure stable transmission of audio and video data in a complex network environment. The specific implementation is as follows:

[0116] The media transmission module receives audio and video RTP streams transmitted from upstream source devices (such as conference systems or terminal devices) through the network interface. RTP (Real-time Transport Protocol) is a protocol designed for real-time audio and video transmission, which can ensure the real-time and integrity of data under unstable network conditions.

[0117] The media transmission module performs preliminary processing on the received RTP stream, including packet parsing, sequence number checking, and timestamp alignment, to ensure the orderliness and continuity of the data stream and provide basic support for subsequent forwarding operations.

[0118] The media transmission module supports multi-point transmission and forwarding of RTP streams. In a weak network environment, multiple converged communication gateways may be distributed in different network nodes. The media transmission module needs to forward the received RTP stream according to the path planning to ensure that the data can smoothly reach the target gateway or terminal device.

[0119] The module uses a dynamic routing mechanism to select the optimal transmission path based on real-time monitoring of network conditions to reduce packet loss, delay and jitter.

[0120] The media transmission module can fragment the RTP stream to adapt it to the bandwidth requirements of different links, and reassemble it at the receiving end to ensure data integrity and consistency.

[0121] Data optimization during the release process

[0122] The media transmission module supports real-time encoding adjustment of RTP streams. When the network condition on the transmission path is detected to be poor, the module can dynamically reduce the frame rate and resolution of the RTP stream, reduce the transmission bandwidth usage, and ensure the continuous transmission of audio and video streams.

[0123] The module also supports the RTCP (Real-time Control Protocol) feedback mechanism, which is used to collect the transmission quality information of the receiving end (such as packet loss rate, delay, jitter, etc.) and optimize the transmission strategy based on the feedback results.

[0124] When multiple converged communication gateways with weak network processing jointly participate in audio and video data transmission, the media transmission module can work together between gateways to achieve seamless data stream transmission across gateways through a distributed forwarding mechanism.

[0125] The module distributes traffic and load balances the RTP streams between gateways to prevent a single node from affecting overall transmission performance due to overload.

[0126] The media transmission module receives and transfers audio and video data streams based on the RTP protocol, which can ensure the real-time and orderliness of audio and video data in a weak network environment. By parsing and optimizing the received RTP stream, the module ensures the timing correctness of the data stream and avoids the problem of audio and video asynchrony caused by network jitter.

[0127] The module uses a dynamic routing mechanism to select the optimal transmission path according to network conditions, effectively reducing packet loss and delay problems. The application of fragmented transmission technology further improves the adaptability of data streams in complex network environments and ensures efficient transmission of data streams on different links.

[0128] In a weak network environment, the media transmission module can dynamically adjust the encoding parameters of the RTP stream (such as frame rate and resolution) to reduce the network bandwidth usage while ensuring the basic quality of the data stream. This intelligent optimization strategy effectively improves bandwidth utilization, allowing the system to continue to provide stable audio and video services under limited network resources.

[0129] In multi-gateway collaboration scenarios, the media transmission module supports distributed transmission and load balancing of RTP streams, avoiding performance bottlenecks caused by overload of a single gateway. The distributed design of the module ensures that data stream transmission across gateways is more efficient and robust.

[0130] The media transmission module obtains the quality information of the transmission link in real time through the RTCP feedback mechanism and adjusts the transmission strategy according to the feedback results. This closed-loop optimization mechanism significantly improves the transmission quality of audio and video streams and reduces abnormal playback problems in weak network environments.

[0131] Whether it is a single-gateway scenario or a multi-gateway collaborative scenario, the media transmission module can flexibly adapt to complex network conditions. The module supports multi-path transmission, dynamic routing, and cross-gateway collaboration, allowing the system to maintain high transmission efficiency and data integrity in various complex network environments.

[0132] To sum up, the media transmission module of this embodiment realizes real-time, efficient and stable transmission of audio and video data by receiving, transmitting and forwarding RTP streams, solves the problems of data transmission delay, packet loss and lag in weak network environments in the background technology, and enhances the system's adaptability to complex network environments.

[0133] In this embodiment, when the service quality module detects that the network transmission status returns to normal, it initiates a request to increase the frame rate and resolution, and re-encodes the RTP stream.

[0134] When the Quality of Service (QoS) module detects that the network transmission status has returned to normal, it can initiate a request to increase the frame rate and resolution, and re-encode the RTP stream to restore the quality of audio and video transmission. The specific implementation method is as follows:

[0135] The QoS module monitors key indicators of network transmission in real time, including packet loss rate, jitter and delay, etc. By continuously analyzing the changing trends of these indicators, the module can determine whether the network status has returned to normal levels.

[0136] When the network packet loss rate drops to less than 30% and the jitter is less than 200ms, the QoS module will determine that the network transmission status has returned to normal and trigger the parameter recovery mechanism.

[0137] Once it is detected that the network status returns to normal, the QoS module initiates a request to the media transmission module to increase the frame rate and resolution.

[0138] The process of increasing the frame rate involves adjusting the inter-frame interval of the RTP stream to restore the smoothness of the audio and video streams to normal levels. Increasing the resolution involves adjusting the encoding resolution parameters to improve the clarity of images and videos.

[0139] To ensure the restoration of audio and video quality, the QoS module instructs the media transmission module to re-encode the RTP stream. The main goal of re-encoding is to improve the resolution and frame rate of the audio and video streams to adapt to the restored network bandwidth conditions.

[0140] During the re-encoding process, the module dynamically adjusts the encoding parameters according to network conditions to ensure efficient and stable transmission. For example, the module will give priority to selecting the appropriate encoding format and bit rate to ensure audio and video quality while avoiding excessive bandwidth usage.

[0141] The re-encoding of the RTP stream will also optimize the synchronization information to ensure the timing of the audio and video and avoid playback asynchrony caused by parameter recovery.

[0142] After the network condition is restored, the QoS module will not stop real-time monitoring of the network. If the network quality drops again after recovery, the module can quickly switch back to low frame rate and low resolution mode to ensure the continuous stability of audio and video transmission.

[0143] This dynamic adjustment capability ensures that the system always provides optimal transmission quality in an environment where network conditions fluctuate repeatedly.

[0144] After the network status returns to normal, the QoS module restores the quality and smoothness of audio and video to the best state by increasing the frame rate and resolution and re-encoding the RTP stream. High frame rate can significantly improve the smoothness of the video and reduce the sense of image freeze; high resolution can enhance the clarity of the video and improve the viewing experience. The process of re-encoding the RTP stream optimizes the structure and transmission characteristics of the data stream, ensuring the stable transmission of high-quality audio and video streams.

[0145] The network monitoring capability and dynamic adjustment mechanism of the QoS module enable the system to adapt to changes after network recovery in a timely manner. When network bandwidth is sufficient, the module can make full use of available resources, improve transmission parameters, and maximize audio and video quality. This dynamic adaptation mechanism avoids resource waste or transmission quality degradation caused by fixed parameter configuration.

[0146] The technical solution in this embodiment automatically restores the frame rate and resolution of audio and video, so that users can quickly enjoy high-quality audio and video services after the network is restored, without having to manually adjust or wait for a long parameter recovery process. This automated recovery mechanism significantly improves the consistency and comfort of user experience.

[0147] After the network is restored, the QoS module optimizes the encoding parameters of the RTP stream based on the real-time detected network bandwidth to ensure that the audio and video quality is improved without occupying bandwidth resources beyond the network's allowed range. By dynamically adjusting the frame rate, resolution, and bit rate, the system achieves efficient use of bandwidth resources.

[0148] The QoS module enables the converged communication gateway to have a high level of intelligence through automated network monitoring, parameter adjustment and re-encoding. The system can autonomously optimize audio and video transmission parameters according to real-time changes in network conditions without manual intervention or pre-set complex rules. This intelligent design enhances the system's adaptability in a changing network environment and significantly improves the reliability and stability of the audio and video communication system.

[0149] The parameter adjustment and re-encoding function after network recovery effectively solves the problem of audio and video quality degradation caused by network fluctuations. For example, when the network condition recovers from a weak network to normal, the resolution and frame rate of audio and video can be quickly improved, eliminating the problems of blurred image quality and playback jams, and maintaining the continuity and high quality of communication.

[0150] In summary, this embodiment achieves dynamic optimization of audio and video quality through the frame rate and resolution improvement mechanism of the QoS module after network recovery, as well as the re-encoding function of the RTP stream, solves the problem of failure to adjust the transmission quality in time after network recovery, and improves the intelligence and adaptability of the converged communication gateway for weak network processing.

[0151] In one embodiment, an audio and video communication method is provided, characterized in that the method comprises the following steps:

[0152] Through the signaling module, use the SIP protocol to connect calls with external devices, and use the RTC protocol to establish calls between converged communication gateways that handle weak networks;

[0153] Through the media transmission module, audio and video data are transmitted based on the RTP protocol;

[0154] Monitor network transmission quality through the service quality module, including detecting network packet loss rate and jitter;

[0155] According to the monitoring results:

[0156] When the packet loss rate is less than or equal to 30% and the jitter is less than or equal to 200ms, a packet loss retransmission request is initiated;

[0157] When the packet loss rate is greater than 30% or the jitter is greater than 200ms, the frame rate and resolution of the transmission are reduced and the RTP stream is re-encoded.

[0158] In this embodiment, when it is detected that the network transmission condition returns to normal, a request to increase the frame rate and resolution is initiated, and the RTP stream is re-encoded.

[0159] In this embodiment, the step of monitoring the network transmission quality includes:

[0160] Detect the packet loss rate of network transmission;

[0161] Detect the jitter of network transmission.

[0162] In one embodiment, this embodiment provides an audio and video communication system, characterized in that the system includes:

[0163] Video conferencing system;

[0164] At least two converged communication gateways for weak network processing according to claim 1;

[0165] The terminal device transmits audio and video data between the video conferencing system and the terminal device through the fusion communication gateway processed by the weak network.

[0166] In this embodiment, the converged communication gateways for weak network processing perform call connection through the RTC protocol.

[0167] In one embodiment, a device is provided for improving network quality by using an RTC (real-time communication) mechanism when a weak network condition occurs in a multi-network aggregation environment, so as to solve the problem of network quality.

[0168] With the continuous development of the network, various types of data are growing. In order to meet the actual application needs, it is necessary to aggregate and transmit various types of data to the same node for data management and analysis. Multi-network aggregation refers to multiple broadband lines, which superimpose their bandwidths together to achieve higher transmission speeds and more stable connection quality; more servers use multi-network aggregation when configuring to improve network reliability. At present, the network environment is becoming more and more complex, and abnormal situations such as data loss, data errors, and unstable data transmission often occur during data transmission, which leads to a decrease in data aggregation transmission efficiency and data aggregation transmission quality that does not meet requirements.

[0169] In a weak network environment, because the Sdes (synchronization information descriptor) frame transmission speed is slow, it is easy to have problems such as packet loss due to long waiting time. Therefore, it is necessary to use signaling to reduce the amount of data when building a link to reduce latency and speed up data transmission. RTC is an open standard protocol and API for transmitting media between web browsers. Its purpose is to replace the previous complex plug-in architecture with a standard HTTP channel to solve cross-browser compatibility issues. The present application is to provide a device that uses the RTC (real-time communication) mechanism to improve network quality in a multi-network aggregation environment when a weak network occurs. The specific technical solution is as follows:

[0170] The device is a gateway, which is composed of the following parts, including a signaling module, a media transmission module, and a quality of service (QoS) module.

[0171] Signaling module: provides SIP (Session initialization Protocol) and RTC (Real time communication) standard protocols, among which SIP is used for external device call connection, and RTC standard protocol is used for call connection between gateways.

[0172] Media transmission module: uses RTP (Real-time Transport Protocol) and RTCP (Real-time Control Protocol) for media transmission and control.

[0173] Quality of Service (QoS) module: The QoS module is a strategy used to resolve network delays and congestion.

[0174] like Figure 2 When the weak network processing gateway is not used, the normal conference system call process is as follows: Figure 2 - As shown in ①, the conference system directly initiates a call to the conference terminal. This kind of call will experience delays and freezes when encountering a weak network.

[0175] When a weak network gateway is used, a weak network processing system will be added between the conference system and the terminal to improve the stability and quality of audio and video when the network bandwidth is poor. Figure 2-②As shown, the video conferencing system finds the weak network processing gateway through SIPTrunk (SIP trunk), and the weak network processing gateway connected to the conference system finds the weak network processing gateway connected to the terminal through Trunk, and the weak network processing gateway connected to the terminal invites the terminal called by the conference system. In the case of a normal network, the weak network processing gateway will only monitor the network status and transcribe the audio and video stream; in the case of a weak network, the weak network processing gateway will process in different ways according to the specific situation of the monitored network: when the weak network processing gateway detects that the network packet loss rate is not greater than 30% and the jitter is less than 200ms, the weak network processing gateway will initiate a packet loss retransmission request; when the packet loss is greater than 30% and the jitter is greater than 200ms, the weak network processing gateway will initiate a request to reduce the frame rate and resolution, and the weak network processing gateway will automatically reduce the transmission bandwidth and re-encode the transmitted RTP stream.

[0176] like Figure 3 As shown, the conference system initiates an invitation flow chart. There are two ways for the conference system to invite terminals, the difference is whether it passes through the weak network processing gateway. The call protocol can be sip or h323. Taking SIP calls as an example, the conference system discovers the weak network processing gateway through SIPTrunk (SIP trunk). The IP of weak network processing gateway 1 is 192.168.1.3, and the IP of weak network processing gateway 2 is 192.168.1.4. Terminal 201 registers information to weak network processing gateway 2 through a standard private protocol, and weak network processing gateways 1 and 2 discover each other through SIPTrunk.

[0177] Figure 4 This is the call process through the weak network processing gateway. The following takes SIP call as an example.

[0178] Step 1: Conference system A calls the terminal through the SIP protocol. It will first be sent to the weak network processing gateway 1 by SIPTrunk; the weak network processing gateway 1 receives and processes the SIP call request initiated by conference system A, and gateway 1 sends the call to the weak network processing gateway 2 at the other end through the RTC protocol. The weak network processing gateway 2 then initiates a call to terminal B through the SIP protocol; the terminal responds to the call request and sends it to the weak network processing gateway 2, which then goes through the weak network processing gateway 2 to the weak network processing gateway 1 and then to conference system A, and the call connection is established.

[0179] Step 2: After the call is established, the terminal sends the audio and video stream via RTP to the weak network processing gateway 2, which then forwards the RTP stream to the weak network processing gateway 1, which then sends the RTP stream to the conference system A. The conference system A will also send the audio and video stream to the terminal B via the reverse path.

[0180] Step 3: When weak network processing gateway 1 and weak network processing gateway 2 detect that the transmission of video media is in a weak network environment. When the network packet loss rate between weak network processing gateway 1 and weak network processing gateway 2 is not greater than 30%, and the jitter is less than 200ms, weak network processing gateway 2 will send a packet loss retransmission request to request weak network processing gateway 1 to retransmit the lost packet; when the packet loss is greater than 30% and the jitter is greater than 200ms, weak network processing gateway 2 will send a request to reduce the frame rate and resolution to weak network processing gateway 1, and weak network processing gateway 1 will automatically reduce the transmission bandwidth, re-encode the transmitted RTP stream and transmit it to weak network processing gateway 2.

[0181] Step 4: When the network returns to normal, the weak network processing gateway 2 will send a request to increase the frame rate and resolution to the weak network processing gateway 1. The weak network processing gateway 1 will process the request to increase the transmission bandwidth, re-encode the transmitted RTP stream again, and transmit it to the weak network processing gateway 2.

[0182] like Figure 5 Gateway processing flow chart in a weak network environment. Weak network processing gateway 1 and weak network processing gateway 2 detect that the transmission of audio and video media is in a weak network environment. When the network packet loss rate between weak network processing gateway 1 and weak network processing gateway 2 is detected to be no more than 30%, and the jitter is less than 200ms, the weak network processing gateway will send a packet loss retransmission request to the other weak network processing gateway to request retransmission of the lost packet; when the packet loss is greater than 30% and the jitter is greater than 200ms, the other weak network processing gateway will send a request to reduce the frame rate and resolution to the weak network processing gateway, and the weak network processing gateway will automatically reduce the transmission bandwidth, re-encode the transmitted RTP stream and transmit it to the other weak network processing gateway.

[0183] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A fusion communication gateway for weak network processing, characterized in that: The gateway comprises: The signaling module is used to provide SIP and RTC standard protocols, where SIP is used for call connection between external devices and RTC is used for call connection between gateways; Media transmission module, used for transmission and control of audio and video media data through RTP and RTCP; The service quality module is used to monitor the network transmission quality and process according to the packet loss rate and jitter in a weak network environment. The processing includes: When the packet loss rate is less than or equal to 30% and the jitter is less than or equal to 200ms, a packet loss retransmission request is initiated; When the packet loss rate is greater than 30% or the jitter is greater than 200ms, the frame rate and resolution of the transmission are reduced and the RTP stream is re-encoded.

2. The fusion communication gateway for weak network processing according to claim 1, characterized in that: The service quality module includes: A packet loss detection unit, used to detect the packet loss rate in network transmission; A jitter detection unit, used to detect jitter in network transmission; The bandwidth adjustment unit is used to dynamically adjust the transmission bandwidth according to the detected network transmission status.

3. The fusion communication gateway for weak network processing according to claim 1, characterized in that: The signaling module establishes a call connection with an external conference system through a SIP Trunk, and establishes a communication connection with other fusion communication gateways for weak network processing through an RTC protocol.

4. The fusion communication gateway for weak network processing according to claim 1, characterized in that: The media transmission module is used to receive audio and video RTP streams, and transmit and forward the RTP streams between multiple converged communication gateways for weak network processing.

5. The fusion communication gateway for weak network processing according to claim 1, characterized in that: When the service quality module detects that the network transmission status returns to normal, it initiates a request to increase the frame rate and resolution, and re-encodes the RTP stream.

6. An audio and video communication method, characterized in that: The method comprises the following steps: Through the signaling module, use the SIP protocol to connect calls with external devices, and use the RTC protocol to establish calls between converged communication gateways that handle weak networks; Through the media transmission module, audio and video data are transmitted based on the RTP protocol; Monitor network transmission quality through the service quality module, including detecting network packet loss rate and jitter; According to the monitoring results: When the packet loss rate is less than or equal to 30% and the jitter is less than or equal to 200ms, a packet loss retransmission request is initiated; When the packet loss rate is greater than 30% or the jitter is greater than 200ms, the frame rate and resolution of the transmission are reduced and the RTP stream is re-encoded.

7. The method according to claim 6, characterized in that When it is detected that the network transmission condition returns to normal, a request to increase the frame rate and resolution is initiated, and the RTP stream is re-encoded.

8. The method according to claim 6, characterized in that The step of monitoring network transmission quality comprises: Detect the packet loss rate of network transmission; Detect the jitter of network transmission.

9. An audio and video communication system, characterized in that: The system comprises: Video conferencing system; At least two converged communication gateways for weak network processing according to claim 1; The terminal device transmits audio and video data between the video conferencing system and the terminal device through the fusion communication gateway processed by the weak network.

10. The audio and video communication system according to claim 9, characterized in that: The converged communication gateways for weak network processing perform call connections through the RTC protocol.

Images