A webRTC-based weak network remote access method and system

Abstract

The application discloses a weak network remote access method and system based on WebRTC, relates to the technical field of weak network remote access, and constructs a stable and reliable remote access method and system by using WebRTC technology, can intelligently monitor network bandwidth conditions, and in particular in a weak network condition, the system automatically switches an end-to-end network connection mode and selects the best network path. In addition, when the network path cannot meet the quality of service transmission, the application has an active degradation strategy, provides stable and reliable remote access experience for users, and simultaneously does not need to depend on a specific client, so that the application can easily adapt to the requirements of different platforms and devices, can provide high-quality real-time communication services, and creates more convenience and rich internet experience for users.

Description

A WebRTC-based method and system for remote access in weak network conditions Technical Field

[0001] This invention belongs to the field of weak network remote access technology, specifically a weak network remote access method and system based on WebRTC. Background Technology

[0002] Remote access technology refers to accessing and controlling computers, devices, or resources located in another location from one place via a network connection. The emergence of remote access technology allows users to perform various operations, obtain information, or manage resources without physically being near the accessed object. This demand for remote access covers multiple fields, including but not limited to remote work, remote technical support, remote server management, remote control, remote presentations, and remote monitoring. With the popularization of technologies such as cloud computing, the Internet of Things, and remote collaboration, remote access has profoundly changed the way people work and live.

[0003] While remote access brings immense convenience, the instability of the network environment has become a major factor hindering its development. Local area networks (LANs) can provide a relatively secure, efficient, and controlled network environment for establishing end-to-end connections. However, establishing end-to-end connections between LANs via the internet requires overcoming the limitations of firewalls and NAT devices, as well as considering weak network environments. Due to the complex and diverse network structure of the internet, issues such as packet loss, jitter, congestion, and latency are inevitable. These problems directly affect the effectiveness and experience of remote access, causing phenomena such as abnormal sound, screen flickering, black screens, and unresponsiveness.

[0004] WebRTC (Web Real-Time Communication) is an open technology standard for real-time audio and video communication between browsers; it serves as both an API and a protocol. WebRTC allows developers to implement real-time communication in web applications, such as audio, video, file sharing, and data transfer, without the need for plugins or other external software support, and is widely used in the field of real-time communication. However, WebRTC focuses on the protocol-level technical architecture and does not address the issue of audio and video transmission quality in weak network environments. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art.

[0006] A method for remote access in weak network conditions based on WebRTC, the method specifically includes the following steps:

[0007] After the remote server and the client establish an end-to-end connection, the remote server periodically sends heartbeat events to the client through the SCTP data channel. After receiving the heartbeat event, the client replies with a heartbeat response event to the remote server.

[0008] Obtain the approximate bandwidth BandWidth(Heart) of the current network;

[0009] The approximate bandwidth required to transmit the server's audio and video data is obtained separately, and the two are added together to obtain the approximate bandwidth BandWidth(Remote) required to transmit the audio and video data of the remote server.

[0010] When BandWidth(Heart) is greater than BandWidth(Remote), it means that the current network bandwidth can meet the quality transmission requirements for remote access; when the BandWidth(Heart) collected multiple times is less than BandWidth(Remote), it means that the current network bandwidth cannot meet the requirements for remote access, and it is necessary to try ICE candidate address switching or quality degradation.

[0011] When the approximate bandwidth of the end-to-end candidate address still cannot meet the quality transmission requirements for remote access, switch to relay mode.

[0012] When the quality degrades, the remote server Peer-A proactively reduces the audio and video sampling bitrate.

[0013] Furthermore, the method for obtaining the recent bandwidth (Audio) required by the transmission server is as follows:

[0014] BandWidth(Audio) = SampleRate * BitDepth * Channel * CompressRate * 8 calculates the approximate bandwidth required for transmitting audio to the server.

[0015] In the formula, SampleRate is the sampling rate of the audio file, BitDepth is the bit depth of the audio file, Channel is the number of channels of the audio file, and CompressRate is the compression ratio of the audio compression format.

[0016] Furthermore, the specific method for obtaining the approximate bandwidth required for the video transmission server is as follows:

[0017] The approximate bandwidth (BandWidth) required for transmitting video to the server is calculated using the following formula:

[0018] BandWidth(Vedio)=Resolution*FPS*CompressRate*8;

[0019] In the formula, Resolution is the resolution of the video file, FPS is the frame rate of the video file, and CompressRate is the compression ratio of the video format.

[0020] Furthermore, the remote server and client establish end-to-end RTP and SCTP connections by selecting the best combination of candidate addresses through the ICE protocol.

[0021] Furthermore, the approximate bandwidth of the current network is calculated using the following formula:

[0022] BandWidth(Heart)=2*X*8 / RTT;

[0023] In the formula, X is the number of bytes transmitted by the remote server in a single heartbeat data packet, and RTT is the heartbeat response time.

[0024] Furthermore, the remote server and client end-to-end connection method is as follows:

[0025] After the remote server starts up and is ready, it establishes a WebSocket connection to the signaling server and sends a connect event request to establish the connection; then it calls the getUserMedia method to obtain the local audio and video tracks and sends a join event request to register with the signaling service; the signaling server saves the status information of the remote server and waits for the client to connect;

[0026] The client sends a connect event request to the signaling server, carrying login authentication information; the signaling server matches the login information in the connect request with the credentials stored in the database; if the match is successful, it returns a login success message; if the match fails, it rejects the connect request.

[0027] After receiving the login success message, the client sends a join request to the signaling server to establish a connection with the remote server. After authenticating the user's operation, the signaling server forwards the connection request new-peer event to the remote server. After the remote server responds to the connection request, the signaling server responds with a resp-join event and returns it to the client. The client sends a ready event request, and both parties prepare to negotiate media information.

[0028] Furthermore, the specific method for negotiating media information between the two parties is as follows: the remote server creates an RTCPeerConnection object, collects local information, and calls the addTrack method to add local audio and video tracks to the RTCPeerConnection object for transmitting real-time video information from the server.

[0029] The `createDataChannel` method is called to create an SCTP data channel stream for transmitting interaction information between the client and the remote server; the `onIceCandidate` method is called to bind the ICE callback function and listen for the ICE protocol candidate event; then `createOffer` is called to create a session description.

[0030] The remote server calls setLocalDescription to save local information and generates an SDPoffer, which is then sent to the signaling server. The signaling server forwards the SDPoffer to the client. Upon receiving the SDPoffer from the remote server, the client creates an RTCPeerConnection object, calls setRemoteDescription to save remote information, and binds the ICE callback function using onIceCandidate. Subsequently, it calls createAnswer to generate a local SDPanswer.

[0031] The client filters out audio and video codecs or bitrate formats that are not supported by the remote server Peer-ASDPoffer based on its own supported attributes; it calls setLocalDescription to save the negotiated media information locally and sends SDPanswer to the remote server through the signaling server; then it calls ontrack to attempt to establish an end-to-end connection with the remote server using the ICE protocol.

[0032] After receiving the SDPanswer, the remote server calls setRemoteDescription to save the client's media description information, and then calls ontrack to attempt to establish an end-to-end connection with the client;

[0033] Both the remote server and the client send ICE requests to the Coturn server to obtain local network candidate information. Through the onIceCandidate callback function, they send their own network candidate information to the other party through the signaling server and call addIceCandidate to store the other party's candidate address.

[0034] Furthermore, the interactive information includes mouse, keyboard, latency detection, change, and heartbeat events generated during remote access.

[0035] Furthermore, the session description includes audio and video bitrate information, supported codec formats, ICE candidate addresses and authentication details, as well as the certificate fingerprint used for secure communication.

[0036] A WebRTC-based remote access system for weak networks is provided, which uses the aforementioned remote access method for weak networks to achieve remote access in weak networks.

[0037] Compared with the prior art, the beneficial effects of the present invention are:

[0038] This invention utilizes WebRTC technology to construct a stable and reliable remote access method and system. It can intelligently monitor network bandwidth, especially in weak network conditions, automatically switching end-to-end network connection modes and selecting the optimal network path. Furthermore, when the network path cannot meet the Quality of Service (QoS) transmission requirements, it has an active degradation strategy to provide users with a stable and reliable remote access experience.

[0039] Meanwhile, this application does not rely on a specific client, making it easy to adapt to the requirements of different platforms and devices, and can provide high-quality real-time communication services, creating more convenience and a richer Internet experience for users. Attached Figure Description

[0040] Figure 1 is a timing diagram of the present invention. Detailed Implementation

[0041] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Please refer to Figure 1. This application provides a WebRTC-based remote access method for weak network conditions. The method specifically includes the following steps:

[0043] S1. After the remote server Peer-A is started and ready, it establishes a WebSocket connection to the signaling server and sends a connect event request to establish the connection;

[0044] Then, the getUserMedia method is called to retrieve the local audio and video tracks, and a join event request is sent to register with the signaling service;

[0045] The signaling server saves the status information of the remote server Peer-A and waits for the client Peer-B to connect;

[0046] S2. Client Peer-B sends a connect event request to the signaling server, carrying login authentication information.

[0047] The signaling service matches the login information in the connect request with the credentials stored in the database;

[0048] If a match is found, a login success message is returned; if a match fails, the connect request is rejected.

[0049] S3. After receiving the login success message, the client then sends a join request to the signaling server to request a connection with the remote server Peer-A.

[0050] After authenticating the user's operation, the signaling server forwards the connection request new-peer event to the remote server Peer-A;

[0051] After the remote server Peer-A responds to the connection request, the signaling server responds to the resp-join event and returns it to the client Peer-B;

[0052] The client Peer-B sends a ready event request, and both parties prepare to negotiate media information;

[0053] S4. The remote server Peer-A creates an RTCPeerConnection object, collects local information, and calls the addTrack method to add local audio and video tracks to the RTCPeerConnection object for transmitting real-time video information from the server.

[0054] The createDataChannel method is called to create an SCTP data channel stream, which is used to transmit interactive information generated between the client Peer-B and the remote server Peer-A, including mouse, keyboard, latency detection, change, heartbeat and other events generated during remote access.

[0055] Call the onIceCandidate function to bind the ICE callback function and listen for the ICE protocol candidate event;

[0056] Then call createOffer to create a session description, including audio and video bitrate information, supported codec formats, ICE candidate addresses and authentication details, as well as certificate fingerprints used for secure communication;

[0057] S5. Remote server Peer-A calls setLocalDescription to save local information and generates SDPoffer, which is then sent to the signaling server. The signaling server forwards it to client Peer-B.

[0058] S6. When the client Peer-B receives the SDPoffer sent by the remote server Peer-A, it creates an RTCPeerConnection object, calls setRemoteDescription to save the remote information, and uses onIceCandidate to bind the ICE callback function.

[0059] Then createAnswer is called to generate a local SDP answer;

[0060] The client Peer-B filters out audio and video codecs or bitrate formats that are not supported by the remote server Peer-ASDPoffer based on its own supported attributes.

[0061] Call setLocalDescription to save the negotiated media information locally, and send SDPanswer to the remote server Peer-A through the signaling server;

[0062] Then, ontrack is invoked to attempt to establish an end-to-end connection with the remote server Peer-A using the ICE protocol;

[0063] S7. After receiving the SDPanswer, the remote server Peer-A calls setRemoteDescription to save the client's media description information, and then calls ontrack to attempt to establish an end-to-end connection with the client Peer-B.

[0064] S8. Both remote server Peer-A and client Peer-B send ICE requests to the Coturn server to obtain local network candidate information. Through the onIceCandidate callback function, they send their own network candidate information to the other party through the signaling server and call addIceCandidate to store the other party's candidate address.

[0065] Remote server Peer-A and client Peer-B establish end-to-end RTP and SCTP connections by selecting the best combination of candidate addresses through the ICE protocol.

[0066] S9. After the remote server Peer-A and the client Peer-B establish an end-to-end connection, the remote server Peer-A periodically sends heartbeat events to the client Peer-B through the SCTP data channel.

[0067] After receiving the heartbeat event, client Peer-B replies with a heartbeat response event to remote server Peer-A;

[0068] Assuming that the remote server Peer-A transmits X bytes of heartbeat data packets in a single transmission and the response time to the heartbeat event is RTT, the approximate bandwidth BandWidth(Heart) of the current network can be calculated using the formula BandWidth(Heart) = 2*X*8 / RTT.

[0069] Audio file size depends on the sample rate (SampleRate), bit depth (BitDepth), number of channels (Channel), and compression ratio (CompressRate) of the audio compression format, according to the formula:

[0070] BandWidth(Audio)=SampleRate*BitDepth*Channel*CompressRate*8;

[0071] Obtain the approximate bandwidth (BandWidth, Audio) required by the transmission server;

[0072] Video file size depends on resolution, frame rate (FPS), and the compression ratio (CompressRate) of the video format, according to the formula:

[0073] BandWidth(Vedio)=Resolution*FPS*CompressRate*8;

[0074] To obtain the approximate bandwidth required for the video transmission server;

[0075] The approximate bandwidth required to transmit audio and video data to a remote server can be obtained as BandWidth(Remote) = BandWidth(Audio) + BandWidth(Vedio);

[0076] When BandWidth(Heart) is greater than BandWidth(Remote), it means that the current network bandwidth can meet the quality transmission requirements for remote access; when the BandWidth(Heart) collected multiple times is less than BandWidth(Remote), it means that the current network bandwidth cannot meet the requirements for remote access, and it is necessary to try ICE candidate address switching or quality degradation.

[0077] S10. When remote server Peer-A performs ICE candidate address switching, it first sends an ICE candidate address switching event to client Peer-B.

[0078] Subsequently, remote server Peer-A and client Peer-B send ICE requests to the Coturn server to reacquire network candidate information;

[0079] When the approximate bandwidth of the candidate end-to-end address still cannot meet the quality transmission requirements of remote access, the system switches to relay mode, and the remote server Peer-A and the client Peer-B re-establish a connection with the Coturn server as the relay node.

[0080] S11. When the quality is degraded, the remote server Peer-A actively reduces the audio and video sampling bitrate, calls the setLocalDescription method to reset SDPoffer, and sends it to the client Peer-B through the SDP server.

[0081] The client Peer-B calls the setRemoteDescription method to save the remote media description information, selects the supported audio and video codec formats from it, regenerates the SDPanswer, and sends it to the remote server Peer-A.

[0082] After the remote server Peer-A saves the media description information of the peer, it retransmits the audio and video streams.

[0083] S12. When client Peer-B actively disconnects, it sends a leave event to remote server Peer-A via the SDP server. After both parties close the connection channel, remote access control is terminated.

[0084] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A method for remote access in weak network conditions based on WebRTC, characterized in that, The method specifically includes the following steps: After the remote server and the client establish an end-to-end connection, the remote server periodically sends heartbeat events to the client through the SCTP data channel. After receiving the heartbeat event, the client replies with a heartbeat response event to the remote server. The approximate bandwidth BandWidth(Heart) of the current network is obtained. The approximate bandwidths required for transmitting the server's audio and video data are obtained separately, and then added together to obtain the approximate bandwidth BandWidth(Remote) required for transmitting audio and video data from the remote server. When BandWidth(Heart) is greater than BandWidth(Remote), it indicates that the current network bandwidth can meet the quality transmission requirements for remote access. When the BandWidth(Heart) collected multiple times is less than BandWidth(Remote), it indicates that the current network bandwidth cannot meet the requirements for remote access, and it is necessary to try switching ICE candidate addresses or degrading the quality. When the approximate bandwidth of the end-to-end candidate address still cannot meet the quality transmission requirements for remote access, the system switches to relay mode. During quality degradation, the remote server Peer-A actively reduces the audio and video sampling bitrate. The method for obtaining the recent bandwidth BandWidth(Audio) required for transmitting the server's audio is as follows: The approximate bandwidth required for transmitting audio to the server is calculated as follows: BandWidth(Audio) = SampleRate * BitDepth * Channel * CompressRate * 8. Where SampleRate is the sampling rate of the audio file, BitDepth is the bit depth of the audio file, Channel is the number of channels in the audio file, and CompressRate is the compression ratio of the audio compression format. The specific method for obtaining the approximate bandwidth required for transmitting video to the server is as follows: BandWidth(Vedio) is calculated using the following formula: BandWidth(Vedio) = Resolution * FPS * CompressRate * 8. Where Resolution is the resolution of the video file, FPS is the frame rate of the video file, and CompressRate is the compression ratio of the video format. The approximate bandwidth of the current network is calculated using the following formula: BandWidth(Heart) = 2 * X * 8 / RTT; where X is the number of bytes transmitted by the remote server in a single heartbeat data packet, and RTT is the heartbeat response time.

2. The WebRTC-based remote access method for weak network conditions according to claim 1, characterized in that, The remote server and client establish end-to-end RTP and SCTP connections by selecting the best combination of candidate addresses through the ICE protocol.

3. The WebRTC-based remote access method for weak network conditions according to claim 1, characterized in that, The remote server and client end-to-end connection method is as follows: After the remote server starts up and is ready, it establishes a WebSocket connection to the signaling server and sends a connect event request to establish the connection; then it calls the getUserMedia method to obtain the local audio and video tracks, and sends a join event request to register with the signaling service; The signaling server stores the status information of the remote server and waits for client connections; the client sends a connect event request to the signaling server carrying login authentication information; The signaling service matches the login information in the connect request with the credentials stored in the database; if a match is found, it returns a login success message. If a match fails, the connect request is rejected; upon receiving a successful login message, the client then sends a join request to the signaling server to establish a connection with the remote server. After authenticating the user's operation, the signaling server forwards the connection request new-peer event to the remote server; after the remote server responds to the connection request, the signaling server responds with the resp-join event and returns it to the client; the client sends a ready event request, and both parties prepare to negotiate media information.

4. The WebRTC-based remote access method for weak network conditions according to claim 3, characterized in that, The specific method for media information negotiation between the two parties is as follows: the remote server creates an RTCPeerConnection object, collects local information, and calls the addTrack method to add local audio and video tracks to the RTCPeerConnection object for transmitting real-time video information from the server. The `createDataChannel` method is called to create an SCTP data channel stream for transmitting interaction information between the client and the remote server. The `onIceCandidate` method is called to bind the ICE callback function and listen for ICE protocol candidate events. Then, `createOffer` is called to create a session description. The remote server calls `setLocalDescription` to save local information and generates an `SDPoffer` which is sent to the signaling server, which forwards it to the client. Upon receiving the `SDPoffer` from the remote server, the client creates an `RTCPeerConnection` object, calls `setRemoteDescription` to save remote information, and binds the ICE callback function using `onIceCandidate`. Subsequently, `createAnswer` is called to generate a local `SDPanswer`. The client then filters the responses based on its supported attributes. The remote server Peer-ASDPoffer does not support audio / video codecs or bitrate formats; it calls setLocalDescription to save the negotiated media information locally and sends SDPanswer to the remote server through the signaling server; then it calls ontrack to attempt to establish an end-to-end connection with the remote server using the ICE protocol; after receiving the SDPanswer, the remote server calls setRemoteDescription to save the client's media description information, and then calls ontrack to attempt to establish an end-to-end connection with the client; both the remote server and the client send ICE requests to the Coturn server to obtain local network candidate information, and send their own network candidate information to the other party through the signaling server via the onIceCandidate callback function, and call addIceCandidate to store the other party's candidate address.

5. A method for remote access in weak network conditions based on WebRTC according to claim 4, characterized in that, Interactive information includes mouse, keyboard, latency detection, change, and heartbeat events generated during remote access.

6. The WebRTC-based remote access method for weak network conditions according to claim 1, characterized in that, The session description includes audio and video bitrate information, supported codec formats, ICE candidate addresses and authentication details, as well as the certificate fingerprint used for secure communication.

7. A WebRTC-based remote access system for weak network conditions, characterized in that, The system uses the weak network remote access method as described in any one of claims 1-6 to achieve weak network remote access.

Images