A browser-based real-time communication system based on P2MP technology

Abstract

The present invention relates to a browser-based real-time communication system based on P2MP technology, belonging to the field of network audio and video technology, using OSPF (Open Shortest Path First) to calculate the network status between each user browser state machine, produce a routing table; process the topological link transmission election of the browser state machine, and process a series of operations such as call forwarding and tearing of the communication chain of each state machine; the entire page client module is developed based on HTML5 pages, and the gRPC duplex stream end is established based on the browser to negotiate and transmit the media stream. The network interaction topology structure returned by the signaling server is established, and the point-to-point or point-to-multipoint transmission mode is used to encapsulate and transmit the audio and video data to the remote participants to realize NAT penetration, and the TURN server itself realizes remote penetration. The present invention solves the compatibility problem between various browsers on the basis of the technology of webrtc, and the flexible mode solves the transmission delay problem of P2MP, and improves the flexibility and availability of the network.

Description

Technical Field

[0001] The present invention relates to the field of real-time communication and audio and video, and in particular to a user-to-user real-time communication system based on P2MP technology and private protocol. Background Art

[0002] The existing audio and video real-time communication technology is mainly divided into two directions. First, the media is forwarded by the public network cloud server. The biggest problem is that a large amount of network data will be borne by the operating server, which will generate a large amount of network charges. Since the data is forwarded through the public network, its communication delay is also determined by the delay between the user's network and the public network server, and it cannot be reasonably distributed. Second, the real-time communication technology based on Google's open source webRtc realizes the establishment of peer-to-peer connections between browsers without the help of intermediate media, but the browser manufacturers' support for the protocol is not the same. For different browsers, it is necessary to develop corresponding SDKs for calling, and its compatibility cannot be guaranteed; if the internal and external penetration cannot be achieved based on the STUN protocol during data transmission, it is still necessary to provide a relay server to achieve forwarding.

[0003] This development will be based on the HTML5 front-end server, which greatly improves the user's ease of use. Participants only need to visit a specified page to complete client access; for the public network server, only a signaling server system needs to be deployed, which improves the efficiency of service construction and integration.

[0004] The HTML5 front-end server is compatible with various browsers, which solves the browser compatibility problem in webRtc services. The various compatibility problems that the original webRtc supporting products often encounter when upgrading browsers or switching to other browsers have been uniformly solved, improving the stability of the system.

[0005] Many existing streaming media servers are set up in the public network. A large amount of media forwarding will bring a lot of additional costs, and there are also maintenance and management issues. The present invention is based on the front-end server and realizes the function of network management to accept forwarding by itself. While realizing peer-to-peer connection based on the browser, it also realizes the function of network relay, and each user is both a participant and a forwarding server. The cloud server greatly reduces the interaction of network data. A large amount of media stream data is borne by the user's broadband, which solves the relay forwarding problem in the original webRtc and greatly reduces the bandwidth occupancy of the cloud server. The unified protocol bearer based on the gRPC channel also improves the security of remote connections.

[0006] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0007] The purpose of the present invention is to overcome the shortcomings of the prior art and to implement a browser-based real-time communication system based on P2MP technology, which solves the problem of large bandwidth occupation in the streaming media forwarding mode of the cloud server and adopts the OSPF (Open Shortest Path First) mode for network management topology construction, thereby reducing the delay in real-time communication to a certain extent and improving the user experience.

[0008] It also solves the browser compatibility issues in the point-to-point real-time communication mode based on webRtc, as well as the shortcomings of relying on relay servers in special circumstances, improves service integration efficiency, and improves necessary functions.

[0009] The purpose of the present invention is achieved through the following technical solutions: A browser-based real-time communication system based on P2MP technology, which includes a signaling server, a page client and various modules implemented by the client:

[0010] The signaling server is a service server deployed on the public network, which includes the operating system and system hardware deployed on the cloud server, the server operating system liunx system and the service program deployed in the system; the service program includes a sip server for processing signaling, a network negotiation server using OSPF, an authorization management server, and a series of application services such as a database deployed in the operating system. Unified authority authentication; responsible for group building, command interaction and interactive negotiation with the participant client;

[0011] The network negotiation server is used to guide the page client of the participant to create or update a point-to-point communication channel with other participants, including coordinating a certain participant to act as a relay server; it also provides part of the computing power to optimize the changing topology tree network tunnel;

[0012] The network tree-shaped network tunnel is used to build an internal and external penetration tunnel based on gRPC, which may include one or more relay nodes. The relay node is also created by the network transmission module of the page client and is responsible for forwarding network protocol data that cannot be directly transmitted point-to-point, thereby building a tree-shaped network transmission tunnel to maximize the establishment of remote channels and achieve point-to-point remote penetration.

[0013] It also includes a module with OSPF function and an open shortest path first mode based on the public network. Usually, the OSPF protocol is used for addressing between LAN routers. The present invention expands this and derives the function idea from the session participants in the public network group, thereby realizing the network neighbor formation guided by the network negotiation server. At the same time, the database is used to record the routing status of the entire network.

[0014] The signaling server includes a sip server that uses the gRPC channel established by the participant client as a carrier to encapsulate the data of the media negotiation server and the network negotiation server, and uses a private protocol for transmission, so as to control the operation of the participants. The participants will also report the real-time network status through the channel with the sip server, which serves as a data basis for the decision-making of the signaling server. At the same time, it also realizes the functions of the STUN server, and the user page client obtains its own public network information.

[0015] The signaling server should include an authorization server, which performs authentication management on the connections established by subsequent clients after the participant page client initiates registration and login, and coordinates the mutual trust process between participants in the group to ensure the security of peer-to-peer network transmission between participants.

[0016] The steps of initiating registration by the page client include: obtaining authorization method to obtain page client configuration user information; obtaining network information of the user server, browser information, and hardware information that can be used for audio and video transmission; initiating registration with the server by logging in with an account; and obtaining a server registration response.

[0017] The page client establishes point-to-point remote penetration including:

[0018] The page client obtains the public network IP port information of each participant in the member group by interacting with the signaling server;

[0019] The page client creates a new gRPC duplex stream (including client and server), and tries to obtain its own public IP address and the port opened by NAT to allow incoming traffic to be transmitted back to the intranet through the integrated STUN server of the signaling server, and reports its own public network information through the gRPC channel with the signaling server created after registration;

[0020] After the signaling server obtains the public network information of all participants, it will send the member network information to the page client;

[0021] The page client will first try to establish a gRPC channel with all members as a gRPC client, and report the success or failure result back to the signaling server;

[0022] The network negotiation service of the signaling server will select the sub-tunnel information to be used and send it back to each client. If there is at least one tunnel between all members that can be used for point-to-point media data transmission, the network is successfully established, otherwise it fails. If gRPC channels can be established from A to B and from B to A, the signaling server will decide which one to use and enable one of them.

[0023] The client will report the network delay between each participant based on the sub-tunnel to the signaling server at regular intervals. When the backup connection needs to be re-established, the signaling server will issue a specified command, and the client will repeat the above negotiation operation until the backup connection is successfully or failed to be created for subsequent switching of the sub-tunnel;

[0024] After the participants have established interoperable sub-tunnels, based on their own streaming media module, the SDP protocol (generic session negotiation) for processing streaming media negotiation is adopted to negotiate with each participant, thereby selecting and adopting a streaming media protocol that can be supported by all to prepare for real-time media transmission.

[0025] The psychological server will provide a web management terminal, which will provide the overall system

[0026] At least C(2, N) Sub-LSP tunnels will be established between the participant clients and the signaling server, and at least one available channel will be ensured to any participant in the group, and each participant will have an available interaction channel with the signaling server.

[0027] The audio and video module of the page client will be responsible for obtaining the media hardware operation permissions on the user's PC, such as cameras, microphones, etc., through HTML5 calling the browser interface. It also supports the implementation of some extended functions such as screen sharing. Its main function is to obtain the media source and generate media stream data to be loaded on the gRPC channel established with each participant; it is also responsible for decoding the streaming media data obtained from the gRPC channel according to the media protocol and rendering it to the user's hardware device, ultimately completing point-to-point real-time audio and video communication.

[0028] The private protocol includes two parts: a private header and protocol data content, which bears the carrier of the protocol data, thereby achieving unified interaction and increasing security and compatibility with various browsers;

[0029] The private header is divided into 4 bytes of MAGIC custom encapsulation header, 4 bytes of custom encapsulation header bytes, 4 bytes of member identifier, 4 bytes of message length, 1 byte of forwarding identifier, and 2 bytes of protocol type;

[0030] The member representation can be matched with the member participant information of the group sent by the signaling server to achieve mutual trust in the interaction within the group. By judging the member ID of the participant, if there is a mismatch, the data packet is considered unreliable and discarded. The forwarding identifier allows the client to quickly identify whether the current database is obtained through forwarding.

[0031] The present invention has the following advantages: a browser-based real-time communication system based on P2MP technology solves the browser compatibility problem in webRtc, is compatible with multiple application layer protocols through private protocols, and improves security. The design concept of the user client as a relay server is adopted to reduce the network bandwidth occupation of the cloud server, increase the throughput and carrying capacity of the cloud server in disguise, strengthen the network data processing capability, and improve the flexibility and availability of the network. The network topology of the optimal path effectively solves the point-to-point intranet penetration while reducing network delays. The simple construction of the entire system frees up the work of developers, and the convenient participant joining mode is better than various existing P2P real-time communication technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a structural schematic diagram of the system of the present invention;

[0033] Figure 2 It is a schematic diagram of the structure of the signaling server system;

[0034] Figure 3 This is a schematic diagram of the page client module structure;

[0035] Figure 4 Schematic diagram of the authorization process for page client registration;

[0036] Figure 5 Schematic diagram of the P2MP traversal process for establishing a tree-shaped Sub-LSP tunnel;

[0037] Figure 6 A schematic diagram of the network topology constructed for the signaling server using OSPF;

[0038] Figure 7 Schematic diagram of the protocol data structure encapsulation for the gRPC private protocol. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application provided below in conjunction with the drawings is not intended to limit the scope of protection of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application. The present invention is further described below in conjunction with the drawings.

[0040] like Figure 1 As shown, a browser-based real-time communication system based on P2MP technology includes a signaling server, a participant client built based on a browser page, and module units constituting the page client:

[0041] Among them, the signaling server and the cloud server include the operating system and system hardware deployed on the cloud server, and the service application based on the server operating system. The server application includes the media negotiation management application, the network negotiation management application, and the authorization management application. The media negotiation management application includes the application based on the SIP service and interacts with the page client, that is, the participant, for media negotiation data. The network negotiation management is used to perform dynamic network negotiation to the participant's page client, guide the establishment of data interaction channels between participants, serve as the STUN protocol server, and optimize the network topology. The authorization management application is used to manage user client login and group creation.

[0042] Participants are the integrated client modules that register and authorize with the signaling server; create network channels between participants through network negotiation of the signaling server; call user media-related hardware through the browser, and render audio and video media data; it is mainly divided into secure access service, signaling interaction module, streaming media module, audio and video processing module, and network transmission module.

[0043] The secure access service is used to initiate a registration request to the signaling server, perform permission verification on authorized access requests, and perform preliminary filtering of data through the user's unique identifier in the interactive protocol in the private gRPC channel. It is the unified entrance and exit of the entire page client.

[0044] Furthermore, the page client also includes a signaling interaction module, which is developed based on an html5 page and integrated into a web page, using a gRPC channel as a carrier; the browser-related information and the network-related information are interacted with the signaling server. After the signaling server aggregates the relevant information of each participant, media negotiation and network negotiation will be performed, and a network group will be formed for unified management of the external network addresses of multiple participants.

[0045] like Figure 2 As shown in the figure, the signaling server on the cloud server is an application service program deployed based on the operating system. Its application mainly includes three modules: the media negotiation management module installed on the server is responsible for parsing the SDP media negotiation protocol, negotiating and determining the streaming media interaction protocol between two participants, such as (h264, h265, AAC, G711 audio and video format); the network negotiation management module installed on the server includes the STUN server and the network module that implements the OSPF open shortest path first method; the authorization module uniformly manages the connection SESSION with the signaling server, and finally provides a unified network interface to the outside world; the network interface can be divided into a management web-related interface based on the signaling server, and a registration command-related interface for interacting with participants.

[0046] The web page management provided by the network interface is a platform for unified management of all participants who use this mechanism to achieve P2MP real-time communication. It is provided to management personnel, who can try to view the network bandwidth of the current signaling server, system occupancy, and the currently participating groups. It can provide intuitive service status viewing for operation and maintenance personnel, and as a management end, it can expand and push unified messages to each group. By issuing commands, one of the participants can be specified to forward media data to the signaling server (and the signaling server is a virtual participant). If the group is detected to be involved in illegal video dissemination, the group can be closed, further strengthening security management.

[0047] like Figure 3 As shown in the figure, the page client is a service program developed based on HTML5 and runs on the user's browser. It is divided into one service and four functional modules. The secure access service is to uniformly manage the registration connection initiated by the page registration and the signaling server, as well as the security verification of the data connection between the established participants. The entire client service is developed by HTML5 to call the NetWork I / O end of the user's operating system to realize network interactions such as network card establishment.

[0048] Its four modules are signaling interaction module, streaming media module, audio and video processing module, and network transmission module. After the secure access service completes registration and authentication, the audio and video processing module further obtains the multimedia-related hardware and supported media protocols of the participants, and the signaling interaction module uniformly reports them to the signaling server. After completing the pairwise media negotiation on the signaling server, the network transmission module will be notified to establish its own gRPC client for streaming media transmission, and send the established server information to the signaling server.

[0049] Furthermore, after the server aggregates the network information, the signaling interaction module obtains all network information of the participants in the group, and the network transmission module is responsible for establishing a point-to-point streaming media channel.

[0050] After establishing a point-to-point data channel between two participants, the streaming media module will transmit media data based on the negotiated media information. Common media streams include RTSP / STMP, etc., which will be transmitted under the private gRPC channel and processed by the streaming media server according to different protocols; the audio and video processing module will obtain the permission of multimedia devices from the user's computer through the API provided by the browser, and render the data parsed by the streaming media module. The media hardware devices we can obtain through the browser include cameras, microphones, and amplifiers (sound);

[0051] Furthermore, if Figure 4 As shown, the steps for the page client to initiate registration authentication include: obtaining browser information and user device permissions; entering the account and password to complete the login and establish a gRPC channel; creating a group to obtain the group sharing code or join the group through the sharing code; obtaining the member information sent by the server; and adding the unique identifier of the member to the mutual trust.

[0052] This registration method includes the use of private gRPC channels. Based on the compatibility of HTML5 on browsers, this registration program is used on servers based on Linux, window, uinx and other operating systems. Through the creation of groups, the signaling server manages them uniformly, establishes network neighbors in the public network, and filters according to the unique identification permissions of the group members issued by the signaling server. For example, if A receives data packets from two participants B and C, but the member identification of C is no longer in the group, A will not accept C's data, while B's data will be authenticated. Similarly, the data from A to B is authenticated after B's judgment, so A and B are in a state of mutual trust. This state of mutual trust is only related to the unique identification of the group members. If the IP information changes after the network sends the edge, when A and B still trust each other, this will greatly ensure the reliability of data transmission.

[0053] Furthermore, if Figure 5As shown in the figure, the tree-shaped Sub-LSP is the gRPC network tunnel from the client to a single leaf client, which is subsequently referred to as the Sub-LSP tunnel. The P2MP penetration process includes:

[0054] S001. Participants first create their own gRPC duplex stream (including client and server);

[0055] S002. Access the stun server of the signaling server through the stun protocol and obtain its own external network IP and port;

[0056] S003. After the signaling server obtains the external network IP and port of all members in the current group, it will use the gRPC channel between the server and the client to send the external network IP and port of the members in the group, and send them to all members by binding the participant's unique identification and IP address through a private protocol. After the participant client obtains the external network IP and unique identification of other participants, it establishes trust rules;

[0057] S004. Further, the participant client will create a gRPC connection to each participant in the group based on the external network IP and port of other participants in the external network. If the connection can be established directly, a direct Sub-LSP tunnel is established. If the connection cannot be established between the participants in the group, the signaling server needs to coordinate and try to create a forwarding Sub-LSP tunnel.

[0058] S005. The client reports to the signaling server whether the direct Sub-LSP tunnel with the participants in the group is successfully created. The signaling server selects one or more participants as relays based on the currently reported connection establishment status, and sends the forwarded line to the participants through which the forwarding channel passes. The participants will parse and save the source and destination of the data to be forwarded. After all the participants through which the channel passes have accepted and applied it, the result will be reported to the signaling server. At this time, a forwarded Sub-LSP tunnel is established.

[0059] S006. If all attempts fail to establish the Sub-LSP tunnel between participants A and B, the networks A and B are unreachable.

[0060] like Figure 6 As shown in the figure, it is a schematic diagram of the network topology formed by the signaling server using OSPF. Here, six participant nodes P1, P2, P3, P4, P5, and P6 are abstracted, and the premise is set that a direct Sub-LSP tunnel can be established only between two adjacent nodes. This is to illustrate how the signaling server forms the network topology.

[0061] Furthermore, we can obtain six directly connected Sub-LSP tunnels: P1-P2, P2-P3, P3-P6, P5-P6, P4-P5, and P1-P4. The participants report the direct network delays of these channels in real time, and the signaling server records the current network status:

[0062] If you need to build a direct network tunnel between P1 and P3, the server will have two options: P1-P2-P3 or P1-P4-P5-P6-P3. The server will compare the network delays of the two methods. If the network delays are close, it will also choose the combination with fewer forwarding nodes, and obtain the path (Open Shortest Path First) through the signaling server using OSPF. At this time, when we choose the P1-P2-P3 path, the signaling server will send commands to the clients of P1, P2, and P3, telling P1 to send the data sent to P3 to P2, and telling P2 to forward the data from P1 to P3. At this time, a Sub-LSP tunnel forwarded by P1-P2-P3 is established.

[0063] like Figure 6 As shown in the figure, we have established four forwarding Sub-LSP tunnels, and the signaling server coordinates to ensure that there is at least one usable Sub-LSP tunnel between any two nodes. If there is a large delay between P1 and P6 due to network reasons, or there is a network unreachable situation between P1, P2, and P3, the signaling server will re-elect a connectable Sub-LSP tunnel after obtaining the network change, and repeat the previous establishment method to send commands to the nodes along the channel. At this time, we have established a backup Sub-LSP tunnel P1-P4-P5-P6. When the previous Sub-LSP tunnel forwarded by P1-P2-P3-P6 cannot be used, it will immediately switch to the backup channel to ensure the reliability of network transmission, and can also make real-time use of network resources to achieve the optimal network topology construction.

[0064] Most of the network data transmission between the signaling server and participants, and between participants, is carried out through private protocol interaction using gRPC as the carrier. The network is established by using the forwarding flag and participant unique identifier in the private protocol. That is, the command interaction between the signaling server and participants is also a custom protocol.

[0065] like Figure 7 As shown, the private protocol will be divided into two parts, a 15-byte private header and the subsequent protocol data.

[0066] The private header is divided into 4 bytes of MAGIC custom encapsulation header, 4 bytes of member identification, 4 bytes of message length, 1 byte of forwarding identification, and 2 bytes of protocol type. The MAGIC encapsulation header is used to determine the starting position of the packet. The member identification is generated by the signaling server after the user logs in to the signaling server. 4 bytes can represent 16^4 data, which is sufficient for member expansion. The member representation is also a judgment of the legality of network transmission. If the unique identifier is not included in the member representation of the current group, the message is illegal and will be discarded; the 1-byte forwarding representation can indicate whether the current packet comes from the forwarding, which can speed up the efficiency of the client's message parsing. If it is direct connection data, it can be directly accepted and put into the corresponding protocol processing module according to the protocol type. If it is a forwarding packet, it will be delivered to the next node channel according to the saved forwarding line; the last one is the protocol type field, which is two bytes in total, indicating that there are multiple application layer protocols such as SDP, SIP, RTP, RTCP, RTSP, etc., and the client delivers it to the corresponding protocol processing module according to the protocol type. The above private protocol is adopted to ensure information security in protocol interaction.

[0067] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A browser-based real-time communication system based on P2MP technology, characterized in that: It includes a signaling server, a participant page client built on a browser, and its module units that make up the page client: The participant page client constructed based on the browser includes a signaling interaction module, a streaming media module, an audio and video module, a network transmission module and a secure access service; The secure access service includes a module for registering and authenticating with the signaling server using a specific private protocol, a page client using a specific mutual trust method, and an authentication module for performing permission management on other business modules. The steps for the page client to initiate registration and authentication include: obtaining browser information, obtaining user device permissions, entering an account and password to complete login, establishing a gRPC channel, creating a group, obtaining a group sharing code or joining a group through a sharing code, obtaining member information issued by the signaling server, and adding the member's unique identifier to mutual trust; The signaling interaction module is responsible for the interaction of network negotiation and streaming media negotiation. Based on P2MP, it establishes a "tree-shaped" Sub-LSP tunnel from one entry node to multiple destination nodes to provide network communication between participant page clients and negotiation of user basic information. The streaming media module is responsible for encoding and decoding audio and video sources and forwarding video sources based on the gRPC / HTTP / 2.0 protocol; The audio and video processing module provides users with an interface to access the media of hardware devices through the existing HTML5 getUserMediaAPI of the browser. Based on this interface, it accesses the hardware media devices without relying on any browser plug-ins and is compatible with major common browsers. It is responsible for calling the operation permissions of peripherals such as PC cameras and microphones, collecting audio and video data and putting them in the streaming media module, as well as calling the streaming media source in the device hardware playback streaming media module; The network transmission module is responsible for establishing a "tree-shaped" Sub-LSP tunnel to multiple destination nodes according to the network topology result negotiated by the streaming media server, and sending the data loaded in the gRPC / HTTP / 2.0 protocol track in the streaming media module. For the media data from other participants that needs to be forwarded and delivered, it relies on the node Sub-LSP tunnel to forward in a point-to-point or point-to-multipoint manner. The step of establishing the "tree-shaped" Sub-LSP tunnel includes: accessing the STUN server of the signaling server through the STUN protocol, and obtaining its own external network IP and port, using the gRPC channel between the signaling server and the participant page client to send the external network IP and port of the members in the group, and sending it to all members by binding the participant's unique identifier and IP address through a private protocol. After the participant page client obtains the external network IP and unique identifier of other participants, it establishes trust rules, and creates a gRPC connection to each participant in the group according to the obtained external network IP and port of other participants in the external network, forming the "tree-shaped" Sub-LSP tunnel; The signaling server deployed on the public network implements basic permission verification based on secure access and uses open shortest path first (OSPF) to provide dynamic network connection construction, including the election mode of transit nodes when the network is poor and the re-generated link construction mechanism. The signaling server includes media negotiation management, network negotiation management, and authorization management; The user accesses the signaling server through the page client to verify the authority, negotiates the media protocol within the participant's group through the signaling interaction module, and detects whether the public network node of the participant can establish a valid point-to-point connection through the STUN protocol. After the results are aggregated, the network negotiation management of the signaling server is unified to connect the tree-shaped Sub-LSP tunnel in an open shortest path first manner; P2MP streaming media data transmission is achieved through the coordination and cooperation of media negotiation management, network negotiation management, and authorization management; The OSPF protocol is an extension of the intranet routing protocol. Based on the network conditions of each participant establishing a point-to-point channel, it reasonably allocates relay forwarding paths and retains the priority sequence for subsequent election paths for network switching. When the participant's network fluctuates, it can actively or manually switch channels by the user to achieve effective utilization of network bandwidth. Based on the signaling server, a web management terminal will be provided to monitor the server, provide management of multiple groups and the network connection of all participants, and virtually monitor the media stream within the group as a participant.

2. The browser-based real-time communication system based on P2MP technology according to claim 1, characterized in that: The client system is implemented based on the page front-end. The client system is developed based on HTML5, transparently transmitted through gRPC, compatible with its various signaling protocols, byte stream protocols, package disassembly and assembly processing, and compatible with hardware calls of various browsers.

3. The browser-based real-time communication system based on P2MP technology according to claim 2, characterized in that: Based on the client system implemented on the front end of the page, a private protocol based on HTML5 is developed with gRPC as the channel carrier to be compatible with various streaming protocols; After logging in to the signaling server for authentication, users can form their own group members; Establish multi-terminal connections based on STUN service management, use the duplex streaming mode with gRPC channels between browsers as the carrier, and encapsulate the streaming media protocol in a private protocol encryption manner. The private protocol will carry the group identifier and user identifier to control permissions. The client will also interact with the signaling server through the gRPC private protocol, and regularly report its network status with other participants to facilitate the subsequent tunnel election of the signaling server. Support for signaling control commands based on streaming media protocols; When a gRPC channel is established between participants, multiple media protocols can be transmitted simultaneously and gRPC connection resources can be reused; When the client detects that the gRPC channel with a participant is disconnected and cannot be reconnected directly, it will actively initiate a connection reconstruction command to the signaling server, and the signaling server will issue the new address or relay address of the other end to the client and re-establish a new gRPC tunnel; When the client detects that the network condition is poor, it will actively initiate a command to establish a backup connection to the signaling server. The server will build a new Sub-LSP tunnel through election. When the tunnel is established, a new gRPC channel will be established. When the data in the original gRPC channel is sent, it will switch to the newly constructed gRPC channel to continue sending subsequent data, so as to achieve flexible network switching.

4. The browser-based real-time communication system based on P2MP technology according to claim 2, characterized in that: The client system based on the page front-end realizes the network forwarding function autonomously: when the participants cannot achieve point-to-point network penetration through the STUN protocol, the server will coordinate and use the network transmission module of the front-end page client to realize TURN data forwarding; that is, each participant includes a TURN server, which is selected by the signaling server according to the current network negotiation status between the participants, and one or more participants are designated to act as relay servers in a Sub-LSP tunnel, and forward data using the TURN protocol.

5. The browser-based real-time communication system based on P2MP technology according to claim 2, characterized in that: The STUN protocol or TURN protocol, and other SIP streaming protocols are encapsulated according to private protocols, using the data channel established by the gRPC protocol as a carrier to facilitate unified reception and parsing.

6. The browser-based real-time communication system based on P2MP technology according to claim 1, characterized in that: The steps of establishing a Sub-LSP tunnel include: Use the page client to log in and authenticate; Obtain user browser information and bandwidth information; The signaling server uses the STUN protocol to negotiate with group members and try to establish a point-to-point data channel. Report to the signaling server based on whether the network is established successfully or the network delay status after successful establishment; The signaling server builds the Sub-LSP tunnel through the OSPF protocol and completes the network topology establishment.

7. The browser-based real-time communication system based on P2MP technology according to claim 3, characterized in that: The browser client in the group will establish at least C(2, N) Sub-LSP tunnels and a signaling interaction channel with the signaling server, where N is the participant in the group; When a forwarding request needs to be undertaken, a new gRPC connection needs to be established as a forwarding channel; that is, the one or more gRPC channels between participant A and participant B are Sub-LSP tunnels, which will actively establish a backup channel when the network fluctuates, and abandon the previous tunnel after the backup channel is put into use.

8. The browser-based real-time communication system based on P2MP technology according to claim 3, characterized in that: The client implemented by the front end of the page will initiate network status reporting through the gRPC channel established with the signaling server at regular intervals. When the server summarizes the network status of the participants in the group, it will send a command to the selected client to establish a server based on the TURN protocol or to establish a backup channel. After successful execution, the client returns the result to the server, and the server will send a network negotiation command again, which will specify the new IP address and port of the changed participant so as to establish a new gRPC channel through the STUN protocol.

9. The browser-based real-time communication system based on P2MP technology according to claim 3, characterized in that: The private protocol includes two parts: a private header and protocol data content. The private header is divided into 4 bytes of MAGIC custom encapsulation header bytes, 4 bytes of member identifier, 4 bytes of message length, 1 byte of forwarding identifier, and 2 bytes of protocol type. When the external network IP of the participant page client changes during point-to-point data transmission, the member ID issued by the server is used as the unique identification. After the signaling server completes the network negotiation, it will issue the member IDs of all participants in the current group to the page client. When the channel is established and network data is generated, the member ID will be used to determine whether the message is legal or not. If it is illegal, it will be discarded. The message length is 4 bytes, which records the actual length of the protocol data and performs integrity check for the page client to disassemble and assemble the package; The forwarding flag indicates whether the current data packet is forwarded and obtained through the relay module; The protocol type of the last two bytes indicates what type of protocol data is carried in the current channel. According to the protocol type, the subsequent complete protocol data is sent to the corresponding protocol processing module to complete the protocol parsing.

Images