Conference capacity adaptive processing method, server and readable storage medium

Abstract

The application provides a conference capacity adaptive processing method, a server and a readable storage medium, and relates to the technical field of communication. The method comprises the following steps: receiving first address information sent by a media scheduling server; wherein the first address information corresponds to a first media server, and the first media server is allocated by the media scheduling server for a first user in an overload condition of a second media server; a media channel has been established between the second media server and a second user in a conference applied to join by the first user; and updating a mapping relationship between conference users and media servers according to the first address information. The scheme of the application solves the problem of low resource utilization rate in allocating media server resources.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a conference capacity adaptive processing method, a server, and a readable storage medium. Background Technology

[0002] Web Real-Time Communication (WebRTC) allows web applications or websites to establish peer-to-peer connections between browsers without the need for an intermediary, enabling the transmission of video and / or audio streams or other arbitrary data.

[0003] However, in WebRTC-based online audio and video conferencing systems, participants push audio and video streams to a central server, which does not perform decoding or mixing operations but directly distributes them to the servers of other participants.

[0004] However, due to the uncertainty of the number of participants, allocating media server resources may result in low resource utilization in order to avoid situations where users cannot join the meeting. Summary of the Invention

[0005] The purpose of this application is to provide a conference capacity adaptive processing method, server, and readable storage medium to avoid the problem of low resource utilization in media server resource allocation.

[0006] To achieve the above objectives, embodiments of this application provide a conference capacity adaptive processing method, executed by a signaling server, comprising:

[0007] Receive first address information sent by the media scheduling server; wherein, the first address information corresponds to the first media server, the first media server is allocated by the media scheduling server to the first user in the event of overload of the second media server, and a media channel has been established between the second media server and the second user in the conference to which the first user has applied to join;

[0008] Based on the first address information, update the mapping relationship between conference users and media servers.

[0009] Optionally, after receiving the first address information sent by the media scheduling server, the method further includes:

[0010] A concatenation message is sent to the first media server and the second media server. The concatenation message is used to notify the first media server and the second media server to establish a transmission channel, and to notify the second media server to forward the media stream of the conference to the first media server through the transmission channel.

[0011] Optionally, the cascading message includes at least one of the following:

[0012] The IP address and port information of the first media server;

[0013] The IP address and port information of the second media server;

[0014] Information about the third user, who is the audio and video production user of the meeting.

[0015] Optionally, after receiving the first address information sent by the media scheduling server, the method further includes:

[0016] A message queue is created for the first media server, and the message queue is used for signaling interaction between the first media server and the signaling server.

[0017] Optionally, the method further includes:

[0018] When the second user requests to join the meeting, the second address information is obtained; wherein the second address information corresponds to the second media server, which is assigned to the second user by the media scheduling server;

[0019] Based on the second address information, the address information of the media proxy server is fed back to the second user.

[0020] Optionally, the method further includes:

[0021] If the second user is the first user of the conference, a mapping relationship between the conference user and the media server is created based on the second address information.

[0022] To achieve the above objectives, embodiments of this application provide a meeting capacity adaptive processing method, executed by a media scheduling server, comprising:

[0023] In the event of an overload on the second media server, the first address information is determined; wherein the first address information corresponds to the first media server, the first media server is assigned to the first user, and a media channel has been established between the second media server and the second user in the conference that the first user has applied to join.

[0024] Send the first address information to the signaling server.

[0025] Optionally, determining the first address information includes:

[0026] Candidate media servers are determined based on the location information and / or carrier type of the first user;

[0027] Based on the load status of the candidate media servers, the first media server and the first address information corresponding to the first media server are determined.

[0028] Optionally, the method further includes:

[0029] If the second user requests to join the meeting, second address information is determined; wherein, the second address information corresponds to the second media server, which is assigned to the second user;

[0030] Send the second address information to the signaling server.

[0031] To achieve the above objectives, embodiments of this application provide a meeting capacity adaptive processing method, executed by a media server, comprising:

[0032] Receive cascading messages sent by the signaling server;

[0033] According to the cascading message, a transmission channel is established with the target media server; the transmission channel is used to forward the media stream of the conference to the target media server, or to receive the media stream forwarded by the target media server.

[0034] Optionally, the cascading message includes at least one of the following:

[0035] The IP address and port information of the media server;

[0036] The IP address and port information of the target media server;

[0037] Information about the third user, who is the audio and video production user of the meeting.

[0038] Optionally, the method further includes:

[0039] Signaling interaction is performed with the signaling server via a message queue.

[0040] Optionally, the method further includes:

[0041] Receive media streams sent by the media proxy server; or,

[0042] Send the media stream to the media proxy server.

[0043] To achieve the above objectives, embodiments of this application provide a meeting capacity adaptive processing method, executed by a media proxy server, comprising:

[0044] Retrieve the updated mapping relationship between meeting users and media servers;

[0045] Upon receiving the media stream from the first user, the media stream from the first user is forwarded to the first media server according to the mapping relationship;

[0046] Upon receiving a media stream from the first media server, the media stream from the first media server is forwarded to the first user according to the mapping relationship;

[0047] The first media server is allocated to the first user by the media scheduling server in the event of an overload of the second media server, and a media channel has been established between the second media server and the second user in the conference to which the first user has applied to join.

[0048] Optionally, the method further includes:

[0049] Retrieve the mapping relationship between the created meeting users and the media server;

[0050] Upon receiving the media stream from the second user, the media stream from the second user is forwarded to the second media server according to the mapping relationship;

[0051] Upon receiving a media stream from the second media server, the media stream from the second media server is forwarded to the second user according to the mapping relationship.

[0052] To achieve the above objectives, embodiments of this application provide a meeting capacity adaptive processing device, comprising:

[0053] The first receiving module is used to receive first address information sent by the media scheduling server; wherein the first address information corresponds to the first media server, and the first media server is allocated by the media scheduling server to the first user when the second media server is overloaded, and a media channel has been established between the second media server and the second user in the conference that the first user applied to join.

[0054] The first processing module is used to update the mapping relationship between conference users and media servers based on the first address information.

[0055] To achieve the above objectives, embodiments of this application provide a meeting capacity adaptive processing device, comprising:

[0056] The second processing module is used to determine the first address information when the second media server is overloaded; wherein the first address information corresponds to the first media server, and the first media server is allocated to the first user, and a media channel has been established between the second media server and the second user in the conference applied to join by the first user.

[0057] The first sending module is used to send the first address information to the signaling server.

[0058] To achieve the above objectives, embodiments of this application provide a meeting capacity adaptive processing device, comprising:

[0059] The second receiving module is used to receive cascading messages sent by the signaling server;

[0060] The third processing module is used to establish a transmission channel with the target media server according to the cascading message; the transmission channel is used to forward the media stream of the conference to the target media server, or to receive the media stream forwarded by the target media server.

[0061] To achieve the above objectives, embodiments of this application provide a meeting capacity adaptive processing device, comprising:

[0062] The first acquisition module is used to obtain the updated mapping relationship between conference users and media servers;

[0063] The second sending module is used to forward the media stream of the first user to the first media server according to the mapping relationship when the media stream of the first user is received.

[0064] The third sending module is used to forward the media stream from the first media server to the first user according to the mapping relationship when the media stream from the first media server is received.

[0065] The first media server is allocated to the first user by the media scheduling server in the event of an overload of the second media server, and a media channel has been established between the second media server and the second user in the conference to which the first user has applied to join.

[0066] To achieve the above objectives, embodiments of this application provide a server, including: a transceiver, a processor, a memory, and a program or instructions stored in the memory and executable on the processor; characterized in that, when the processor executes the program or instructions, it implements the aforementioned conference capacity adaptive processing method executed by a signaling server, or the aforementioned conference capacity adaptive processing method executed by a media scheduling server, or the aforementioned conference capacity adaptive processing method executed by a media server, or the aforementioned conference capacity adaptive processing method executed by a media proxy server.

[0067] To achieve the above objectives, embodiments of this application provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the aforementioned conference capacity adaptive processing method executed by a signaling server, or the aforementioned conference capacity adaptive processing method executed by a media scheduling server, or the aforementioned conference capacity adaptive processing method executed by a media server, or the aforementioned conference capacity adaptive processing method executed by a media proxy server.

[0068] The beneficial effects of the above technical solution in this application are as follows:

[0069] In the method of this application embodiment, after the signaling server receives the first address information of the first media server allocated to the first user from the media scheduling server, it can update the mapping relationship between the conference user and the media server. The media proxy server only needs to forward the media stream between the first user and the first media server based on the updated mapping relationship, which avoids the problem of excessive or insufficient allocation of media server resources and improves resource utilization. Attached Figure Description

[0070] Figure 1 This is one of the flowcharts for the meeting capacity adaptive processing method in this application embodiment;

[0071] Figure 2 This is a schematic diagram illustrating the application of the method in the embodiments of this application;

[0072] Figure 3 This is a schematic diagram of a cascading conference room.

[0073] Figure 4 This is a schematic diagram of the conference system structure;

[0074] Figure 5 This is a second flowchart of the meeting capacity adaptive processing method according to an embodiment of this application;

[0075] Figure 6 This is the third flowchart of the meeting capacity adaptive processing method according to an embodiment of this application;

[0076] Figure 7 This is the fourth flowchart of the meeting capacity adaptive processing method according to an embodiment of this application;

[0077] Figure 8 for Figure 1 The corresponding device's module structure diagram;

[0078] Figure 9 for Figure 5 The corresponding device's module structure diagram;

[0079] Figure 10 for Figure 6 The corresponding device's module structure diagram;

[0080] Figure 11 for Figure 7 The corresponding device's module structure diagram;

[0081] Figure 12 This is a structural diagram of the server in an embodiment of this application. Detailed Implementation

[0082] To make the technical problems, technical solutions and advantages of this application clearer, a detailed description will be provided below in conjunction with the accompanying drawings and specific embodiments.

[0083] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0084] In the various embodiments of this application, it should be understood that the sequence number of each process described below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0085] In addition, the terms "system" and "network" are often used interchangeably in this article.

[0086] In the embodiments provided in this application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0087] For ease of understanding, the following describes some aspects of the embodiments of this application:

[0088] In WebRTC-based online audio and video conferencing systems, the following three solutions are typically used:

[0089] 1. The Mesh solution, which involves interconnecting participants in pairs, requires each participant to send video streams to other participants and also to pull audio and video streams from other participants. This results in high bandwidth requirements and obvious drawbacks.

[0090] 2. Using the MCU solution, based on a centralized MCU server, participants push audio and video streams to the central server, which then decodes, mixes, encodes, and distributes them to each participant. This puts a lot of pressure on the server side, and the processing capacity of a single server is limited, so the disadvantages are obvious.

[0091] 3. The SFU (Streaming over Unit) solution, based on a centralized SFU server, involves participants pushing audio and video streams to the central server. The server does not perform decoding or mixing; it directly distributes the streams to other participants, resulting in relatively low server load and moderate bandwidth requirements. However, with signaling and media integrated, scaling becomes impossible when there are too many media streams, failing to meet high concurrency and high availability requirements. Furthermore, the number of participants in an online video conference is uncertain. Allocating too many media server resources leads to low resource utilization, while allocating too few resources results in some users being unable to join the conference. When a client joins the conference room, allocating media server resources based on the requirement of all users turning on audio and video is uncertain. If only a few users turn on audio and video during the actual meeting, the resource utilization of the media server will be too low. Analyzing media server resources based on the requirement of partially turning on audio and video can lead to lag or even disconnections if too many users turn on audio and video during the actual meeting.

[0092] like Figure 1 As shown, an embodiment of this application provides a conference capacity adaptive processing method, executed by a signaling server, comprising:

[0093] Step 101: Receive first address information sent by the media scheduling server; wherein, the first address information corresponds to the first media server, the first media server is allocated by the media scheduling server to the first user when the second media server is overloaded, and a media channel has been established between the second media server and the second user in the conference that the first user applied to join.

[0094] Here, the second media server is assigned by the media scheduling server to a second user in the same conference as the first user. Media streaming between the second user and the second media server does not overload the second media server. In the event of overload on the second media server, the media scheduling server determines the first address information, which is the address information of the first media server. The media scheduling server will assign the first media server to the first user, determine the first address information, and send it to the signaling server. The signaling server will then receive the first address information corresponding to that first media server.

[0095] Step 102: Update the mapping relationship between conference users and media servers based on the first address information.

[0096] Here, the mapping relationship between conference users and media servers includes the correspondence between the user's address information and the address information of the media server (the media server assigned to the user by the media scheduling server).

[0097] In this step, after receiving the first address information in step 101, the signaling server can update the mapping relationship between conference users and media servers based on the first address information, so that the media proxy server can forward the media stream based on the updated mapping relationship.

[0098] Thus, following the steps described above, after receiving the first address information sent by the media scheduling server, the signaling server can update the mapping relationship between conference users and media servers. The media proxy server only needs to forward the media stream between the first user and the first media server based on the updated mapping relationship, which avoids the problem of excessive or insufficient allocation of media server resources and improves resource utilization.

[0099] Here, "user" can be understood as "user terminal" (UE) or "client," and "conference" refers to an online video conference.

[0100] It should be understood that in this embodiment, the second user can be a user who has already joined the meeting before the first user applies to join the meeting. That is, the first user applies to join the meeting again, resulting in too many users joining the meeting, which exceeds the maximum load capacity of the second media server and causes the second media server to overload. Alternatively, the first user can be a user who has already joined the meeting, but since the second media server is already overloaded, the first user starts other meeting functions, causing the second media server to overload.

[0101] In this embodiment, media stream forwarding between the user and the media server is handled by a media proxy server. The media proxy server acts as an intermediary between the client and the media server, implementing UDP port convergence for the client, exposing only one receive and one send port. Thus, when the media proxy server forwards media streams according to the updated mapping relationship between conference users and media servers, the client remains unaware, improving the user experience.

[0102] Optionally, the media proxy server obtains the updated mapping relationship between conference users and media servers; upon receiving the media stream of the first user, it forwards the media stream of the first user to the first media server according to the mapping relationship; upon receiving the media stream of the first media server, it forwards the media stream of the first media server to the first user according to the mapping relationship.

[0103] In this embodiment, the media scheduling server employs a strategy of assigning users of the same meeting to the same media server. When a media server becomes overloaded, the media scheduling server will assign a new media server to a newly requested user based on the load status of other media servers. Alternatively, the media scheduling server may first determine candidate media servers based on the location information and / or carrier type of the newly requested user. For example, if the user is a user of carrier A in city x, the media scheduling server will determine candidate media servers deployed within city x that support carrier A; then, based on the load status of the candidate media servers, a new media server will be assigned to the user from among the candidate servers.

[0104] Furthermore, in this embodiment, the media scheduling server is able to know the load status of multiple pre-configured media servers for adaptive allocation of media servers.

[0105] Optionally, after receiving the first address information sent by the media scheduling server, the method further includes:

[0106] A concatenation message is sent to the first media server and the second media server. The concatenation message is used to notify the first media server and the second media server to establish a transmission channel, and to notify the second media server to forward the media stream of the conference to the first media server through the transmission channel.

[0107] In other words, after receiving the first address information, the signaling server sends a concatenation message to both the first and second media servers to notify them to establish a transmission channel and to instruct the second media server to forward the conference media stream to the first media server through this transmission channel. Thus, after receiving the conference media stream, the first media server can send it to the first user. If the first user generates a conference media stream, the first media server will also forward it to the second media server so that the second user can access it.

[0108] In this process, after receiving the cascading message sent by the signaling server, the first media server and the second media server establish a transmission channel with each other based on the cascading message.

[0109] Optionally, the cascading message includes at least one of the following:

[0110] The IP address and port information of the first media server;

[0111] The IP address and port information of the second media server;

[0112] Information about the third user, who is the audio and video production user of the meeting.

[0113] The port information refers to the Internet Protocol (IP) port information. The third-party user's information includes identification information, address information, etc.

[0114] Optionally, after receiving the first address information sent by the media scheduling server, the method further includes:

[0115] A message queue is created for the first media server, and the message queue is used for signaling interaction between the first media server and the signaling server.

[0116] Here, message queues can be used for signaling interactions such as Session Description Protocol (SDP), connection operations, and media operations, for example, the cascading messages mentioned above.

[0117] In this way, the signaling server can provide signaling services to users, such as signaling services based on the Transmission Control Protocol (TCP). Moreover, after learning that the media scheduling server has assigned a first media server to the first user, it creates a message queue for that user to ensure signaling interaction between the signaling server and the first media server. This allows for flexible deployment of signaling and media streams, enabling elastic scaling and achieving the goals of high availability and high concurrency.

[0118] Of course, the signaling server is not limited to creating message queues for the first media server. The signaling server will create message queues independently for the assigned media servers, such as creating a message queue for the second media server to interact with the signaling server.

[0119] In this embodiment, the media scheduling server determines the second address information when a second user requests to join the conference and sends it to the signaling server. That is, for users within the maximum load capacity of the second media server (i.e., the second user), the media scheduling server will allocate a second media server to them according to the policy of allocating the same media server for the same conference, and send their address information (i.e., the second address information) to the signaling server.

[0120] Therefore, optionally, in this embodiment, the method further includes:

[0121] When the second user requests to join the meeting, the second address information is obtained; wherein the second address information corresponds to the second media server, which is assigned to the second user by the media scheduling server;

[0122] Based on the second address information, the address information of the media proxy server is fed back to the second user.

[0123] That is, when the signaling server receives the second address information, it will send the corresponding media proxy server address information back to the second user so that the second user can interact with the second media server through the media proxy server.

[0124] It's important to understand that after receiving a user's conference request, the signaling server forwards it to the media scheduling server. Upon receiving the request, the media scheduling server assigns a media server to the user. Here, "user" includes both the first user and the second user. Of course, for the first user's conference request, after receiving the first address information, the signaling server will also provide the first user with the address information of the media proxy server. Since the first user and the second user are in the same conference, the media proxy server is the same. Thus, the media proxy server only needs to expose one receive port and one send port for User Datagram Protocol (UDP) port convergence with the user.

[0125] When the signaling server receives a meeting request from a user (the first user and the second user), it will authenticate the user (such as by token verification). After successful authentication, it will forward the meeting request (HTTP interface request) to the media scheduling server. The media scheduling server will respond to the meeting request and send the address information of the media server assigned to the user, such as IP address and port information.

[0126] Alternatively, in this embodiment, the method further includes:

[0127] If the second user is the first user of the conference, a mapping relationship between the conference user and the media server is created based on the second address information.

[0128] Here, the mapping relationship between conference users and media servers is created, recording the correspondence between the first user of the conference and the second media server. Of course, for other users of the second user, this mapping relationship will also include their corresponding correspondence with the second media server. Thus, after the media scheduling server assigns the second media server to the first user, as long as the second media server is not overloaded, the media proxy servers will forward the second user's media stream according to the address information of the second media server.

[0129] Of course, the first user to send a meeting request will include a meeting ID so that the signaling server can create a meeting room and initialize the room's media stream forwarding capabilities.

[0130] In this embodiment, the mapping relationship between conference users and media servers created by the signaling server is written into a preset file. The media proxy server can read the mapping relationship from the preset file to complete the transmission of the media stream.

[0131] For example, in the method of this application embodiment, when an online audio and video conference is held, the default allocation strategy of the media server is that users in the same conference are assigned to the same media server 1. When the number of people joining the conference is too large, causing the maximum load capacity of a media server to be exceeded, the signaling server can request a new media server 2 from the media scheduling server. The users who join later can connect to the new media server 2, create a room with the same conference ID on the media server 2, and create a host mirror image on the media server 1 on the media server 2. That is, the media server 1 forwards the host's audio and video stream to the media server 2, and the users connected to the media server 2 pull the audio and video stream of the host mirror image, thereby achieving the purpose of expanding the conference capacity.

[0132] The application of the method in the embodiments of this application will be described below with reference to specific scenarios:

[0133] like Figure 2 , Figure 3 As shown, in step 1: the client (user B's client) sends a conference application request to the signaling server. The signaling server verifies the user's token and, after successful verification, sends an HTTP interface request to the media scheduling server to allocate media server resources for the newly joined conference user.

[0134] Step 2: The media scheduling server returns the address information of available media servers, such as IP addresses, based on the conference number and the current load of the media server cluster.

[0135] Step 3: After receiving the media server's IP address, the signaling server creates a message queue for the media server.

[0136] Step 4: The client sends a request signaling to create a media receiving channel. The signaling server forwards it to the message queue created in Step 3. After the media server subscribes to the signaling, it listens on a UDP port and replies to the signaling server through the message queue. The signaling server then creates a mapping relationship between the conference user (client ufrag) and the media server.

[0137] Step 5: After the client receives the address information of the media proxy server returned by the signaling server, it establishes an ICE connection media channel. After receiving the STUN packet, the media proxy server reads the mapping relationship between the conference user (client ufrag) and the media server created by the signaling server in step 4, and establishes the mapping relationship between the client's IP port and the media server's IP port.

[0138] Step 6: The client retrieves the media streams of users who have opened audio and video in the room.

[0139] Step 7: The client turns on the camera device and simultaneously sends a request to create and send a media channel to the signaling server. Since the media server it is on exceeds the maximum load, the signaling server receives the request and requests the media scheduling server to allocate media server resources through the HTTP interface. Then, it executes steps similar to steps 2, 3, 4, and 5 in sequence. In the case of overload of the current media server, a new media server is allocated to the client.

[0140] In this way, migrating user B from the media server to another available media server is seamless. Since the IP port of the media proxy server remains unchanged for the client, only the mapping table between the client and the media server used by the media proxy server and the related objects need to be modified and reconstructed. This is imperceptible to the client.

[0141] In this embodiment, the user (e.g., client), signaling server, media scheduling server, media server, and media proxy server constitute the conferencing system, such as... Figure 4 As shown, where,

[0142] Signaling Server: When a user joins a conference, they first authenticate with the signaling server. If it's the first user, the server needs to create a conference room using the provided conference ID and initialize the room's media forwarding capabilities. Next, it requests (e.g., sends a conference request) the media scheduling server to allocate a media server for the user. If it's a new media server, a message queue is created for that server, used for signaling interactions between the signaling server and the media server, including SDP, connection operations, and media operations. If the conference room has too many participants and requires multiple media servers to support the conference, the signaling server is also responsible for notifying multiple media servers to cascade (e.g., sending cascade messages), forwarding the ports, audio / video producer user information, etc., needed for media cascading. After cascading, the multimedia servers can forward user audio and video streams to each other.

[0143] Media server: Communication between users and the media server is based on UDP. The client obtains the IP address and port information of the media server for direct communication with the help of the signaling server and the media scheduling server. The audio and video streams produced by the client are pushed to the media server, and then the media server forwards them to other users in the conference room.

[0144] Media scheduling server: After all media servers start up, they register with the media scheduling server and periodically report their current load information to the media scheduling server. When a user joins a meeting room, the system allocates a suitable media server to the meeting user based on the media server currently in use in the room and the real-time load information of the media servers in the cluster.

[0145] Media proxy server: Acting as an intermediary between the client and the media server, it implements UDP port convergence for the client, exposing only one receive and one send port; and enables the mutual forwarding of streaming media data between the client and the media server.

[0146] like Figure 5 As shown, an adaptive meeting capacity processing method according to an embodiment of this application is executed by a media scheduling server, including:

[0147] Step 501: In the event of an overload on the second media server, determine the first address information; wherein the first address information corresponds to the first media server, the first media server is assigned to the first user, and a media channel has been established between the second media server and the second user in the conference that the first user applied to join.

[0148] Step 502: Send the first address information to the signaling server.

[0149] Thus, through steps 501-502, after receiving the first address information sent by the media scheduling server, the signaling server can update the mapping relationship between conference users and media servers. The media proxy server only needs to forward the media stream between the first user and the first media server based on the updated mapping relationship, which avoids the problem of excessive or insufficient allocation of media server resources and improves resource utilization.

[0150] Optionally, determining the first address information includes:

[0151] Candidate media servers are determined based on the location information and / or carrier type of the first user;

[0152] Based on the load status of the candidate media servers, the first media server and the first address information corresponding to the first media server are determined.

[0153] Optionally, the method further includes:

[0154] If the second user requests to join the meeting, second address information is determined; wherein, the second address information corresponds to the second media server, which is assigned to the second user;

[0155] Send the second address information to the signaling server.

[0156] It should be noted that this method is used in conjunction with the method executed by the signaling server described above. The conference capacity adaptive processing method executed by the signaling server described above is applicable to this method and can achieve the same technical effect.

[0157] like Figure 6 As shown, an embodiment of this application provides a meeting capacity adaptive processing method, executed by a media server, comprising:

[0158] Step 601: Receive the cascading message sent by the signaling server;

[0159] Step 602: Establish a transmission channel with the target media server according to the cascading message; the transmission channel is used to forward the media stream of the conference to the target media server, or to receive the media stream forwarded by the target media server.

[0160] Here, the cascading message is sent by the signaling server after receiving the address information of the newly assigned media server (the media server assigned by the media scheduling server in the event that the original media server of the conference is overloaded).

[0161] Thus, through steps 601-602, the media server can respond to the signaling server and establish a transmission channel with the target media server, enabling media stream transmission between the newly allocated media server and the original media server. Upon receiving the aforementioned first address information, the signaling server will send a concatenation message to both the first and second media servers to notify them of the establishment of a transmission channel and to instruct the second media server to forward the conference media stream to the first media server through this transmission channel. In this way, after receiving the conference media stream, the first media server can send it to the first user. If the first user generates a conference media stream, the first media server will also forward it to the second media server so that the second user can access it.

[0162] Optionally, the cascading message includes at least one of the following:

[0163] The IP address and port information of the media server;

[0164] The IP address and port information of the target media server;

[0165] Information about the third user, who is the audio and video production user of the meeting.

[0166] Optionally, the method further includes:

[0167] Signaling interaction is performed with the signaling server via a message queue.

[0168] Optionally, the method further includes:

[0169] Receive media streams sent by the media proxy server; or,

[0170] Send the media stream to the media proxy server.

[0171] It should be noted that this method is used in conjunction with the method executed by the signaling server described above. The conference capacity adaptive processing method executed by the signaling server described above is applicable to this method and can achieve the same technical effect.

[0172] like Figure 7 As shown, an adaptive meeting capacity processing method according to an embodiment of this application is executed by a media proxy server, including:

[0173] Step 701: Obtain the updated mapping relationship between meeting users and media servers;

[0174] Step 702: Upon receiving the media stream from the first user, forward the media stream from the first user to the first media server according to the mapping relationship;

[0175] Step 703: Upon receiving the media stream from the first media server, forward the media stream from the first media server to the first user according to the mapping relationship;

[0176] The first media server is allocated to the first user by the media scheduling server in the event of an overload of the second media server, and a media channel has been established between the second media server and the second user in the conference to which the first user has applied to join.

[0177] Thus, through steps 701-703 above, when the media proxy server forwards media streams according to the updated mapping relationship between conference users and media servers after the signaling server receives the first address information sent by the media scheduling server, the client can be unaware of the changes, thereby improving the user experience.

[0178] Optionally, the method further includes:

[0179] Retrieve the mapping relationship between the created meeting users and the media server;

[0180] Upon receiving the media stream from the second user, the media stream from the second user is forwarded to the second media server according to the mapping relationship;

[0181] Upon receiving a media stream from the second media server, the media stream from the second media server is forwarded to the second user according to the mapping relationship.

[0182] It should be noted that this method is used in conjunction with the method executed by the signaling server described above. The conference capacity adaptive processing method executed by the signaling server described above is applicable to this method and can achieve the same technical effect.

[0183] like Figure 8 As shown in the figure, an adaptive meeting capacity processing device according to an embodiment of this application includes:

[0184] The first receiving module 810 is used to receive first address information sent by the media scheduling server; wherein, the first address information corresponds to the first media server, the first media server is allocated by the media scheduling server to the first user when the second media server is overloaded, and a media channel has been established between the second media server and the second user in the conference applied to join by the first user.

[0185] The first processing module 820 is used to update the mapping relationship between conference users and media servers based on the first address information.

[0186] Optionally, the device further includes:

[0187] The fourth sending module is used to send a concatenation message to the first media server and the second media server. The concatenation message is used to notify the first media server and the second media server to establish a transmission channel, and to notify the second media server to forward the media stream of the conference to the first media server through the transmission channel.

[0188] Optionally, the cascading message includes at least one of the following:

[0189] The IP address and port information of the first media server;

[0190] The IP address and port information of the second media server;

[0191] Information about the third user, who is the audio and video production user of the meeting.

[0192] Optionally, the device further includes:

[0193] The message queue creation module is used to create a message queue for the first media server, and the message queue is used for signaling interaction between the first media server and the signaling server.

[0194] Optionally, the device further includes:

[0195] The second acquisition module is used to acquire second address information when the second user requests to join the meeting; wherein the second address information corresponds to the second media server, and the second media server is assigned to the second user by the media scheduling server;

[0196] The fifth sending module is used to send the address information of the media proxy server back to the second user based on the second address information.

[0197] Optionally, the device further includes:

[0198] The fourth processing module is used to create a mapping relationship between conference users and media servers based on the second address information, when the second user is the first user of the conference.

[0199] The apparatus of this embodiment, after receiving the first address information sent by the media scheduling server, can update the mapping relationship between conference users and media servers. The media proxy server only needs to forward the media stream between the first user and the first media server based on the updated mapping relationship, which avoids the problem of excessive or insufficient allocation of media server resources and improves resource utilization.

[0200] This device is an apparatus that applies the method executed by the signaling server described above. The implementation of the above method embodiments is applicable to this device and can achieve the same technical effect.

[0201] like Figure 9 As shown in the figure, an adaptive meeting capacity processing device according to an embodiment of this application includes:

[0202] The second processing module 910 is used to determine the first address information when the second media server is overloaded; wherein the first address information corresponds to the first media server, the first media server is allocated for the first user, and a media channel has been established between the second media server and the second user in the conference that the first user applied to join.

[0203] The first sending module 920 is used to send the first address information to the signaling server.

[0204] Optionally, the second processing module is further configured to:

[0205] Candidate media servers are determined based on the location information and / or carrier type of the first user;

[0206] Based on the load status of the candidate media servers, the first media server and the first address information corresponding to the first media server are determined.

[0207] Optionally, the device further includes:

[0208] The fifth processing module is used to determine the second address information when the second user requests to join the meeting; wherein the second address information corresponds to the second media server, and the second media server is assigned to the second user;

[0209] The sixth sending module is used to send the second address information to the signaling server.

[0210] The device sends the first location information to the signaling server, enabling the signaling server to update the mapping relationship between conference users and media servers after receiving the first address information sent by the media scheduling server. The media proxy server only needs to forward the media stream between the first user and the first media server based on the updated mapping relationship, thus avoiding the problem of excessive or insufficient allocation of media server resources and improving resource utilization.

[0211] This device is an apparatus that applies the method executed by the media scheduling server described above. The implementation of the above method embodiments is applicable to this device and can achieve the same technical effect.

[0212] like Figure 10 As shown in the figure, an adaptive meeting capacity processing device according to an embodiment of this application includes:

[0213] The second receiving module 1010 is used to receive cascading messages sent by the signaling server;

[0214] The third processing module 1020 is used to establish a transmission channel with the target media server according to the cascading message; the transmission channel is used to forward the media stream of the conference to the target media server, or to receive the media stream forwarded by the target media server.

[0215] Optionally, the cascading message includes at least one of the following:

[0216] The IP address and port information of the media server;

[0217] The IP address and port information of the target media server;

[0218] Information about the third user, who is the audio and video production user of the meeting.

[0219] Optionally, the device further includes:

[0220] The sixth processing module is used to interact with the signaling server via a message queue.

[0221] Optionally, the device further includes:

[0222] The media stream sending and receiving module is used to receive media streams sent by the media proxy server; or,

[0223] Send the media stream to the media proxy server.

[0224] This device can respond to a signaling server and establish a transmission channel with a target media server to enable media stream transmission between the newly assigned media server and the original media server. Upon receiving the aforementioned first address information, the signaling server will send a concatenation message to both the first and second media servers to notify them of the establishment of a transmission channel and to instruct the second media server to forward the conference media stream to the first media server through this transmission channel.

[0225] This device is an apparatus that applies the method executed by the media server described above. The implementation of the above method embodiments is applicable to this device and can achieve the same technical effect.

[0226] like Figure 11 As shown in the figure, an adaptive meeting capacity processing device according to an embodiment of this application includes:

[0227] The first acquisition module 1110 is used to acquire the updated mapping relationship between conference users and media servers;

[0228] The second sending module 1120 is used to forward the media stream of the first user to the first media server according to the mapping relationship when the media stream of the first user is received.

[0229] The third sending module 1130 is used to forward the media stream of the first media server to the first user according to the mapping relationship when the media stream of the first media server is received.

[0230] The first media server is allocated to the first user by the media scheduling server in the event of an overload of the second media server, and a media channel has been established between the second media server and the second user in the conference to which the first user has applied to join.

[0231] Optionally, the method further includes:

[0232] The third acquisition module is used to obtain the mapping relationship between the created conference users and the media server;

[0233] Upon receiving the media stream from the second user, the media stream from the second user is forwarded to the second media server according to the mapping relationship;

[0234] The seventh sending module is used to forward the media stream from the second media server to the second user according to the mapping relationship when the media stream from the second media server is received.

[0235] This device enables clients to remain unaware of the updated mapping relationship between conference users and media servers after the signaling server receives the first address information sent by the media scheduling server, and to forward media streams according to the updated mapping relationship, thereby improving the user experience.

[0236] This device is an apparatus that applies the method executed by the media proxy server described above. The implementation of the above method embodiments is applicable to this device and can achieve the same technical effect.

[0237] One embodiment of this application provides a server, such as... Figure 12 As shown, it includes a transceiver 1210, a processor 1200, a memory 1220, and a program or instructions stored in the memory 1220 and executable on the processor 1200; when the processor 1200 executes the program or instructions, it implements the conference capacity adaptive processing method executed by the signaling server, or the conference capacity adaptive processing method executed by the media scheduling server, or the conference capacity adaptive processing method executed by the media server, or the conference capacity adaptive processing method executed by the media proxy server.

[0238] The transceiver 1210 is used to receive and send data under the control of the processor 1200.

[0239] Among them, Figure 12 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits together, including one or more processors represented by processor 1200 and memory represented by memory 1220. The bus architecture can also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 1210 can be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium.

[0240] The processor 1200 is responsible for managing the bus architecture and general processing, while the memory 1220 can store the data used by the processor 1200 when performing operations.

[0241] This application provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the steps of the conference capacity adaptive processing method executed by the signaling server, the conference capacity adaptive processing method executed by the media scheduling server, the conference capacity adaptive processing method executed by the media server, or the conference capacity adaptive processing method executed by the media proxy server, as described above, and achieve the same technical effect. To avoid repetition, these steps will not be repeated here.

[0242] The processor mentioned above is the processor in the server described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0243] It should be further noted that many of the functional components described in this specification are referred to as modules in order to emphasize the independence of their implementation.

[0244] In this embodiment, the module can be implemented in software so that it can be executed by various types of processors. For example, an identified executable code module may include one or more physical or logical blocks of computer instructions, which may be constructed as objects, procedures, or functions. Nevertheless, the executable code of the identified module does not need to be physically located together, but may include different instructions stored in different bits, which, when logically combined, constitute the module and achieve the module's intended purpose.

[0245] In practice, an executable code module can be a single instruction or many instructions, and can even be distributed across multiple different code segments, different programs, and across multiple memory devices. Similarly, operational data can be identified within the module and can be implemented in any suitable form and organized within any suitable type of data structure. This operational data can be collected as a single dataset or distributed across different locations (including different storage devices), and can exist, at least in part, solely as electronic signals within the system or network.

[0246] When a module can be implemented using software, considering the level of hardware technology, modules that can be implemented in software can be implemented using hardware circuits by those skilled in the art to achieve the corresponding functions, without considering cost. These hardware circuits include conventional very large-scale integrated circuits (VLSI) or gate arrays, as well as semiconductors such as logic chips and transistors, or other discrete components. Modules can also be implemented using programmable hardware devices, such as field-programmable gate arrays, programmable array logic, and programmable logic devices.

[0247] The exemplary embodiments described above are with reference to the accompanying drawings. Many different forms and embodiments are feasible without departing from the spirit and teachings of this application. Therefore, this application should not be construed as limiting the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided to make this application complete and convey the scope of this application to those skilled in the art. In these drawings, component dimensions and relative dimensions may be exaggerated for clarity. The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. As used herein, unless clearly indicated otherwise, the singular forms “a,” “an,” and “the” are intended to include all such forms. It will be further understood that the terms “comprising” and / or “including”, when used in this specification, indicate the presence of the stated features, integers, steps, operations, components, and / or elements, but do not exclude the presence or addition of one or more other features, integers, steps, operations, components, and / or groups thereof. Unless otherwise indicated, when stated, a range of values ​​includes the upper and lower limits of the range and any subranges in between.

[0248] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for adaptive processing of meeting capacity, characterized in that, Performed by the signaling server, including: Receive first address information sent by the media scheduling server; wherein, the first address information corresponds to the first media server, the first media server is allocated by the media scheduling server to the first user in the event of overload of the second media server, and a media channel has been established between the second media server and the second user in the conference to which the first user has applied to join; Update the mapping relationship between conference users and media servers based on the first address information; In this process, media streams are forwarded between conference users and media servers through a media proxy server, which forwards media streams according to the updated mapping relationship between conference users and media servers. The method further includes, after receiving the first address information sent by the media scheduling server: Send a concatenation message to the first media server and the second media server. The concatenation message is used to notify the first media server and the second media server to establish a transmission channel, and to notify the second media server to forward the media stream of the conference to the first media server through the transmission channel. The cascading messages include at least one of the following: The IP address and port information of the first media server; The IP address and port information of the second media server; Information about the third user, who is the audio and video production user of the meeting.

2. The method according to claim 1, characterized in that, After receiving the first address information sent by the media scheduling server, the method further includes: A message queue is created for the first media server, and the message queue is used for signaling interaction between the first media server and the signaling server.

3. The method according to claim 1, characterized in that, The method further includes: When the second user requests to join the meeting, the second address information is obtained; wherein the second address information corresponds to the second media server, which is assigned to the second user by the media scheduling server; Based on the second address information, the address information of the media proxy server is fed back to the second user.

4. The method according to claim 3, characterized in that, The method further includes: If the second user is the first user of the conference, a mapping relationship between the conference user and the media server is created based on the second address information.

5. A method for adaptive processing of meeting capacity, characterized in that, Performed by the media scheduling server, including: In the event of an overload on the second media server, the first address information is determined; wherein the first address information corresponds to the first media server, the first media server is assigned to the first user, and a media channel has been established between the second media server and the second user in the conference that the first user has applied to join. The signaling server sends the first address information to the signaling server, so that the signaling server updates the mapping relationship between conference users and media servers based on the first address information. After receiving the first address information from the media scheduling server, the signaling server also sends a concatenation message to the first media server and the second media server. The concatenation message is used to notify the first media server and the second media server to establish a transmission channel, and to notify the second media server to forward the conference media stream to the first media server through the transmission channel. The media stream forwarding between conference users and media servers is performed through a media proxy server, which forwards the media stream according to the updated mapping relationship between conference users and media servers. The cascading messages include at least one of the following: The IP address and port information of the first media server; The IP address and port information of the second media server; Information about the third user, who is the audio and video production user of the meeting.

6. A method for adaptive processing of meeting capacity, characterized in that, Executed by the media server, including: Receive cascading messages sent by the signaling server; According to the cascading message, a transmission channel is established with the target media server; the transmission channel is used to forward the media stream of the conference to the target media server, or to receive the media stream forwarded by the target media server; The method further includes: Receive a media stream sent by a media proxy server; or send a media stream to the media proxy server; wherein the media stream sent by the media proxy server is a media stream sent according to the updated mapping relationship between conference users and media servers; The cascading messages include at least one of the following: The IP address and port information of the media server; The IP address and port information of the target media server; Information about the third user, who is the audio and video production user of the meeting.

7. A method for adaptive processing of meeting capacity, characterized in that, Performed by the media proxy server, including: Retrieve the updated mapping relationship between meeting users and media servers; Upon receiving the media stream from the first user, the media stream from the first user is forwarded to the first media server according to the mapping relationship; Upon receiving a media stream from the first media server, the media stream from the first media server is forwarded to the first user according to the mapping relationship; The first media server is assigned to the first user by the media scheduling server in the event of an overload on the second media server. A media channel has been established between the second media server and the second user in the conference to which the first user has applied to join. Before sending the media stream to the media proxy server, the first media server receives a concatenation message from the signaling server. The concatenation message is used to notify the first media server and the second media server to establish a transmission channel, and to notify the second media server to forward the media stream of the conference to the first media server through the transmission channel. The concatenation message includes at least one of the following: the IP address and port information of the first media server; the IP address and port information of the second media server; and information of the third user, who is the audio and video production user of the conference.

8. A server, comprising: A transceiver, a processor, a memory, and a program or instructions stored in the memory and executable on the processor; characterized in that, when the processor executes the program or instructions, it implements the meeting capacity adaptive processing method as described in any one of claims 1 to 4, or the meeting capacity adaptive processing method as described in claim 5, or the meeting capacity adaptive processing method as described in claim 6, or the meeting capacity adaptive processing method as described in claim 7.

9. A readable storage medium having a program or instructions stored thereon, characterized in that, When the program or instructions are executed by the processor, they implement the steps of the meeting capacity adaptive processing method as described in any one of claims 1 to 4, or the meeting capacity adaptive processing method as described in claim 5, or the meeting capacity adaptive processing method as described in claim 6, or the meeting capacity adaptive processing method as described in claim 7.

Images