Multi-party audio and video call method and device

Abstract

The invention relates to the technical field of core network communication, in particular to a multi-party audio and video call method and device. The multi-party audio and video call method is applied to a control plane, and comprises the following steps: receiving a multi-party call request sent by a calling terminal, the multi-party call request carrying an original session identifier of a current session of the calling terminal; in response to the multi-party call request, determining a session state of a current session of the calling terminal; determining whether the current session needs to be expanded based on the multi-party call request and the session state; and if the capacity expansion is needed, retaining the original session identifier of the current session based on the current session and the multi-party call request, and generating a new session context. According to the invention, the current session does not need to be hung up, the current session is upgraded to support the multi-party call, and the user experience is improved.

Description

Technical Field

[0001] This disclosure relates to the field of core network communication technology, and in particular to a method and apparatus for multi-party audio and video calls. Background Technology

[0002] With the rapid popularization of smart mobile terminals and the high-speed development of network communication technology, people's communication methods have gradually evolved from the earliest text-based communication to real-time communication using audio and video. Point-to-point (P2P) calling mode has become a widely used communication solution in the industry due to its advantages such as simple architecture and low latency. This mode can meet the needs of one-to-one real-time communication and effectively guarantee the basic call experience, thus enabling its widespread application.

[0003] However, end-to-end calls cannot be upgraded to multi-party calls online. If an upgrade to a multi-party call is required, the terminal calls need to re-establish a connection, which severely impacts the user experience. Therefore, how to achieve online upgrades to support multi-party calls based on P2P calls has become a pressing technical problem that needs to be solved. Summary of the Invention

[0004] In view of the above-mentioned technical problems, this disclosure provides a method and apparatus for multi-party audio and video calls.

[0005] In a first aspect, this disclosure provides a method for multi-party audio and video calls, applied to the control plane, the method including: Receive a multi-party call request sent by the calling terminal. The multi-party call request carries the original session identifier of the calling terminal's current session. In response to a multi-party call request, determine the session state of the calling terminal's current session; Based on the multi-party call requests and session status, determine whether the current session needs to be expanded; If expansion is required, based on the current session and multi-party call requests, the original session identifier of the current session is retained, and a new session context is generated.

[0006] Furthermore, according to the method of the first aspect of this disclosure, determining whether the current session needs to be expanded based on multi-party call requests and session state includes: Based on multi-party call requests, obtain the first terminal device that is conducting audio and video calls with the calling terminal; Based on the first terminal device and the calling terminal, a set of terminal devices is generated; When the session status is a two-party session, the set of participating terminals in the current session is determined based on the original session identifier; Based on the participating terminal set and terminal device set, determine whether the current session needs to be expanded.

[0007] Furthermore, according to the method of the first aspect of this disclosure, the method also includes: If the number of terminal devices in the terminal device cluster is greater than the number of participating terminals in the participating terminal cluster, it is determined that the current session needs to be expanded.

[0008] Furthermore, according to the method of the first aspect of this disclosure, based on the current session and the multi-party call request, the original session identifier of the current session is retained, and a new session context is generated, including: Based on multi-party call requests, determine the new participating terminals required to expand the current session; Based on the original session context of the current session and the new participating terminal, the original session identifier of the current session is retained, and a new session context is generated.

[0009] Furthermore, according to the method of the first aspect of this disclosure, the method also includes: If the session status is a two-way call, determine the target media plane; In response to a multi-party call request, a first routing control instruction is generated. The first routing control instruction includes the identifier of each terminal device and an execution action instruction. The execution action instruction is used to instruct the target media to send a media stream aggregation invitation to each terminal device in the terminal device set. Send the first routing control command to the target media plane; This enables each terminal device in the terminal device cluster to respond to the media stream aggregation invitation sent by the target media plane based on the first routing control command, and send its corresponding target audio and video streams to the target media plane.

[0010] Furthermore, according to the method of the first aspect of this disclosure, the media stream aggregation invitation includes media stream rules, which are used to instruct terminal devices to send target audio and video at a uniform bitrate.

[0011] Furthermore, according to the method of the first aspect of this disclosure, the media flow rules are determined by the target media surface based on the media flow standards of the two-party call.

[0012] Furthermore, according to the method of the first aspect of this disclosure, the method also includes: When each terminal device receives a media stream aggregation invitation, it responds to the confirmation response sent by each terminal device and establishes a multi-party call.

[0013] Furthermore, according to the method of the first aspect of this disclosure, the method also includes: When at least one first target terminal device in the terminal device group is detected to be in a weak network state, the target bitrate of the audio and video streams of all terminal devices in the terminal device group is determined based on the network capabilities and current bitrate of all terminal devices in the terminal device group.

[0014] Furthermore, according to the method of the first aspect of this disclosure, the method also includes: Based on the target bitrate of the audio and video streams of all terminal devices, generate target control instructions; The target control command is sent to the target media plane so that the target media plane responds to the target control command by performing layered transcoding on the audio and video containing the target bitrate received from all terminal devices, generating a target aggregate stream containing a base layer and an enhancement layer, sending the base layer to the first target terminal device, and sending the target aggregate stream to other terminal devices excluding the first target terminal device.

[0015] Furthermore, according to the method of the first aspect of this disclosure, the method includes: Receive an exit request sent by the second target terminal device in the terminal device set; In response to the exit request, release the control plane resources corresponding to the second target terminal device.

[0016] Furthermore, according to the method of the first aspect of this disclosure, the method includes: Send a resource reclamation command to the target media surface, causing the target media surface to respond to the resource reclamation command and release the media surface resources corresponding to the second target terminal device.

[0017] Furthermore, according to the method of the first aspect of this disclosure, the method also includes: If the session status is at least a three-way call, a second routing control instruction is generated in response to a multi-way call request; Send the second routing control command to the first target media plane corresponding to the current session; To enable the new participating terminal to respond to the target media stream aggregation invitation sent by the first target media plane based on the second routing control command, and send its corresponding first target audio and video stream to the first target media plane.

[0018] Secondly, this disclosure provides a multi-party audio and video calling device for use in a control plane, the device comprising: The receiving module is used to receive multi-party call requests sent by the calling terminal. The multi-party call request carries the original session identifier of the calling terminal's current session. The first determining module is used to determine the session state of the calling terminal's current session in response to a multi-party call request. The second determining module is used to determine whether the current session needs to be expanded based on the multi-party call request and the session status. The generation module is used to generate a new session context based on the current session and multi-party call requests, while retaining the original session identifier of the current session, if expansion is required.

[0019] Furthermore, according to the apparatus of the second aspect of this disclosure, the second determining module is also configured to: Based on multi-party call requests, obtain the first terminal device that is conducting audio and video calls with the calling terminal; Based on the first terminal device and the calling terminal, a set of terminal devices is generated; Based on the session state, determine the set of participating terminals in the current session; Based on the participating terminal set and terminal device set, determine whether the current session needs to be expanded.

[0020] Furthermore, according to the apparatus of the second aspect of this disclosure, the apparatus further includes: If the number of terminal devices in the terminal device cluster is greater than the number of participating terminals in the participating terminal cluster, it is determined that the current session needs to be expanded.

[0021] Furthermore, according to the apparatus of the second aspect of this disclosure, the generation module is also used for: Based on multi-party call requests, determine the new participating terminals required to expand the current session; Based on the original session context of the current session and the new participating terminal, the original session identifier of the current session is retained, and a new session context is generated.

[0022] Furthermore, according to the apparatus of the second aspect of this disclosure, the apparatus is also used for: If the session status is a two-way call, determine the target media plane; In response to a multi-party call request, a first routing control instruction is generated. The first routing control instruction includes the identifier of each terminal device and an execution action instruction. The execution action instruction is used to instruct the target media to send a media stream aggregation invitation to each terminal device in the terminal device set. Send the first routing control command to the target media plane; This enables each terminal device in the terminal device cluster to respond to the media stream aggregation invitation sent by the target media plane based on the first routing control command, and send its corresponding target audio and video streams to the target media plane.

[0023] Furthermore, according to the apparatus of the second aspect of this disclosure, the media stream aggregation invitation includes media stream rules, which are used to instruct terminal devices to send target audio and video at a uniform bitrate.

[0024] Furthermore, according to the apparatus of the second aspect of this disclosure, the media streaming rules are determined by the target media surface based on the media streaming standards of the two-party call.

[0025] Furthermore, according to the apparatus of the second aspect of this disclosure, the apparatus is also used for: When each terminal device receives a media stream aggregation invitation, it responds to the confirmation response sent by each terminal device and establishes a multi-party call.

[0026] Furthermore, according to the apparatus of the second aspect of this disclosure, the apparatus is also used for: When at least one first target terminal device in the terminal device group is detected to be in a weak network state, the target bitrate of the audio and video streams of all terminal devices in the terminal device group is determined based on the network capabilities and current bitrate of all terminal devices in the terminal device group.

[0027] Furthermore, according to the apparatus of the second aspect of this disclosure, the apparatus is also used for: Based on the target bitrate of the audio and video streams of all terminal devices, generate target control instructions; The target control command is sent to the target media plane so that the target media plane responds to the target control command by performing layered transcoding on the audio and video containing the target bitrate received from all terminal devices, generating a target aggregate stream containing a base layer and an enhancement layer, sending the base layer to the first target terminal device, and sending the target aggregate stream to other terminal devices excluding the first target terminal device.

[0028] Furthermore, according to the apparatus of the second aspect of this disclosure, the apparatus is also used for: Receive an exit request sent by the second target terminal device in the terminal device set; In response to the exit request, release the control plane resources corresponding to the second target terminal device.

[0029] Furthermore, according to the apparatus of the second aspect of this disclosure, the apparatus is also used for: Send a resource reclamation command to the target media surface, causing the target media surface to respond to the resource reclamation command and release the media surface resources corresponding to the second target terminal device.

[0030] Furthermore, according to the apparatus of the second aspect of this disclosure, the apparatus is also used for: If the session status is at least a three-way call, a second routing control instruction is generated in response to a multi-way call request; Send the second routing control command to the first target media plane corresponding to the current session; To enable the new participating terminal to respond to the target media stream aggregation invitation sent by the first target media plane based on the second routing control command, and send its corresponding first target audio and video stream to the first target media plane.

[0031] Thirdly, this disclosure provides an electronic device, including: a memory for storing computer-readable instructions; and a processor for executing the computer-readable instructions, causing the electronic device to perform the method as described in any embodiment of the first aspect.

[0032] Fourthly, this disclosure provides a non-transitory computer-readable storage medium for storing computer-readable instructions that, when executed by a processor, cause the processor to perform the method as described in any embodiment of the first aspect.

[0033] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any embodiment of the first aspect.

[0034] This disclosure provides a method and apparatus for multi-party audio and video calls. The method and apparatus receive multi-party call requests sent by the calling terminal through the control plane, determine the session state of the current session in which the calling terminal is currently participating, and determine whether the current session needs to be expanded based on the session state and the multi-party call request. If expansion is required, a new session context is generated with the original session identifier of the current session as the new session identifier. This allows the current session to be upgraded to support multi-party calls without hanging up, thereby improving the user experience.

[0035] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0036] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0037] Figure 1 This is a schematic diagram of the architecture of a communication system provided in an embodiment of the present disclosure; Figure 2 A flowchart illustrating a multi-party audio and video call method provided in this embodiment of the disclosure; Figure 3 A flowchart illustrating yet another multi-party audio and video call method provided in this disclosure embodiment; Figure 4 This is a schematic diagram of the interaction flow of a multi-party audio and video call method provided in an embodiment of the present disclosure; Figure 5 A schematic diagram of the interaction flow of another multi-party audio and video call method provided in this embodiment of the disclosure; Figure 6 A schematic diagram of the interaction flow of another multi-party audio and video call method provided in this embodiment of the disclosure; Figure 7 This is a structural block diagram of a multi-party audio and video calling device provided in an embodiment of the present disclosure; Figure 8 A hardware block diagram of an electronic device provided in an embodiment of this disclosure; Figure 9 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this disclosure. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this disclosure more apparent, exemplary embodiments according to this disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments of this disclosure. It should be understood that this disclosure is not limited to the exemplary embodiments described herein.

[0039] First, refer to Figure 1 Overview of application scenarios according to embodiments of this disclosure.

[0040] Figure 1 This is a schematic diagram of the architecture of a communication system provided in an embodiment of this disclosure. This communication system is applicable to the multi-party audio and video call method provided in this disclosure.

[0041] like Figure 1 As shown, the communication system includes at least: a control plane 101, a media plane 102, multiple terminal devices (e.g., terminal 1031, terminal 1032 and terminal 1033) and a billing system 104.

[0042] It is understood that the control plane 101 includes a control center 1011 and capability network elements 1012. The media plane 102 includes a dynamic media anchoring module 1021, a media processor 1022, and a resource pool 1023.

[0043] The control plane 101, through the control center 1011 and capability network element 1012, receives and parses multi-party call requests, detects session status and expansion necessity, and generates a new session context that retains the original session identifier. Simultaneously, it dynamically selects the target media plane, issues routing and rate control commands, monitors terminal network status and triggers a global weak network adaptation strategy, receives terminal exit requests, and coordinates with the media plane 102 to complete the atomic release of resources from both the control plane 101 and media plane 102, thus leading the strategy formulation and state management of the entire session process. The media plane 102, through the dynamic media anchoring module 1021, media processor 1022, and resource pool 1023, responds to the control plane 101 commands by receiving audio and video streams from all terminals. It determines unified media stream rules based on the two-party call media stream standard and verifies terminal stream specifications. Through a multi-stream aggregation engine, it performs mixing, noise reduction, and layered encoding. According to the control plane strategy, it distributes the basic layer to weak network terminals and the complete aggregated stream to strong network terminals. Simultaneously, when a terminal exits, it reclaims the corresponding media resources and reassembles the mixing, achieving efficient convergence, processing, and differentiated distribution of media streams. The billing system 104 is used to receive QoS data (latency / stuttering rate) submitted by the control plane.

[0044] For example, the control center 1011 issues a "policy instruction" to the capability network element 1012. The capability network element 1012 transmits an "anchoring control" instruction to the dynamic media anchoring module 1021 of the media plane 102, a "bitrate adjustment" instruction to the media processor 1022, and a "resource adjustment" instruction to the resource pool 1023 according to the "policy instruction", thereby realizing the direct control of the media plane 102 by the policy of the control plane 101.

[0045] Understandably, the control plane described above is the "decision and management module" for 5G calls, primarily responsible for handling call signaling information. The media plane is the "data transmission module" for 5G calls, specifically responsible for transmitting the actual audio and video.

[0046] This disclosure provides a method for multi-party audio and video calls. Please refer to [reference needed]. Figure 2 , Figure 2 This is a flowchart illustrating a multi-party audio / video call method provided in an embodiment of this disclosure. Figure 2 As shown, this method is executed by the control plane and specifically includes the following steps: Step 201: Receive a multi-party call request sent by the calling terminal. The multi-party call request carries the original session identifier of the calling terminal's current session.

[0047] In one embodiment of this disclosure, when a primary terminal wants to make calls with multiple terminal devices simultaneously, it sends a multi-party call request to the control plane, informing the control plane that a multi-party call needs to be established. The control plane receives the multi-party call request, which carries the original session identifier of the calling terminal's current session.

[0048] Step 202: In response to a multi-party call request, determine the session state of the calling terminal's current session.

[0049] In one embodiment of this disclosure, after receiving the multi-party call request, the control plane responds to the request and determines the session state of the current session in which the calling terminal is currently participating by using the original session identifier of the current session. The session state refers to whether the current session is a two-party call or a three-party or more-party call.

[0050] Step 203: Based on the multi-party call requests and session status, determine whether the current session needs to be expanded.

[0051] In one embodiment of this disclosure, determining whether the current session needs to be expanded based on multi-party call requests and session state includes: Based on multi-party call requests, obtain the first terminal device that is conducting audio and video calls with the calling terminal; Based on the first terminal device and the calling terminal, a set of terminal devices is generated; When the session status is a two-party session, the set of participating terminals in the current session is determined based on the original session identifier; Based on the participating terminal set and terminal device set, determine whether the current session needs to be expanded.

[0052] If the number of terminal devices in the terminal device cluster is greater than the number of participating terminals in the participating terminal cluster, it is determined that the current session needs to be expanded.

[0053] In one embodiment of this disclosure, the multi-party call request includes the original session identifier of the current session in which the calling terminal is currently participating, and also includes the identifier information of all terminal devices that want to conduct a multi-party call (i.e., an audio / video call). The identifier information of all terminal devices is obtained by parsing the multi-party call request. Using the identifier information of all terminal devices, the first terminal device that will conduct the audio / video call with the calling terminal is obtained. A terminal device set is generated by combining the first terminal device and the calling terminal. If the current session state is a two-way call, the existing participating terminal set is searched by parsing the original session identifier obtained from the multi-party call request. By comparing the number of terminal devices in the terminal device set with the number of participating terminals in the participating terminal set, it is determined whether the current session needs to be expanded. If the number of terminal devices in the terminal device set is greater than the number of participating terminals in the participating terminal set, it is determined that the current session needs to be expanded.

[0054] Step 204: If expansion is required, generate a new session context based on the current session's multi-party call request, using the original session identifier of the current session as the new session identifier.

[0055] In one embodiment of this disclosure, a new session context is generated based on the current session and a multi-party call request, using the original session identifier of the current session as the new session identifier, including: Based on multi-party call requests, determine the new participating terminals required to expand the current session; Based on the original session context of the current session and the new participating terminal, a new session context is generated, which uses the original session identifier of the current session as the new session identifier.

[0056] In one embodiment of this disclosure, when the current session needs to be expanded, the current session identifier obtained by parsing the multi-party call request is used to determine the set of participating terminals for the current session, and the set of terminal devices that need to establish a multi-party call with the calling terminal is obtained by parsing the multi-party call request. By comparing the set of participating terminals and the set of terminal devices, the new participating terminals required for the expansion of the current session are determined. While retaining the original session identifier of the current session, a new session context is generated by combining the original session context and the new participating terminals, while retaining the original session identifier, using the original session identifier of the current session as the new session identifier.

[0057] For example, the multi-party call request includes the identification information of the new participating terminal. When expansion is required, the original session's Context ID (original session identifier) ​​is retained, and a corresponding communication connection address (callURL) is generated for the new participating terminal. The corresponding communication connection address generated by the new participating terminal is added to the negotiation parameters of the current session. Finally, a new session context is generated, which contains the callURLs of all participants and is uniquely identified by the original session's Context ID.

[0058] In summary, according to the technical solution provided in the embodiments of this disclosure, this disclosure receives a multi-party call request sent by the calling terminal through the control plane, determines the session state of the current session in which the calling terminal is currently participating, and determines whether the current session needs to be expanded based on the session state and the multi-party call request. If expansion is required, a new session context is generated with the original session identifier of the current session as the new session identifier, so as to upgrade the current session to support multi-party calls without hanging up, thereby improving the user experience.

[0059] Figure 3 This is a flowchart illustrating yet another multi-party audio / video call method provided in an embodiment of this disclosure, as shown below. Figure 3 As shown, the multi-party audio and video call method also includes the following steps: Step 301: If the conversation status is a two-way call, determine the target media plane.

[0060] In one embodiment of this disclosure, if the session is a two-way call, metrics such as network quality and resource load of each node are collected, a weighted score is calculated, and the node with the highest overall score is selected as the target media plane. Here, a node refers to an independent computing unit with media stream processing capabilities.

[0061] Step 302: In response to the multi-party call request, generate a first routing control instruction. The first routing control instruction includes the identifier of each terminal device and the execution action instruction. The execution action instruction is used to instruct the target media to send a media stream aggregation invitation to each terminal device in the terminal device cluster.

[0062] In one embodiment of this disclosure, a first routing control instruction is generated based on the received multi-party call request. The first routing control instruction includes the identifier of each terminal device and an execution action instruction. The execution action instruction is used to instruct the target media to send a media stream aggregation invitation to each terminal device in the terminal device set.

[0063] Step 303: Send the first routing control command to the target media plane.

[0064] In one embodiment of this disclosure, the control plane sends a first routing control command to the target media plane.

[0065] Step 304: Each terminal device in the terminal device cluster responds to the media stream aggregation invitation sent by the target media plane based on the first routing control command, and sends its corresponding target audio and video streams to the target media plane. Specifically, the target media plane first parses the first routing control command, extracts the terminal identifier, and then generates an aggregation invitation with stream label rules and sends it to the terminal. When each terminal device sends the target audio and video stream, it will embed the label into the header of the media stream according to the rules, making the link of stream label generation, carrying, and identification clearer.

[0066] In one embodiment of this disclosure, the media stream aggregation invitation includes media stream rules, which are used to instruct the terminal device to send target audio and video at a uniform bitrate.

[0067] In one embodiment of this disclosure, the media streaming rules are determined by the media plane based on the media streaming standards of the two-party call.

[0068] In one embodiment of this disclosure, the media stream aggregation invitation includes media stream rules, which instruct terminal devices to send target audio and video at a uniform bitrate. The target media plane parses the received first routing control command to obtain the identifiers of each terminal device and execution action commands included in the first routing control command. Based on the execution action commands, it generates a media stream aggregation invitation containing the media stream rules. According to the identifiers of each terminal device, the media stream aggregation invitation is sent to each terminal device in the terminal device set, causing each terminal device in the terminal device set to respond to the received media stream aggregation invitation. By parsing the media stream aggregation invitation, it sends the target audio and video stream conforming to the media stream rules to the target media plane. The media stream rules are determined by the control plane based on the media stream standard of the two-party call and are used to force the media streams of all terminal devices to be consistent with the global settings of the current session. Furthermore, the target media plane performs mixing, noise reduction, and layered encoding processing on the received target audio and video streams.

[0069] In summary, according to the technical solution provided in the embodiments of this disclosure, by selecting a target media plane and issuing a first routing control command to the media plane, the media plane sends a media stream aggregation invitation to the terminal device according to the first routing control command, so that the terminal device sends the target audio and video stream that conforms to the media stream rules to the media plane, realizing a seamless online upgrade from two-party call to multi-party call. This not only preserves the original session identifier and avoids reconstruction interruption, but also compresses the traditional O(n²) level terminal direct connection path into an O(n) level centralized transmission path through dynamic media anchoring technology, greatly reducing port resource consumption. At the same time, relying on the media stream rules formulated by the control plane, it ensures that the media stream specifications of all terminals are consistent, avoiding high-specification streams from impacting the current session.

[0070] Furthermore, the aforementioned multi-party audio and video call method also includes: receiving confirmation responses from each terminal device in response to the media stream aggregation invitation, and establishing a multi-party call.

[0071] Specifically, when each terminal device receives a media stream aggregation invitation sent by the target media plane, it sends an confirmation response to the control plane upon receiving the media stream aggregation invitation and confirming that it can be executed. After the control plane collects the confirmation responses from all terminal devices, it establishes a multi-party call.

[0072] Furthermore, the aforementioned methods for multi-party audio and video calls also include: When at least one first target terminal device in the terminal device group is detected to be in a weak network state, the target bitrate of the audio and video streams of all terminal devices in the terminal device group is determined based on the network capabilities and current bitrate of all terminal devices in the terminal device group.

[0073] Specifically, when at least one target terminal device in the terminal device cluster is detected to be in a weak network state, the control plane triggers global dynamic bitrate negotiation. Based on the network capabilities and current bitrates of all terminal devices in the cluster, the target bitrate for the audio and video streams of all terminal devices is uniformly determined. All terminal devices then encode their audio and video streams according to the target bitrate before sending them to the target media plane.

[0074] Furthermore, the aforementioned methods for multi-party audio and video calls also include: Based on the target bitrate of the audio and video streams of all terminal devices, generate target control instructions; The target control command is sent to the target media plane so that the target media plane responds to the target control command by performing layered transcoding on the audio and video containing the target bitrate received from all terminal devices, generating a target aggregate stream containing a base layer and an enhancement layer, sending the base layer to the first target terminal device, and sending the target aggregate stream to other terminal devices excluding the first target terminal device.

[0075] Specifically, the control plane determines the target bitrate of the audio and video streams from all terminal devices, generates a target control command, and informs the target media plane to initiate layered transcoding. The target media plane then performs layered transcoding on the received audio and video streams from all terminal devices at the target bitrate, generating a target aggregated stream containing a base layer and an enhancement layer. The base layer is then sent to the first target terminal device. Finally, the target aggregated stream is sent to the other terminal devices excluding the first target terminal device.

[0076] Furthermore, the aforementioned methods for multi-party audio and video calls also include: Receive an exit request sent by the second target terminal device in the terminal device set; In response to the exit request, release the control plane resources corresponding to the second target terminal device.

[0077] Specifically, the second target terminal refers to the device in the terminal device group that needs to exit the multi-party call. The control plane receives the exit request sent by the second target terminal device, parses the exit request, obtains the identifiers of the multi-party audio and video involved by the second terminal device, and releases the control plane resources corresponding to the second target terminal based on the identifiers of the multi-party audio and video. Furthermore, the above multi-party audio and video call method also includes: sending a resource reclamation command to the target media plane, causing the target media plane to respond to the resource reclamation command and release the media plane resources corresponding to the second target terminal device.

[0078] Specifically, after releasing the control plane resources corresponding to the second target terminal device, a resource reclamation command is sent to the target media plane. Upon receiving the resource reclamation command, the target media plane releases the media plane resources corresponding to the second target terminal device, such as closing the media stream port, clearing its encoding and decoding context, and reclaiming memory.

[0079] Furthermore, the above-mentioned multi-party audio and video call method also includes: if the session status is at least a three-party call, in response to the multi-party call request, generating a second routing control instruction; Send the second routing control command to the first target media plane corresponding to the current session; To enable the new participating terminal to respond to the target media stream aggregation invitation sent by the first target media plane based on the second routing control command, and send its corresponding first target audio and video stream to the first target media plane.

[0080] Specifically, if the session involves at least three parties, the audio and video of the participating terminals in the current call are processed through the media plane. Based on the multi-party call request, when the control plane determines that expansion is needed based on the current session, it generates a second routing control instruction. This second routing control instruction includes a new terminal device identifier and a target execution action instruction. The target execution action instruction instructs the media plane to send a target media stream aggregation invitation to the new terminal device. The target media stream aggregation invitation includes target media stream rules, which instruct the new participating terminal to send a first target audio and video stream with a uniform bitrate. The target media plane parses the received second routing control instruction to obtain the identifier of the new participating terminal and the target execution action instruction. Based on the target execution action instruction, it generates a target media stream aggregation invitation containing the target media stream rules. Based on the identifier of the new participating terminal, it sends the target media stream aggregation invitation to the new participating terminal, causing the new participating terminal to respond to the received target media stream aggregation invitation. By parsing the target media stream aggregation invitation, it sends the first target audio and video stream conforming to the target media stream rules to the first target media plane. The target media stream rule is determined by the control plane based on the media stream standard of the two-party call, and is used to force the media stream of newly participating terminals to be consistent with the global settings of the current session. After completing the above operations, the control plane synchronizes with all terminal devices: the execution requirements of the target media stream rule are: all audio and video streams subsequently sent by all terminal devices (including newly participating terminals) must conform to the target media stream rule and be sent to the first target media plane.

[0081] In summary, according to the technical solution provided in the embodiments of this disclosure, this disclosure achieves unified management and control of media stream parameters within a session during the expansion phase of a multi-party audio and video call session. This enables rapid adaptation of the media stream between new participating terminals and the current session, avoiding audio and video stuttering, asynchrony, or access failure caused by mismatched media stream parameters when a new terminal joins. Furthermore, by synchronizing unified media stream execution rules to all terminals in the session, the consistency of audio and video transmission parameters across all terminals is ensured after session expansion. This provides a unified foundation for subsequent operations such as weak network layered transcoding, dynamic resource scheduling, and terminal exit and recycling, effectively maintaining the transmission stability, smoothness, and scheduling efficiency of multi-party audio and video calls.

[0082] The following is with reference to the appendix. Figure 4 -Appendix Figure 6 This disclosure provides a detailed description of the multi-party audio and video call method provided in the embodiments of this disclosure.

[0083] Figure 4 This is a schematic diagram of the interaction flow of a multi-party audio and video call method provided in an embodiment of this disclosure.

[0084] like Figure 4 As shown, Figure 4In the multi-party audio and video call method provided in this disclosure, there is a process of interaction between the control plane, the media plane, and the calling terminal. Among them, the service AS, VoNR, and VoTLE_AS belong to the control plane.

[0085] Specifically, Figure 4 Taking the creation of a multi-party call as an example: The core capability is to force all terminals to route audio and video streams to a unified media processing node (i.e., the media plane) through dynamic media anchoring commands, thereby achieving a seamless conversion from two-party calls to multi-party conferences (i.e., multi-party audio and video calls). The unified media policy (resolution / bitrate / encryption protocol) and the maximum number of access parties are dynamically limited through a dynamic policy decision module of a layered and decoupled architecture (this solution reduces port occupancy by 80% through centralized processing of the media plane).

[0086] The process of interaction between the control plane, media plane, and the calling terminal includes the following steps: 1. The calling terminal initiates a multi-party request to the service AS.

[0087] Specifically, the calling terminal initiates a multi-party request (i.e., a multi-party audio and video call request) to the service AS through a mini-program.

[0088] 2. The service AS forwards the multi-party creation request to VoNR.

[0089] Specifically, the service AS will forward the Create Multiple Request received from the calling terminal to the VoNR.

[0090] 3. VoNR performs policy verification.

[0091] Specifically, VoNR collects network quality, resource load, and other metrics from each node for weighted scoring. During the selection of the optimal computing power node, node availability is verified; if a node exceeds its current resource capacity, it is unavailable. The node with the highest overall score is selected as the optimal computing power node (i.e., the target media plane). A weighted scoring algorithm comprehensively evaluates candidate nodes across three core dimensions: processing capability (efficient encoding), network latency (speed), and scenario adaptability (coverage range). This ensures carrier-grade quality while avoiding resource overload, enforces a maximum of nine parties, and maintains unified encoding and decoding.

[0092] 4. VoNR sends a session splitting command to VoLTE_AS.

[0093] Specifically, after receiving the session splitting instruction, VoLTE_AS splits the two sessions into independent UA sessions, preparing for subsequent multi-stream aggregation on the media plane.

[0094] 5. VoLTE_AS sends a message to the media plane to initiate multi-stream aggregation.

[0095] Specifically, VoLTE_AS sends the start multi-stream aggregation command to the media plane (i.e., the aforementioned target media plane), which is the aforementioned first control command. The media plane starts the aggregation engine to compress the traditional end-to-end redundant transmission (N×(N-1) paths) into single-stream aggregation.

[0096] 6. The media plane sends an SDP renegotiation (forced anchoring) to the calling terminal.

[0097] Specifically, in response to VoLTE_AS sending the Initiate Multi-Stream Aggregation message, the media plane sends an SDP renegotiation to all terminal devices (not limited to the calling terminal) in the "Create Multi-Party Request". By forcing the terminal to modify the media receive / send address to the address of the specified media plane node in the updated SDP message, the centralized anchoring of the media stream path is completed.

[0098] 7. The terminal sends a confirmation to the service AS.

[0099] Specifically, when each terminal device confirms that it can perform SDP renegotiation, it will send an acknowledgment creation (i.e., the aforementioned acknowledgment response) to the service AS. After the service AS collects the acknowledgment creations from all terminal devices, it establishes a multi-party call.

[0100] Figure 5 This is a schematic diagram of the interaction flow of another multi-party audio and video call method provided in an embodiment of the present disclosure.

[0101] like Figure 5 As shown, the multi-party audio and video call method provided in this disclosure involves the interaction between the control plane, the media plane, the calling terminal, and the new terminal. VoNR and VoTLE_AS belong to the control plane.

[0102] Specifically, Figure 5 Taking inviting users to join multiple parties as an example: The core capability is to dynamically add new members to an existing multi-party conference through the stream tagging technology of dynamic media anchoring instructions, and to verify the compliance of media streams (such as resolution / bitrate / encryption protocol) in real time based on the dynamic policy decision module of the layered decoupled architecture, and to force all terminals to update media policies synchronously (traditional end-to-end solutions have a policy effective delay of ≥5 seconds due to the lack of a centralized policy execution node, while this solution has a delay of ≤0.2 seconds).

[0103] The process of interaction between the control plane, media plane, calling terminal, and new terminal includes the following steps: 1. The calling terminal initiates an invitation request (i.e., a multi-party audio and video call request) to VoNR.

[0104] 2. VoNR verifies permissions and the number of remaining squares.

[0105] Specifically, VoNR receives invitation requests and verifies the calling terminal's permissions and the remaining access capacity for multi-party conferencing (i.e., multi-party calls).

[0106] 3. Perform global bitrate synchronization on the media side (force new members to adapt to the existing strategy).

[0107] Specifically, when a new terminal joins an ongoing multi-party conference, the media plane will force the new terminal's media stream specifications (bitrate, resolution) to be consistent with the current global settings of the multi-party conference, rather than allowing the new terminal's high-specification stream to disrupt the multi-party conference or forcing the multi-party conference to accommodate the new terminal. When a new terminal attempts to join a 480P conference with 1080P high-definition video, the media plane will immediately force its video stream to be transcoded down to 480P to match the existing specifications of the multi-party conference.

[0108] 4. Forced invitation strategy (H264 / 480P).

[0109] Specifically, media outlets send mandatory policy invitations (i.e., the aforementioned media stream aggregation invitations) to new terminals.

[0110] 5. The new terminal sends an acknowledgment to VoTLE_AS.

[0111] Specifically, the new terminal sends a confirmation to VoTLE_AS, informing VoTLE_AS that it can execute the forced policy invitation.

[0112] 6. Strategy-synchronized broadcasting.

[0113] Specifically, VoTLE_AS, upon receiving the confirmation of joining, broadcasts the testing policy to all terminals. That is, all subsequent audio and video streams sent by all terminal devices must comply with the mandatory policy invitation and be sent to the media plane.

[0114] Figure 6 This is a schematic diagram of the interaction flow of another multi-party audio and video call method provided in an embodiment of this disclosure. VoNR and Service AS represent the control plane.

[0115] like Figure 6 As shown, Figure 6 Taking the withdrawal of multiple parties as an example: The core capability is to unbind the resources of departing members in real time through the stream tag identification technology of dynamic media anchoring instructions, and dynamically reclaim computing resources based on the resource elastic preemption mechanism of the layered decoupled architecture to maintain the stability of the remaining sessions (traditional end-to-end solutions require manual session reconstruction, which takes ≥8 seconds, while this solution achieves millisecond-level resource release with an interruption time ≤0.1 seconds).

[0116] The process of interaction between the control plane, media plane, and the calling terminal includes the following steps: 1. The terminal sends a BYE signaling message to the service AS.

[0117] Specifically, the terminal sends a BYE command (i.e., exit request) to the service AS, carrying the callURL identifier.

[0118] 2. Business AS triggers resource release.

[0119] Specifically, the service AS captures the BYE signaling and notifies the VoNR to release resources.

[0120] 3. VoNR sends atomized resource recycling to the media surface.

[0121] Specifically, based on the atomic resource reclamation instructions sent by VoNR, the media plane packages the release of media plane resources related to the terminal requesting to exit (closing the media stream port, clearing its encoding and decoding context, reclaiming memory, etc.) into an independent transaction.

[0122] 4. The media side performs a remixing and reassembly of the audio stream.

[0123] Specifically, the media side seamlessly removes the data from the terminal requesting to exit from the mixed audio and video stream. It also temporarily stores any unsent media data using a circular buffer. Upon receiving an exit command, it stops retrieving streams from the exiting terminal.

[0124] 5. Seamless update stream.

[0125] Specifically, the media informed other terminals that only existing media streams were recycled, and no new media streams were added.

[0126] 6. Synchronize resource logs.

[0127] Specifically, after VoNR completes resource release (after VoNR and media plane operations are completed), it records information such as "resource changes during this terminal exit, operation time, and related session identifiers (such as callURL)" in the resource log and synchronizes it with the service AS.

[0128] 7. Final status notification.

[0129] Specifically, the service AS synchronizes the final status of the terminal that applied to withdraw from the multi-party audio and video call to all terminals within the multi-party audio and video call.

[0130] The above is for reference only. Figure 2 -Appendix Figure 6 A detailed description of the multi-party audio and video call method provided according to embodiments of this disclosure reveals that existing point-to-point (P2P) communication schemes generally have the following three drawbacks: 1. Irreversible Session Extension: Two-party calls cannot be upgraded to multi-party calls online and require interruption and reconstruction. This solution establishes a dynamic migration mechanism for the session state machine through multi-party session migration control technology. It retains the original session ContextID in the signaling plane and injects multi-party negotiation parameters to achieve online hot upgrade of two-party calls to multi-party calls.

[0131] 2. Low resource efficiency: New terminals need to establish media streams independently (e.g., 20 streams are needed for a 5-party call), and there is a high risk of exhausting media plane ports; This solution can centrally send and receive media streams through unified processing and dynamic anchoring technology, reducing resource consumption.

[0132] 3. Poor adaptability to weak network conditions: In weak network scenarios, only single-terminal bitrate reduction is supported, while other terminals still send high-definition streams, exacerbating congestion. This solution is based on an intelligent viewing mode. When network quality degradation is detected in any terminal, dynamic bitrate negotiation is triggered simultaneously for all participants, reducing bandwidth utilization compared to traditional solutions.

[0133] 4. This disclosure adopts a layered decoupling architecture and dynamic media anchoring technology. By decoupling the control plane and media plane, it achieves elastic resource expansion and forces the terminal media stream to be routed to the optimal computing power node for centralized processing. This compresses the traditional end-to-end O(n²) level transmission path into an O(n) level single path distribution, solving the core problems of traditional multi-party video communication such as session reconstruction interruption, low resource utilization, and severe lag in weak network conditions. At the same time, it supports dynamic addition and deletion of participants during the call (latency <0.2 seconds), intelligent bitrate adaptation (1080P / 360P layered transcoding), and millisecond-level resource reclamation (interruption ≤0.1 seconds), realizing a carrier-grade low-latency and highly reliable multi-party video communication service.

[0134] 5. Building upon point 4, dynamic media anchoring commands force all terminals to route audio and video streams to the media plane. Combined with intelligent mixing algorithms, multi-path encoding and aggregation of multiple raw streams are performed, optimizing the multi-path transmission between terminals in traditional solutions into single-path aggregation and distribution, eliminating redundant stream transmission. Specifically, the optimal computing node is dynamically selected based on real-time sensing of terminal network status (such as latency and packet loss rate), and cross-session resource sharing is achieved based on stream tag identification, resolving the risk of port resource exhaustion caused by direct terminal connections in traditional solutions and the stuttering problem in weak network scenarios.

[0135] 6. Based on point 4, a dynamic policy decision module is deployed at the service bus layer through a decoupled architecture between the control plane and media plane. This module coordinates network status and resource load data in real time to dynamically adjust encoding / decoding policies and resource allocation rules. A global layered transcoding mechanism is triggered based on terminal network quality grading (e.g., high-quality images are retained for terminals with strong networks, while weak networks adaptively degrade). This solves the problems of low resource utilization and rigid response caused by traditional static policies, significantly improving resource utilization efficiency and service carrying capacity, while ensuring carrier-grade stability across networks and multiple terminals.

[0136] This disclosure also provides a multi-party audio and video calling device. Figure 7 This is a structural block diagram of a multi-party audio and video calling device provided in an embodiment of the present disclosure, such as... Figure 7 As shown, the multi-party audio and video call device 700 includes: a receiving module 701, a first determining module 702, a second determining module 703, and a generating module 704.

[0137] In one exemplary embodiment, the receiving module 701 is configured to receive a multi-party call request sent by the calling terminal, wherein the multi-party call request carries the original session identifier of the current session of the calling terminal. In one exemplary embodiment, the first determining module 702 is configured to determine the session state of the current session of the calling terminal in response to a multi-party call request; In one exemplary embodiment, the second determining module 703 is used to determine whether the current session needs to be expanded based on the multi-party call request and the session state; In one exemplary embodiment, the generation module 704 is used to generate a new session context based on the current session and multi-party call requests, while retaining the original session identifier of the current session, if expansion is required.

[0138] In one exemplary embodiment, the second determining module 703 is further configured to: Based on multi-party call requests, obtain the first terminal device that is conducting audio and video calls with the calling terminal; Based on the first terminal device and the calling terminal, a set of terminal devices is generated; Based on the session state, determine the set of participating terminals in the current session; Based on the participating terminal set and terminal device set, determine whether the current session needs to be expanded.

[0139] In one exemplary embodiment, the apparatus further includes: If the number of terminal devices in the terminal device cluster is greater than the number of participating terminals in the participating terminal cluster, it is determined that the current session needs to be expanded.

[0140] In one exemplary embodiment, the generation module 704 is further configured to: Based on multi-party call requests, determine the new participating terminals required to expand the current session; Based on the original session context of the current session and the new participating terminal, the original session identifier of the current session is retained, and a new session context is generated.

[0141] In one exemplary embodiment, the apparatus is further configured to: If the session status is a two-way call, determine the target media plane; In response to a multi-party call request, a first routing control instruction is generated. The first routing control instruction includes the identifier of each terminal device and an execution action instruction. The execution action instruction is used to instruct the target media to send a media stream aggregation invitation to each terminal device in the terminal device set. Send the first routing control command to the target media plane; This enables each terminal device in the terminal device cluster to respond to the media stream aggregation invitation sent by the target media plane based on the first routing control command, and send its corresponding target audio and video streams to the target media plane.

[0142] In one exemplary embodiment, the media stream aggregation invitation includes media stream rules, which instruct the terminal device to send target audio and video at a uniform bitrate.

[0143] In one exemplary embodiment, the media streaming rules are determined by the target media plane based on the media streaming standards of the two-party call.

[0144] In one exemplary embodiment, the apparatus is further configured to: When each terminal device receives a media stream aggregation invitation, it responds to the confirmation response sent by each terminal device and establishes a multi-party call.

[0145] In one exemplary embodiment, the apparatus is further configured to: When at least one first target terminal device in the terminal device group is detected to be in a weak network state, the target bitrate of the audio and video streams of all terminal devices in the terminal device group is determined based on the network capabilities and current bitrate of all terminal devices in the terminal device group.

[0146] In one exemplary embodiment, the apparatus is further configured to: Based on the target bitrate of the audio and video streams of all terminal devices, generate target control instructions; The target control command is sent to the target media plane so that the target media plane responds to the target control command, encodes the audio and video with the target bit rate received from all terminal devices, generates a target aggregate stream containing a base layer and an enhancement layer, sends the base layer to the first target terminal device, and sends the target aggregate stream to other terminal devices excluding the first target terminal device.

[0147] In one exemplary embodiment, the apparatus is further configured to: Receive an exit request sent by the second target terminal device in the terminal device set; In response to the exit request, release the control plane resources corresponding to the second target terminal device.

[0148] In one exemplary embodiment, the apparatus is further configured to: Send a resource reclamation command to the target media surface, causing the target media surface to respond to the resource reclamation command and release the media surface resources corresponding to the second target terminal device.

[0149] In one exemplary embodiment, the apparatus is further configured to: If the session status is at least a three-way call, a second routing control instruction is generated in response to a multi-way call request; Send the second routing control command to the first target media plane corresponding to the current session; To enable the new participating terminal to respond to the target media stream aggregation invitation sent by the first target media plane based on the second routing control command, and send its corresponding first target audio and video stream to the first target media plane.

[0150] Figure 8 This is a hardware block diagram of an electronic device provided according to an embodiment of the present disclosure. The electronic device 800 according to an embodiment of the present disclosure includes at least a processor; and a memory for storing computer-readable instructions. When the computer-readable instructions are loaded and executed by the processor, the processor performs the multi-party audio / video call method described in any of the preceding embodiments of the present disclosure.

[0151] Figure 8 The illustrated electronic device 800 specifically includes a central processing unit (CPU) 801, a graphics processing unit (GPU) 802, and a memory 803. These units are interconnected via a bus 804. The CPU 801 and / or GPU 802 can function as the aforementioned processor, and the memory 803 can function as the aforementioned memory storing computer-readable instructions. Furthermore, the electronic device 800 may also include a communication unit 805, a storage unit 806, an output unit 807, an input unit 808, and an external device 809, all of which are also connected to the bus 804.

[0152] Figure 9 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this disclosure. (As shown...) Figure 9 As shown, a computer-readable storage medium 900 according to an embodiment of the present disclosure stores computer-readable instructions 901 thereon. When the computer-readable instructions 901 are executed by a processor, the multi-party audio and video call method described with reference to the above figures according to any embodiment of the present disclosure is performed. The computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, optical disk, magnetic disk, etc.

[0153] This disclosure further provides a computer program product, including a computer program that, when executed by a processor, implements the multi-party audio and video call method described in any of the preceding embodiments of this disclosure.

[0154] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0155] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0156] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0157] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0158] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0159] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0160] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0161] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for multi-party audio-video call, characterized in that, The method applied to a control plane comprises: receiving a multi-party call request sent by a calling terminal, the multi-party call request carrying an original session identifier of a current session of the calling terminal; determining a session state of the current session of the calling terminal in response to the multi-party call request; determining whether the current session needs to be expanded based on the multi-party call request and the session state; if the current session needs to be expanded, reserving the original session identifier of the current session and generating a new session context based on the current session and the multi-party call request.

2. The multi-party audio video call method of claim 1, wherein, The method further comprises: if the number of terminal devices in the terminal device set is greater than the number of participating terminals in the participating terminal set, determining that the current session needs to be expanded. The method further comprises: if the session state is a two-party call, determining a target media plane; generating a first routing control instruction in response to the multi-party call request, the first routing control instruction containing an identifier of each terminal device and an execution action instruction, the execution action instruction being used to instruct the target media plane to send a media stream aggregation invitation to each terminal device in the terminal device set; 3. The multi-party audio video call method of claim 2, wherein, sending the first routing control instruction to the target media plane; so that each terminal device in the terminal device set sends its corresponding target audio / video stream to the target media plane in response to the media stream aggregation invitation sent by the target media plane based on the first routing control instruction.

4. The multi-party audio video call method of claim 2, wherein, The media stream aggregation invitation contains a media stream rule, and the media stream rule is used to instruct the terminal device to send a target audio / video with a unified code rate. The media stream rule is determined by the target media plane based on a media stream standard of the two-party call. The method further comprises:

5. The multi-party audio video call method according to claim 4, wherein, establishing a multi-party call in response to an acknowledgement response sent by each terminal device when each terminal device receives the media stream aggregation invitation. The method further comprises: when it is detected that at least one first target terminal device in the terminal device set is in a weak network state, determining a target code rate of audio / video streams of all terminal devices in the terminal device set based on network capabilities and current code rates of all terminal devices in the terminal device set. The method further comprises: ​ 6. The multi-party audio video call method of claim 5, wherein, ​ 7. The multi-party audio video call method of claim 6, wherein, ​ 8. The multi-party audio video call method of claim 6, wherein, ​ ​ 9. The method of multi-party audio-video call according to claim 6, characterized in that, ​ ​ 10. The multi-party audio video call method of claim 9, wherein, ​ generate a target control instruction based on the target bit rate of the audio and video streams of all the terminal devices; send the target control instruction to the target media plane to cause the target media plane to, in response to the target control instruction, transcode the audio and video received from all the terminal devices to generate a target aggregate stream containing a base layer and an enhancement layer, send the base layer to a first target terminal device, and send the target aggregate stream to terminal devices other than the first target terminal device in the terminal device set.

11. The method of multi-party audio-video call according to claim 8, characterized in that, The method comprises: receiving an exit request sent by a second target terminal device in the terminal device set; in response to the exit request, releasing the control plane resource corresponding to the second target terminal device.

12. The multi-party audio video call method of claim 11, wherein, The method comprises: sending a resource recycling instruction to the target media plane to cause the target media plane to, in response to the resource recycling instruction, release the media plane resource corresponding to the second target terminal device.

13. The method of multi-party audio-video call according to claim 4, characterized in that, The method further comprises: if the session state is at least a three-party call, generating a second routing control instruction in response to the multi-party call request; sending the second routing control instruction to a first target media plane corresponding to the current session; causing the new participant terminal to send its corresponding first target audio and video stream to the first target media plane in response to a target media stream aggregation invitation sent by the first target media plane based on the second routing control instruction.

14. A multi-party audio-video call apparatus, characterized by: The apparatus applied to a control plane comprises: a receiving module configured to receive a multi-party call request sent by a calling terminal, the multi-party call request carrying an original session identifier of a current session of the calling terminal; a first determining module configured to determine a session state of the current session of the calling terminal in response to the multi-party call request; a second determining module configured to determine whether the current session needs to be expanded based on the multi-party call request and the session state; a generating module configured to, if the current session needs to be expanded, generate a new session context based on the current session and the multi-party call request, and retain the original session identifier of the current session.

15. The apparatus of claim 14, wherein, The second determining module is further configured to: obtain a first terminal device that performs audio and video communication with the calling terminal based on the multi-party call request; generate a terminal device set based on the first terminal device and the calling terminal; determine a participant terminal set of the current session based on the session state; determine whether the current session needs to be expanded based on the participant terminal set and the terminal device set.

16. The apparatus of claim 15, wherein, The apparatus further comprises: if the number of terminal devices in the terminal device set is greater than the number of participant terminals in the participant terminal set, determine that the current session needs to be expanded.

17. The apparatus of claim 15, wherein, The generating module is further configured to: determine a new participant terminal to be added to the current session based on the multi-party call request; generate a new session context based on the original session context of the current session and the new participant terminal, and retain the original session identifier of the current session.

18. The apparatus of claim 17, wherein, The apparatus is further configured to: if the session state is a two-party call, determine a target media plane; In response to the multi-party call request, a first routing control instruction is generated, the first routing control instruction comprising an identifier of each terminal device and an execution action instruction, the execution action instruction being used to instruct a target media to send a media stream aggregation invitation to each terminal device in the terminal device set; The first routing control instruction is sent to the target media; Each terminal device in the terminal device set sends its corresponding target audio and video stream to the target media in response to the media stream aggregation invitation sent by the target media based on the first routing control instruction.

19. The apparatus of claim 18, wherein, The media stream aggregation invitation comprises a media stream rule, the media stream rule being used to instruct the terminal device to send a target audio and video of a unified code rate.

20. The apparatus of claim 19, wherein, The media stream rule is determined by the target media based on a media stream standard of the two-party call.

21. The apparatus of claim 19, wherein, The apparatus is further configured to: When each terminal device receives the media stream aggregation invitation, a multi-party call is established in response to an acknowledgement response sent by each terminal device.

22. The apparatus of claim 19, wherein, The apparatus is further configured to: When it is detected that at least one first target terminal device in the terminal device set is in a weak network state, a target code rate of audio and video streams of all the terminal devices in the terminal device set is determined based on network capabilities and current code rates of all the terminal devices in the terminal device set.

23. The apparatus of claim 22, wherein, The apparatus is further configured to: A target control instruction is generated based on the target code rate of the audio and video streams of all the terminal devices; The target control instruction is sent to the target media, so that the target media performs layered transcoding on audio and video received from all the terminal devices in response to the target control instruction, generates a target aggregated stream comprising a base layer and an enhancement layer, sends the base layer to the first target terminal device, and sends the target aggregated stream to terminal devices other than the first target terminal device in the terminal device set.

24. The apparatus of claim 21, wherein, The apparatus is further configured to: An exit request sent by a second target terminal device in the terminal device set is received; In response to the exit request, a control plane resource corresponding to the second target terminal device is released.

25. The apparatus of claim 24, wherein, The apparatus is further configured to: A resource recycling instruction is sent to the target media, so that the target media releases a media plane resource corresponding to the second target terminal device in response to the resource recycling instruction.

26. The apparatus of claim 17, wherein, The apparatus is further configured to: If the session state is at least a three-party call, a second routing control instruction is generated in response to the multi-party call request; The second routing control instruction is sent to a first target media corresponding to the current session; The new participating terminal sends its corresponding first target audio and video stream to the first target media in response to a target media stream aggregation invitation sent by the first target media based on the second routing control instruction.

Images