A multi-party call processing method, related device, storage medium and computer product

Abstract

The application discloses a multi-party call processing method, related equipment, a storage medium and a computer product. The method comprises the following steps: receiving first information sent by a first network element, wherein the first information indicates a response event of the first network element to a dialing access number of each terminal participating in a multi-party call, and each terminal comprises an initiator terminal and a participant terminal; sending second information to the first network element, wherein the second information indicates that the first network element anchors a call of each terminal to a second network element; receiving third information sent by the first network element, wherein the third information indicates a call control result sent by the second network element to the first network element; and sending fourth information to an intelligent media processing device, wherein the fourth information is used for notifying a media anchoring result and a mixed flow processing.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a method for processing multi-party calls, related equipment, storage media, and computer products. Background Technology

[0002] Currently, all multi-party calls offered by operators are multi-party voice calls and do not support multi-party video calls. Furthermore, the process of joining a multi-party call requires that when two parties are having a normal conversation, they must first suspend the existing call and then add a call to a third party through the native dial pad. Once connected with the third party, the two one-to-one voice calls can be merged into a single three-way voice call through operations on the native dial pad. There is no way for users to actively join multi-party calls. Summary of the Invention

[0003] This application provides a method for processing multi-party calls, related equipment, storage media, and computer products.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a method for processing multi-party calls, applied to an application server, the method including:

[0006] The first network element receives a first message sent by the first network element, the first message indicating the first network element's response to the dialing of access numbers by each terminal participating in the multi-party call, wherein each terminal includes the initiating terminal and the participating terminal;

[0007] Send a second message to the first network element, the second message instructing the first network element to anchor the calls of each terminal to the second network element;

[0008] Receive third information sent by the first network element, wherein the third information indicates the call control result sent by the second network element to the first network element;

[0009] A fourth message is sent to the intelligent media processing device, the fourth message being used to notify the media anchoring result and the mixing process.

[0010] In the above scheme, the method further includes: receiving a fifth message sent by the initiating terminal, the fifth message instructing the initiating terminal to hang up and exit the multi-party call;

[0011] A sixth message is sent to the second network element, the sixth message instructing the second network element to release endpoint resources.

[0012] The method in the above scheme further includes:

[0013] When each terminal is in the multi-party call, it receives the seventh information sent by the initiating terminal, the seventh information indicating the type of smart service to be activated;

[0014] Send an eighth message to the intelligent media processing device, the eighth message instructing the multi-party call to be processed in accordance with the intelligent service type.

[0015] In the above scheme, the intelligent service type includes one or more of the following:

[0016] Intelligent application business;

[0017] Artificial intelligence business;

[0018] Expand existing business operations.

[0019] This application provides a method for processing multi-party calls, applied to a smart media processing device, the method comprising:

[0020] Receive a fourth message sent by the application server, the fourth message being used to notify the media anchoring result and the mixing process;

[0021] Receive the call media streams from each terminal forwarded by the second network element;

[0022] The various call media streams are mixed and screen-combined to obtain a mixed and screen-combined media stream;

[0023] The mixed audio and screen media stream is sent to the first network element.

[0024] The method in the above scheme further includes:

[0025] The system receives a ninth message sent by the second network element. The ninth message is used to notify one or more of the following events: adjusting the forwarding and mixing / screen merging relationships of the remaining media streams of the remaining terminals other than the initiating terminal; updating the screen display information of the remaining terminals; and updating the list information of the remaining terminals.

[0026] Execute one or more of the events.

[0027] The method in the above scheme further includes:

[0028] Send updated screen display information and / or updated list information to the remaining terminals.

[0029] The method in the above scheme further includes:

[0030] When each terminal is in the multi-party call, it receives the eighth information sent by the application server. The eighth information indicates that the multi-party call should be processed in combination with the intelligent service type.

[0031] Accordingly, the mixing and merging of the various call media streams to obtain a mixed and merged media stream includes:

[0032] The various call media streams are mixed and combined based on the intelligent service type to obtain the mixed and combined media stream.

[0033] This application provides a method for processing multi-party calls, applied to a first network element, the method including:

[0034] Receive call requests from terminals participating in a multi-party call to dial the access number;

[0035] Generate first information, which indicates the first network element's response event to each terminal dialing the access number, wherein each terminal includes an initiating terminal and a participating terminal;

[0036] Send the first information to the application server;

[0037] The first network element receives a second message sent by the application server, the second message instructing the first network element to anchor the calls of each terminal to the second network element;

[0038] Create an endpoint to anchor the calls of each terminal to the second network element;

[0039] Receive third information sent by the second network element, the third information indicating the call control result of each terminal dialing the access number;

[0040] Send the third information to the application server;

[0041] Receive mixed and screen-combined media streams sent by intelligent media processing devices;

[0042] The mixed audio and screen media stream is sent to each of the terminals.

[0043] In the above scheme, the mixed audio and screen media stream is obtained by the intelligent media processing device mixing and merging the various call media streams of each terminal in combination with the intelligent service type.

[0044] This application provides a method for processing multi-party calls, applied to a second network element, the method comprising:

[0045] Send a third message to the first network element, the third message indicating the call control result of each terminal participating in the multi-party call dialing the access number;

[0046] Send each call media stream from each terminal to the intelligent media processing device.

[0047] The method in the above scheme further includes:

[0048] Send a ninth message to the intelligent media processing device. The ninth message is used to notify one or more of the following events: adjust the forwarding and mixing / screen merging relationships of the remaining media streams of the remaining terminals other than the initiating terminal, update the screen display information of the remaining terminals, and update the list information of the remaining terminals.

[0049] This application provides a method for processing multi-party calls, applied to a terminal, the method including:

[0050] Send a call request to the first network element to dial the access number for a multi-party call;

[0051] Receive the mixed audio and screen media stream sent by the first network element.

[0052] The method in the above scheme further includes:

[0053] When the terminal is the initiating terminal participating in the multi-party call, it sends a fifth message to the application server, the fifth message instructing the initiating terminal to hang up and exit the multi-party call.

[0054] The method in the above scheme further includes:

[0055] When the terminal is a participating terminal in the multi-party call, it receives updated screen display information and / or updated list information sent by the intelligent media processing device.

[0056] In the above scheme, the mixed audio and screen media stream is obtained by intelligent media processing equipment combining the intelligent service type to mix and screen the various call media streams of each terminal participating in the multi-party call.

[0057] The method in the above scheme further includes:

[0058] When the terminal is the initiating terminal participating in the multi-party call, it sends the tenth information to the service acceptance server. The tenth information is used to activate the call service and apply for the access number corresponding to the multi-party call. The tenth information includes the terminal's primary number.

[0059] The system receives the eleventh message sent by the service acceptance server. The eleventh message indicates that the primary number and the generated access number have been successfully bound. The eleventh message includes the access number.

[0060] An application server includes: a first communication interface and a first processor; wherein,

[0061] The first communication interface is used to receive first information sent by the first network element. The first information indicates the first network element's response event to dialing access numbers by each terminal participating in the multi-party call. Each terminal includes the initiating terminal and the participating terminal.

[0062] Send a second message to the first network element, the second message instructing the first network element to anchor the calls of each terminal to the second network element;

[0063] Receive third information sent by the first network element, wherein the third information indicates the call control result sent by the second network element to the first network element;

[0064] A fourth message is sent to the intelligent media processing device, the fourth message being used to notify the media anchoring result and the mixing process.

[0065] A smart media processing device includes: a second communication interface and a second processor; wherein,

[0066] The second communication interface is used to receive fourth information sent by the application server, the fourth information being used to notify the media anchoring result and the mixing process;

[0067] Receive the call media streams from each terminal forwarded by the second network element;

[0068] The second processor is used to mix and combine the various call media streams to obtain a mixed and combined media stream;

[0069] The second communication interface is used to send the mixed audio and screen media stream to the first network element.

[0070] A first network element includes: a third communication interface and a third processor; wherein,

[0071] The third communication interface is used to receive call requests from each terminal participating in a multi-party call to dial an access number;

[0072] The third processor is used to generate first information, which indicates the first network element's response event to each terminal dialing the access number, wherein each terminal includes an initiating terminal and a participating terminal;

[0073] The third communication interface is used to send the first information to the application server;

[0074] The first network element receives a second message sent by the application server, the second message instructing the first network element to anchor the calls of each terminal to the second network element;

[0075] The third processor is used to create endpoints and anchor the calls of each terminal to the second network element;

[0076] The third communication interface is used to receive third information sent by the second network element, the third information indicating the call control result of each terminal dialing the access number;

[0077] Send the third information to the application server;

[0078] Receive mixed and screen-combined media streams sent by intelligent media processing devices;

[0079] The mixed audio and screen media stream is sent to each of the terminals.

[0080] A second network element includes: a fourth communication interface and a fourth processor; wherein,

[0081] The fourth communication interface is used to send fourth information to the first network element, the fourth information indicating the call control result of each terminal participating in the multi-party call dialing the access number;

[0082] Send each call media stream from each terminal to the intelligent media processing device.

[0083] A terminal includes: a fifth communication interface and a fifth processor; wherein,

[0084] The fifth communication interface is used to send a call request to the first network element to dial an access number for a multi-party call;

[0085] Receive the mixed audio and screen media stream sent by the first network element.

[0086] A communication device includes: a processor and a memory for storing a computer program capable of running on the processor; wherein,

[0087] The processor is configured to execute the steps of any of the above methods when running the computer program.

[0088] This application also provides a computer product, including a computer program, which, when executed by a processor, implements the steps of any of the above methods.

[0089] This application provides a method, related equipment, storage medium, and computer product for processing multi-party calls. The method is applied to an application server and includes: receiving first information sent by a first network element, the first information indicating a response event from the first network element to dialing an access number by each terminal participating in the multi-party call, each terminal including an initiating terminal and participating terminals; sending second information to the first network element, the second information indicating that the first network element anchors the calls of each terminal to a second network element; receiving third information sent by the first network element, the third information indicating a call control result sent by the second network element to the first network element; and sending fourth information to an intelligent media processing device, the fourth information being used to notify the media anchoring result and mixing processing. Each participant in the multi-party call of this application can join the multi-party call by dialing the access number for the multi-party video call, thereby enabling each participant to actively join the multi-party video call based on native calls, and supporting mixing and merging of multiple call media streams. Attached Figure Description

[0090] Figure 1 This is a flowchart illustrating a multi-party call processing method according to an embodiment of this application;

[0091] Figure 2 This is a schematic diagram illustrating the process of implementing and interacting with a multi-party video call according to an embodiment of this application;

[0092] Figure 3 This is a flowchart illustrating an intelligent application extension service according to an embodiment of this application;

[0093] Figure 4 This is a flowchart illustrating another method for processing multi-party calls according to an embodiment of this application;

[0094] Figure 5 This is a flowchart illustrating the third method for processing multi-party calls according to an embodiment of this application;

[0095] Figure 6 This is a flowchart illustrating the fourth method for processing multi-party calls according to an embodiment of this application;

[0096] Figure 7 This is a flowchart illustrating the fifth method for processing multi-party calls according to an embodiment of this application;

[0097] Figure 8 This is a schematic diagram illustrating the process of generating access numbers for multi-party video calls in an embodiment of this application.

[0098] Figure 9 This is a schematic diagram of a multi-party call system according to an embodiment of this application;

[0099] Figure 10 This is a schematic diagram of the multi-party call processing device set on the application server according to an embodiment of this application;

[0100] Figure 11 This is a schematic diagram of the multi-party call processing device installed on a smart media processing device according to an embodiment of this application.

[0101] Figure 12 A schematic diagram of the structure of a multi-party call processing device configured on a first network element according to an embodiment of this application;

[0102] Figure 13 A schematic diagram of the multi-party call processing device configured on the second network element according to an embodiment of this application;

[0103] Figure 14 This is a schematic diagram of the multi-party call processing device installed on a terminal according to an embodiment of this application;

[0104] Figure 15 This is a schematic diagram of the application server structure according to an embodiment of this application;

[0105] Figure 16 This is a schematic diagram of the structure of the intelligent media processing device according to an embodiment of this application;

[0106] Figure 17 This is a schematic diagram of the structure of the first network element in an embodiment of this application;

[0107] Figure 18 This is a schematic diagram of the structure of the second network element in an embodiment of this application;

[0108] Figure 19 This is a schematic diagram of the terminal structure according to an embodiment of this application. Detailed Implementation

[0109] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0110] Currently, all existing multi-party calls offered by operators are voice calls only, and do not support video calls. Furthermore, joining a multi-party call requires suspending the existing call while two parties are in a regular conversation, then adding a call to the third party via the native dial pad. Once connected, the two one-on-one voice calls can be merged into a single three-way voice call using the native dial pad. There is no way for users to actively join multi-party calls. The overall operation and interaction process is complex, resulting in a low user experience and limited adoption. Moreover, 5G technology currently does not support video calls.

[0111] The multi-party calling products in the internet industry are mostly video conferencing or audio group chats, which require users to download and install corresponding applications (APPs). They often need to add each other as friends, create a group with multiple people, or pre-book a video conference to initiate a multi-party call. Moreover, they do not support implementation on the phone's native calling function and have many limitations in terms of usage time and location.

[0112] It is evident that a complete multi-party video call process cannot currently be provided based on the terminal's native call functionality.

[0113] This application provides a method for processing multi-party calls, applied to an application server (AS), such as... Figure 1 As shown, the method includes:

[0114] Step 101: Receive the first information sent by the first network element. The first information indicates the first network element's response to the dialing of access numbers by each terminal participating in the multi-party call. Each terminal includes the initiating terminal and the participating terminals.

[0115] In practical applications, the first network element can be a Voice over New Radio (VoNR+) capability network element. The VoNR+ capability network element is used to provide call event notifications, call control, media stream copying, and forwarding notifications to the multi-party video call server.

[0116] In a multi-party call, each terminal can be referred to as a User End (UE) or simply a user. A multi-party call refers to an application scenario where three or more user ends or users participate in an audio or video call. The multi-party call processing method provided in this application has applications including, but not limited to, video conferencing systems and group chats. A video conferencing system generally consists of video conferencing terminals (hardware or software terminals), a video conferencing server, and a transmission network. Participants need to use a dedicated video conferencing system to hear audio from other meeting rooms, see the images, actions, and expressions of other participants, and send electronic presentations; it can be used in formal business meetings. Group chats, or video group chats, are generally a function of instant messaging software, essentially migrating offline social needs online. Participants can join a video group by adding friends and being invited to initiate multi-party audio and video calls with group members; it is mainly used in familiar social scenarios.

[0117] For example, taking a multi-party video call where the initiating terminal is UEA and other participating terminals UEB and UEC participate in the call, both UEB and UEC can join the multi-party video call by actively dialing UEA's personal dedicated multi-party video access number.

[0118] In a feasible multi-party call processing scenario, referring to Figure 2, the implementation of multi-party call processing method provided in this application will be described in detail:

[0119] Steps 201-202 correspond to the steps by which the AS receives the first information sent by the VoNR+ capability network element.

[0120] Combination Figure 2 As shown, steps 201-202 involve the interaction between the VoNR+ capability network element and the AS. The VoNR+ capability network element reports the response event of the user dialing the multi-party video call access number to the AS.

[0121] Step 201: The initiating user UEA can dial the activated multi-party video call access number; another participating user UEB dials the multi-party video call access number shared by user UEA;

[0122] Step 202: VoNR+ capability network elements report response events to AS.

[0123] The AS here can be a multi-party video call AS. Specifically, the multi-party video call AS is used to support the business logic processing of multi-party video calls, and supports following up on user business settings, system business settings, and other business data. Combined with real-time user interaction information, it completes specified business logic updates and processing such as: initiating multi-party video calls, mixing and merging multi-channel media streams, audio and video switching, muting, and selecting which screen to view.

[0124] In this process, the VoNR+ capability network element reports response events to the multi-party video call AS, and the capability network element and the multi-party video call AS use the Hypertext Transfer Protocol (HTTP) 2.0 protocol for event notification and application data interaction.

[0125] Step 102: Send the second information to the first network element, the second information instructing the first network element to anchor the calls of each terminal to the second network element.

[0126] In practical applications, the second network element can be a VoNR+ media network element. The VoNR+ media network element provides basic media processing capabilities, including media stream copying and forwarding, and some video compositing and rendering capabilities. During a multi-party video call, the AS sends a second message to the VoNR+ capability network element, instructing it to anchor the call to the VoNR+ media network element.

[0127] Combination Figure 2 As shown, steps 203-204 involve the interaction between AS, VoNR+ capabilities, and VoNR+ media network elements, and are the process of anchoring a call to a VoNR+ media network element.

[0128] Step 203: In a multi-party video call, the AS instructs the VoNR+ capability network element to anchor the call to the VoNR+ media network element.

[0129] Step 204: The VoNR+ capability network element creates an endpoint on the VoNR+ media network element to anchor the calls of UEA and UEB to the VoNR+ media network element.

[0130] Step 103: Receive the third information sent by the first network element, the third information indicating the call control result sent by the second network element to the first network element.

[0131] Combination Figure 2 As shown, steps 205-206 involve the interaction between the AS, VoNR+ capability network elements, and VoNR+ media network elements, and the process of reporting call control results to the AS.

[0132] Step 205: After successful call anchoring, the VoNR+ media network element reports the call control result to the VoNR+ capability network element. Step 206: The VoNR+ capability network element reports the call control result to the multi-party video call AS.

[0133] Step 104: Send a fourth message to the intelligent media processing device. The fourth message is used to notify the media anchoring results and mixing process.

[0134] Among them, the intelligent media processing equipment is used to support media stream adaptation, media encoding and transcoding, multi-point control unit (MCU) multi-point control, media stream channels and other capabilities required for business application scenarios, and introduces artificial intelligence (AI) and extended reality (XR) processing capabilities related to business application scenarios, such as speech recognition, machine translation, digital human and so on.

[0135] Combination Figure 2 As shown, steps 207-210 involve the interaction between the intelligent media processing module and the AS, VoNR+ media network element, and VoNR+ capability network element to achieve audio-visual merging and screen integration, and to forward the audio-visual merging and screen integration media stream to the VoNR+ capability network element.

[0136] Step 207: The multi-party video call AS sends the media anchoring result and mixing processing notification to the intelligent media processing module.

[0137] For example, the mixing notification includes, but is not limited to: a first notification message indicating that the audio codec uses a new variable rate multimode wideband voice codec (AMR-WB), and a second notification message indicating that the video codec uses H.264 codec and a resolution of 480P or higher.

[0138] Step 208: The VoNR+ media network element sends the copied UEA and UEB call media streams to the intelligent media processing module. The signaling interface uses the HTTP 2.0 protocol to transmit encoding and decoding information, and the media interface uses the Real-time Transport Protocol (RTP) to send the anchored and copied audio and video streams.

[0139] Step 209: The media processing capabilities of the intelligent media processing module and the MCU multi-point control unit are used to complete the mixing and screen merging of multiple media streams.

[0140] Step 210: The intelligent media processing module forwards the mixed and screen-merged media stream to the VoNR+ capability network element. The media interface uses the RTP protocol.

[0141] Combination Figure 2 As shown, step 211 involves the interaction between VoNR+ capability network elements and various user terminals, and the forwarding of the media stream after mixing and screen merging.

[0142] Step 211: The VoNR+ capability network element forwards the mixed and combined media stream to each user terminal.

[0143] Step 212: If a new participant user (UEC) requests to join the multi-party video call, repeat the process from Step 201 to Step 211 above.

[0144] In some embodiments, the above-described multi-party call processing method further includes the following steps:

[0145] Receive the fifth message sent by the initiating terminal, which instructs the initiating terminal to hang up and exit the multi-party call;

[0146] The sixth message is sent to the second network element, which instructs the second network element to release endpoint resources.

[0147] Combination Figure 2 As shown, steps 213-214 involve the AS performing corresponding processing when the initiating terminal hangs up and exits the multi-party call.

[0148] Step 213: When the initiating user UEA hangs up and exits the multi-party video call, the call event is uploaded to the multi-party video call AS.

[0149] Step 214: The multi-party video call AS sends a notification to the VoNR+ media network element to release the UEA endpoint resources.

[0150] Furthermore, combined Figure 2 As shown, steps 215-216 involve updating the list of other participants and the information displayed on the screen when the initiating terminal hangs up and exits the multi-party call.

[0151] Step 215: The VoNR+ media network element notifies the intelligent media processing module to adjust the forwarding and mixing relationships of the remaining streams, and updates the list of participants and the screen display information.

[0152] Step 216: The intelligent media processing module sends the updated list of participants and screen display information to the remaining terminals.

[0153] In some embodiments, the above-described multi-party call processing method further includes the following steps:

[0154] When all terminals are in a multi-party call, they receive the seventh message sent by the initiating terminal. The seventh message indicates the type of intelligent service requested to be activated.

[0155] Send an eighth message to the intelligent media processing device, which instructs the multi-party call to be processed in accordance with the intelligent service type.

[0156] In practical applications, intelligent business types include one or more of the following:

[0157] Intelligent application business;

[0158] Artificial intelligence business;

[0159] Expand existing business operations.

[0160] Here, the AS can be an intelligent application AS, which supports the business logic processing of various intelligent applications. It also supports the selection of specified AI capability types and the updating and processing of business logic based on user business settings, system business settings, and other business data, combined with the type of intelligent application selected by the user and interaction information. It should be noted that the intelligent application business AS in this application can be merged with the multi-party video call AS to perform multi-party call processing methods.

[0161] This application can also extend multi-party calls by adding intelligent application extension services (such as intelligent translation). The process includes when the initiating user UEA and other participating users UEB and UEC are all in the same multi-party video call, the initiating user UEA enables the intelligent translation function. The intelligent application service AS (which can be merged with the multi-party video call AS) controls the business logic. The intelligent media processing module calls the AI ​​capabilities ASR speech recognition and machine translation algorithms to complete the speech-to-text conversion and translation. The intelligent media processing module then completes the subtitle synthesis to realize the real-time display of the translation results in the multi-party video call.

[0162] In a feasible intelligent business application scenario for multi-party call processing, combined with Figure 3As shown, the multi-party call processing method provided in this application is used to implement multi-party calls in a detailed explanation: Here, the intelligent service takes the initiating user UEA enabling the intelligent translation function as an example.

[0163] Step 301: Enable access for three or more participants. Currently, the initiating user UEA and other participating users UEB and UEC are all in a multi-party video call.

[0164] Step 302: User UEA requests to enable smart applications (e.g., smart translation).

[0165] Step 303: The Intelligent Application Service (AS) sends a service control message (e.g., call control) based on the user's UEA request service type, instructing the Intelligent Media Processing Module to anchor and process the multi-party mixed and screen-combined media stream.

[0166] Step 304: The intelligent media processing module calls the AI ​​capabilities of speech recognition (ASR) and machine translation algorithms to perform speech-to-text conversion and translation on the mixed audio and screen media stream.

[0167] Step 305: Intelligent Media Processing Notification: The Intelligent Business Application (AS) has completed speech recognition and translation processing.

[0168] Step 306: The intelligent business application sends a business control message, instructing the intelligent media processing module to synthesize the multi-party mixed audio and screen combined media stream and the subtitle stream.

[0169] Step 307: The intelligent media processing module combines the subtitle stream content to complete the subtitle synthesis of the downlink video stream.

[0170] Step 308: The intelligent media processing module forwards the mixed and combined media stream to the VoNR+ capability network element.

[0171] Step 309: The VoNR+ capability network element forwards the mixed and combined media stream to each user terminal.

[0172] Step 310: If you need to add other smart applications (such as smart beautification, background replacement, and virtual avatars), repeat the process of steps 301-309 above.

[0173] This application provides a method for processing multi-party calls, applied to intelligent media processing devices, such as... Figure 4 As shown, the method includes:

[0174] Step 401: Receive the fourth message sent by the application server. The fourth message is used to notify the media anchoring result and the mixing process.

[0175] Here, the intelligent media processing module interacts with the AS, receiving media anchoring results and mixing processing notifications sent by the AS.

[0176] Step 402: Receive the call media streams from each terminal forwarded by the second network element.

[0177] Step 403: Mix and combine the media streams of each call to obtain a mixed and combined media stream.

[0178] Here, the media processing capabilities of the intelligent media processing device and the MCU multi-point control unit are used to complete the mixing and screen merging of multiple media streams.

[0179] Step 404: Send the mixed audio and screen media stream to the first network element.

[0180] Here, the intelligent media processing device forwards the mixed and combined media stream to the VoNR+ capability network element, and the media interface adopts the RTP protocol.

[0181] In some embodiments, the above-described multi-party call processing method further includes the following steps:

[0182] Receive the ninth message sent by the second network element. The ninth message is used to notify one or more of the following events: adjust the forwarding and mixing relationship of the remaining media streams of the remaining terminals other than the initiating terminal, update the screen display information of the remaining terminals, and update the list information of the remaining terminals.

[0183] Execute one or more events.

[0184] In some embodiments, the above-described multi-party call processing method further includes the following steps:

[0185] Send updated screen display information and / or updated list information to the remaining terminals.

[0186] In some embodiments, the above-described multi-party call processing method further includes the following steps:

[0187] When each terminal is in a multi-party call, it receives the eighth message sent by the application server. The eighth message indicates how to process the multi-party call in accordance with the intelligent service type.

[0188] Accordingly, the various call media streams are mixed and combined to obtain a mixed and combined media stream, including:

[0189] By combining the intelligent service types, the media streams of each call are mixed and screened to obtain a mixed and screened media stream.

[0190] This application provides a method for processing multi-party calls, applied to a first network element, such as... Figure 5 As shown, the method includes:

[0191] Step 501: Receive call requests from each terminal participating in the multi-party call to dial the access number.

[0192] UEA, UEB, and UEC members can all join a multi-party video call by actively dialing UEA's personal dedicated multi-party video access number.

[0193] Step 502: Generate first information, which indicates the response event of the first network element to the dialing access number of each terminal. Each terminal includes the initiating terminal and the participating terminal.

[0194] Step 503: Send the first message to the application server.

[0195] VoNR+ capability network elements report response events to AS.

[0196] Step 504: Receive the second information sent by the application server. The second information instructs the first network element to anchor the calls of each terminal to the second network element.

[0197] Step 505: Create endpoints to anchor the calls of each terminal to the second network element.

[0198] The VoNR+ capability element creates an endpoint on the VoNR+ media element, anchoring calls from UEA and UEB to the VoNR+ media element.

[0199] Step 506: Receive the third information sent by the second network element, which indicates the call control result of each terminal dialing the access number.

[0200] After a successful call anchoring, the VoNR+ media network element reports the call control result to the VoNR+ capability network element.

[0201] Step 507: Send third-party information to the application server.

[0202] The third information indicates the call control result sent by the VoNR+ media network element to the VoNR+ capability network element.

[0203] Step 508: Receive the mixed media stream sent by the intelligent media processing device.

[0204] Step 509: Send the mixed audio and screen merged media stream to each terminal.

[0205] In practical applications, the mixed audio and screen media stream is obtained by intelligent media processing equipment combining the media streams of each call from each terminal with the intelligent service type.

[0206] This application provides a method for processing multi-party calls, applied to a second network element, such as... Figure 6 As shown, the method includes:

[0207] Step 601: Send third information to the first network element, the third information indicating the call control result of each terminal participating in the multi-party call dialing the access number.

[0208] Step 602: Send the call media streams of each terminal to the intelligent media processing device.

[0209] In some embodiments, the above-described multi-party call processing method further includes the following steps:

[0210] Send a ninth message to the intelligent media processing device. The ninth message is used to notify one or more of the following events: adjust the forwarding and mixing relationships of the remaining media streams of the remaining terminals other than the initiating terminal, update the screen display information of the remaining terminals, and update the list information of the remaining terminals.

[0211] This application provides a method for processing multi-party calls, applied to a terminal, such as... Figure 7 As shown, the method includes:

[0212] Step 701: Send a call request to the first network element to dial the access number for a multi-party call.

[0213] Step 702: Receive the mixed audio and screen media stream sent by the first network element.

[0214] In practical applications, the mixed audio and screen media stream is obtained by intelligent media processing equipment combining the various call media streams of each terminal participating in a multi-party call with intelligent service type.

[0215] In some embodiments, the above-described multi-party call processing method further includes the following steps:

[0216] When the terminal is the initiating terminal in a multi-party call, it sends a fifth message to the application server, which instructs the initiating terminal to hang up and exit the multi-party call.

[0217] In some embodiments, the above-described multi-party call processing method further includes the following steps:

[0218] When the terminal is a participating terminal in a multi-party call, it receives updated screen display information and / or updated list information sent by the intelligent media processing device.

[0219] In some embodiments, the above-described multi-party call processing method further includes the following steps:

[0220] When the terminal is the initiating terminal in a multi-party call, it sends the tenth information to the service acceptance server. The tenth information is used to activate the call service and apply for the access number corresponding to the multi-party call. The tenth information includes the terminal's primary number.

[0221] Receive the eleventh message sent by the service acceptance server. The eleventh message indicates that the main number and the generated access number have been successfully bound. The eleventh message includes the access number.

[0222] The business acceptance server is responsible for connecting the interface between users and the Business & Operation Support System (BOSS) to transmit business acceptance information, including user data management and system configuration.

[0223] Combination Figure 8 As shown, the generation of multi-party video call access numbers can be achieved through the following process: users complete the application and subscription for multi-party video call services through the client management module and service acceptance module, and generate their own exclusive multi-party video call access numbers.

[0224] Step 801: Users can activate multi-party video call services and apply for access numbers through dedicated online service channels such as H5 pages, mini programs, and APP clients.

[0225] Step 802: After the user initiates an application and fills in the application information, the application information will be uploaded to the client management module for further processing.

[0226] Step 803: After receiving a user's business application, the client management module first performs a pre-acceptance check, which mainly includes whether a multi-party video call access number has been processed.

[0227] Then, an access number will be generated based on the business application information. The rules and methods for generating the access number include, but are not limited to, the following: adding a fixed prefix to the user's main number as a personal exclusive multi-party video call access number, randomly generating the access number, or allowing the user to set their own access number.

[0228] After successfully generating the access number, submit an application to the service acceptance module to bind the access number to the primary number;

[0229] Step 804: The service acceptance module binds the primary number and the access number according to the generated access number, and sends a binding request to the province where the number belongs, carrying the individual's real-name information.

[0230] Step 805: If the provincial BOSS acceptance check of the number's origin passes, a subscription relationship is generated and a success result is sent to the business acceptance module; if the provincial BOSS acceptance check of the number's origin fails, a subscription relationship is not generated and the specific reason for failure is returned.

[0231] Step 806: The business acceptance module confirms that it has received a successful response from the provincial BOSS.

[0232] Step 807: The business acceptance module generates a subscription relationship based on the results returned by BOSS. At this point, the main number and the access number are successfully bound.

[0233] Step 808: The access number platform will send the subscription confirmation information and access number information back to the user. At this time, the multi-party video call access number will officially take effect and the multi-party video call process can be initiated.

[0234] Based on this, the multi-party calling method provided in this application covers the following three processes from user application for service activation to usage: first, the multi-party video call access number generation process; second, the multi-party video call implementation and interaction process; and third, the intelligent application extension service process. In the access number generation process, this application supports rich and flexible generation of multi-party video call access numbers. In the multi-party video call implementation and interaction process, this application supports multi-party video calls implemented based on the terminal's native call functionality, and supports mixed audio and screen display with simultaneous multi-screen display and simultaneous voice communication among multiple users. In the intelligent application extension service process, this application supports the introduction of AI and XR capabilities, enabling the expansion of various intelligent applications during calls, increasing service diversity, and meeting diverse business needs. Among these, multi-party video calls implemented based on the terminal's native call functionality meet users' business needs for communication anytime, anywhere, such as remote work, instant meetings, group chats with family and friends, business negotiations, and remote guidance. Furthermore, by expanding various intelligent applications on the terminal's multi-party calling functionality, the value of calls can be gradually amplified, attracting users back to the call service and driving its growth.

[0235] This application also provides a system for multi-party communication, such as... Figure 9 As shown, the system comprises three parts: multi-party video call access number generation (901), multi-party video call implementation and interaction (902), and intelligent application extension service (903). The main modules and network elements involved are as follows: client management module, service acceptance module, VoNR+ capability network element, VoNR+ media network element, intelligent media processing module, multi-party video call AS, and intelligent application AS.

[0236] The client management module is responsible for handling applications for opening multi-party video call access numbers, providing business acceptance pages such as H5 pages, mini programs, and APPs, completing user authentication and business application information reporting, and completing related functions for managing the client interface.

[0237] The multi-party calling system provided in this application offers an enhanced multi-party video calling method based on 5G new calling. This method first involves the user generating an access number through a multi-party video call service acceptance module. Each participating party dials the multi-party video call access number. The intelligent media processing module provides media processing and AI capabilities to achieve audio mixing and screen merging of the multi-party call media streams. It can also combine AI and XR capabilities to support extended functions such as intelligent translation, intelligent beautification, background replacement, content summarization, digital humans, and spatial navigation. It can replace traditional video conferencing and group chats in many scenarios, allowing participants to actively join multi-party video calls without being restricted by their identities, based on native calling, thus meeting users' needs for high-definition intelligent multi-party communication anytime, anywhere.

[0238] To implement the application server-side method of this application embodiment, this application embodiment also provides a multi-party call processing device, which is set on the application server, such as... Figure 10 As shown, the device includes:

[0239] The first receiving unit 1001 is used to receive the first information sent by the first network element. The first information indicates the first network element's response event to the dialing of access numbers by each terminal participating in the multi-party call. Each terminal includes the initiating terminal and the participating terminal.

[0240] The first sending unit 1002 is used to send second information to the first network element, the second information instructing the first network element to anchor the calls of each terminal to the second network element;

[0241] The first receiving unit 1001 is used to receive third information sent by the first network element, the third information indicating the call control result sent by the second network element to the first network element;

[0242] The first sending unit 1002 is used to send fourth information to the intelligent media processing device. The fourth information is used to notify the media anchoring result and the mixing process.

[0243] In some embodiments, the first receiving unit 1001 is used to receive the fifth information sent by the initiating terminal, the fifth information instructing the initiating terminal to hang up and exit the multi-party call;

[0244] The first sending unit 1002 is used to send a sixth message to the second network element, the sixth message instructing the second network element to release endpoint resources.

[0245] In some embodiments, the first receiving unit 1001 is used to receive a seventh message sent by the initiating terminal when each terminal is in a multi-party call, the seventh message indicating the type of smart service to be activated.

[0246] The first sending unit 1002 is used to send an eighth message to the intelligent media processing device. The eighth message indicates that the multi-party call should be processed in combination with the intelligent service type.

[0247] In some embodiments, the intelligent service type includes one or more of the following:

[0248] Intelligent application business;

[0249] Artificial intelligence business;

[0250] Expand existing business operations.

[0251] In practical applications, the first receiving unit 1001 and the first sending unit 1002 can be implemented by the communication interface in the application server.

[0252] To implement the method on the intelligent media processing device side of this application embodiment, this application embodiment also provides a multi-party call processing apparatus, which is installed on the intelligent media processing device, such as... Figure 11 As shown, the device includes:

[0253] The second receiving unit 1101 is used to receive the fourth information sent by the application server. The fourth information is used to notify the media anchoring result and the mixing process.

[0254] The second receiving unit 1101 is used to receive the call media streams of each terminal forwarded by the second network element;

[0255] The second processing unit 1102 is used to mix and combine the media streams of each call to obtain a mixed and combined media stream.

[0256] The second transmitting unit 1103 is used to transmit the mixed audio and screen media stream to the first network element.

[0257] In some embodiments, the second processing unit 1102 is used to receive the ninth information sent by the second network element. The ninth information is used to notify one or more of the following events: adjusting the forwarding and mixing relationship of the remaining media streams of the remaining terminals other than the initiating terminal, updating the screen display information of the remaining terminals, and updating the list information of the remaining terminals.

[0258] The second processing unit 1102 is used to execute one or more events.

[0259] In some embodiments, the second sending unit 1103 is used to send updated screen display information and / or updated list information to the remaining terminals.

[0260] In some embodiments, the second receiving unit 1101 is used to receive an eighth message sent by the application server when each terminal is in a multi-party call, the eighth message indicating that the multi-party call should be processed in combination with the intelligent service type.

[0261] The second processing unit 1102 is used to combine the intelligent service type to mix and merge the media streams of each call to obtain a mixed and merged media stream.

[0262] In practical applications, the second processing unit 1102 can be implemented by a processor in the intelligent media processing device; the second receiving unit 1101 and the second sending unit 1103 can be implemented by a communication interface in the intelligent media processing device.

[0263] To implement the method on the first network element side of this application embodiment, this application embodiment also provides a multi-party call processing device, which is disposed on the first network element, such as... Figure 12 As shown, the device includes:

[0264] The third receiving unit 1201 is used to receive call requests from each terminal participating in a multi-party call to dial an access number;

[0265] The third processing unit 1202 is used to generate first information, which indicates the response events of the first network element to the dialing of access numbers by each terminal, including the initiating terminal and the participating terminal.

[0266] The third sending unit 1203 is used to send the first information to the application server;

[0267] The third receiving unit 1201 is used to receive the second information sent by the application server, the second information instructing the first network element to anchor the calls of each terminal to the second network element;

[0268] The third processing unit 1202 is used to create endpoints and anchor the calls of each terminal to the second network element;

[0269] The third receiving unit 1201 is used to receive the third information sent by the second network element, which indicates the call control result of each terminal dialing the access number;

[0270] The third sending unit 1203 is used to send third information to the application server;

[0271] The third receiving unit 1201 is used to receive the mixed audio and screen combined media stream sent by the intelligent media processing device;

[0272] The third sending unit 1203 is used to send the mixed audio and screen media stream to each terminal.

[0273] In some embodiments, the mixed audio and screen-combined media stream is obtained by the intelligent media processing device mixing and combining the various call media streams of each terminal in combination with the intelligent service type.

[0274] In practical applications, the third processing unit 1202 can be implemented by the processor in the first network element; the third receiving unit 1201 and the third transmitting unit 1203 can be implemented by the communication interface in the first network element.

[0275] To implement the method on the second network element side of this application embodiment, this application embodiment also provides a multi-party call processing device, which is disposed on the second network element, such as... Figure 13 As shown, the device includes:

[0276] The fourth sending unit 1301 is used to send third information to the first network element. The third information indicates the call control result of each terminal participating in the multi-party call dialing the access number.

[0277] The fourth sending unit 1301 is used to send the various call media streams of each terminal to the intelligent media processing device.

[0278] In some embodiments, the fourth sending unit 1301 is used to send a ninth message to the intelligent media processing device. The ninth message is used to notify one or more of the following events: adjusting the forwarding and mixing relationships of the remaining media streams of the remaining terminals other than the initiating terminal, updating the screen display information of the remaining terminals, and updating the list information of the remaining terminals.

[0279] In practical applications, the fourth transmitting unit 1301 can be implemented through the communication interface in the second network element.

[0280] In some embodiments, the device includes a fourth receiving unit 1302 for receiving a notification from the multi-party video call AS to release UEA endpoint resources.

[0281] To implement the terminal-side method of this application embodiment, this application embodiment also provides a multi-party call processing device, which is installed on the terminal, such as... Figure 14 As shown, the device includes:

[0282] The fifth sending unit 1401 is used to send a call request to the first network element to dial the access number for a multi-party call;

[0283] The fifth receiving unit 1402 is used to receive the mixed audio and screen media stream sent by the first network element.

[0284] In some embodiments, the fifth sending unit 1401 is used to send a fifth message to the application server when the terminal is the initiating terminal participating in a multi-party call. The fifth message instructs the initiating terminal to hang up and exit the multi-party call.

[0285] In some embodiments, the fifth receiving unit 1402 is configured to receive updated screen display information and / or updated list information sent by the intelligent media processing device when the terminal is a participating terminal in a multi-party call.

[0286] In some embodiments, the mixed audio and screen-combined media stream is obtained by intelligent media processing equipment mixing and combining the various call media streams of each terminal participating in a multi-party call with intelligent service type.

[0287] In some embodiments, the fifth sending unit 1401 is used to send tenth information to the service acceptance server when the terminal is the initiating terminal participating in a multi-party call. The tenth information is used to activate the call service and apply for the access number corresponding to the multi-party call. The tenth information includes the terminal's main number.

[0288] The fifth receiving unit 1402 is used to receive the eleventh message sent by the service acceptance server. The eleventh message indicates that the main number and the generated access number have been successfully bound. The eleventh message includes the access number.

[0289] In practical applications, the fifth transmitting unit 1401 and the fifth receiving unit 1402 can be implemented by the communication interface in the terminal.

[0290] To implement the application server-side method of this application embodiment, this application embodiment also provides an application server, such as... Figure 15 As shown, the application server 1500 includes: a first communication interface 1501 and a first processor 1502; wherein, the first communication interface 1501 is capable of exchanging information with a first network element, a second network element, an intelligent media processing device and / or other terminals;

[0291] The first processor 1502 is connected to the first communication interface 1501 to enable information interaction with the first network element, the second network element, the intelligent media processing device and / or other terminals, and to execute the methods provided by one or more technical solutions on the application server side when running computer programs.

[0292] The first memory 1503 stores computer programs that can run on the first processor 1502.

[0293] The first communication interface 1501 is used to receive first information sent by the first network element, which instructs the first network element to respond to the dialing access number of each terminal participating in the multi-party call, including the initiating terminal and the participating terminal; send second information to the first network element, which instructs the first network element to anchor the calls of each terminal to the second network element; receive third information sent by the first network element, which instructs the second network element to send the call control result to the first network element; and send fourth information to the intelligent media processing device, which is used to notify the media anchoring result and the mixing process.

[0294] In some embodiments, the first communication interface 1501 is used to receive a fifth message sent by the initiating terminal, the fifth message instructing the initiating terminal to hang up and exit the multi-party call; and to send a sixth message to the second network element, the sixth message instructing the second network element to release endpoint resources.

[0295] In some embodiments, the first communication interface 1501 is used to receive a seventh message sent by the initiating terminal when each terminal is in a multi-party call, the seventh message indicating the intelligent service type to be activated; and to send an eighth message to the intelligent media processing device, the eighth message indicating that the multi-party call should be processed in combination with the intelligent service type.

[0296] In some embodiments, the intelligent service type includes one or more of the following: intelligent application services; artificial intelligence services; and augmented reality services.

[0297] It should be noted that the specific processing procedures of the first communication interface 1501 and the first processor 1502 can be understood by referring to the above method, and will not be repeated here.

[0298] Of course, in practical applications, the various components in application server 1500 are coupled together through the first bus system 1504. It can be understood that the first bus system 1504 is used to implement communication between these components. In addition to a data bus, the first bus system 1504 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 15 The general designated all buses as the first bus system 1504.

[0299] The first memory 1503 in this embodiment is used to store various types of data to support the operation of the application server 1500. Examples of such data include any computer program used to operate on the application server 1500.

[0300] The methods disclosed in the embodiments of this application can be applied to or implemented by the first processor 1502. The first processor 1502 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware or by instructions in the form of software in the first processor 1502. The first processor 1502 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The first processor 1502 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the first memory 1503. The first processor 1502 reads the information in the first memory 1503 and completes the steps of the aforementioned method in combination with its hardware.

[0301] In an exemplary embodiment, the application server 1500 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned methods.

[0302] To implement the method on the intelligent media processing device side of the embodiments of this application, the embodiments of this application also provide an intelligent media processing device, such as... Figure 16 As shown, the intelligent media processing device 1600 includes: a second communication interface 1601 and a second processor 1602; wherein, the second communication interface 1601 is capable of interacting with a first network element, an application server and / or other terminals;

[0303] The second processor 1602 is connected to the second communication interface 1601 to enable information interaction with other intelligent media processing devices and / or terminals, and to execute the methods provided by one or more technical solutions on the intelligent media processing device side when running computer programs.

[0304] The second memory 1603 stores computer programs that can run on the second processor 1602.

[0305] The second communication interface 1601 is used to receive the fourth information sent by the application server, which is used to notify the media anchoring result and the mixing process; and to receive the call media streams of each terminal forwarded by the second network element; the second processor 1602 is used to mix and combine the call media streams to obtain the mixed and combined media stream; and the second communication interface 1601 is used to send the mixed and combined media stream to the first network element.

[0306] In some embodiments, the second communication interface 1601 is used to receive the ninth information sent by the second network element. The ninth information is used to notify one or more of the following events: adjusting the forwarding and mixing relationship of the remaining media streams of the remaining terminals other than the initiating terminal, updating the screen display information of the remaining terminals, and updating the list information of the remaining terminals; the second processor 1602 is used to execute one or more events.

[0307] In some embodiments, the second communication interface 1601 is used to send updated screen display information and / or updated list information to the remaining terminals.

[0308] In some embodiments, the second communication interface 1601 is used to receive the eighth information sent by the application server when each terminal is in a multi-party call. The eighth information indicates that the multi-party call should be processed in combination with the intelligent service type. The second processor 1602 is used to mix and combine the media streams of each call in combination with the intelligent service type to obtain a mixed and combined media stream.

[0309] It should be noted that the specific processing procedures of the second communication interface 1601 and the second processor 1602 can be understood by referring to the above method, and will not be repeated here.

[0310] Of course, in practical applications, the various components in the intelligent media processing device 1600 are coupled together via a second bus system 1604. It can be understood that the second bus system 1604 is used to realize the connection and communication between these components. In addition to a data bus, the second bus system 1604 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 16 The general designated all buses as the second bus system 1604.

[0311] The second memory 1603 in this embodiment is used to store various types of data to support the operation of the smart media processing device 1600. Examples of such data include any computer program used to operate on the smart media processing device 1600.

[0312] The methods disclosed in the embodiments of this application can be applied to, or implemented by, the second processor 1602. The second processor 1602 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware or by instructions in the software form of the second processor 1602. The second processor 1602 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The second processor 1602 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, specifically a second memory 1603. The second processor 1602 reads information from the second memory 1603 and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0313] In an exemplary embodiment, the intelligent media processing device 1600 may be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to perform the aforementioned method.

[0314] To implement the method on the first network element side of the embodiments of this application, the embodiments of this application also provide a first network element, such as... Figure 17 As shown, the first network element 1700 includes: a third communication interface 1701 and a third processor 1702; wherein, the third communication interface 1701 is capable of exchanging information with the first network element, the second network element, the intelligent media processing device and / or other terminals;

[0315] The third processor 1702 is connected to the third communication interface 1701 to enable information interaction with the second network element, intelligent media processing device, application server and / or other terminals, and to execute the methods provided by one or more technical solutions on the first network element side when running computer programs.

[0316] The third memory 1703 stores computer programs that can run on the third processor 1702.

[0317] The third communication interface 1701 is used to receive call requests from each terminal participating in a multi-party call to dial an access number.

[0318] The third processor 1702 is used to generate first information, which indicates the response events of the first network element to the dialing of access numbers by each terminal, including the initiating terminal and the participating terminal.

[0319] The third communication interface 1701 is used to send first information to the application server and receive second information sent by the application server, wherein the second information instructs the first network element to anchor the calls of each terminal to the second network element.

[0320] The third processor 1702 is used to create endpoints and anchor calls from each terminal to the second network element;

[0321] The third communication interface 1701 is used to receive third information sent by the second network element, the third information indicating the call control result of each terminal dialing the access number; send third information to the application server; receive the mixed audio and screen merging media stream sent by the intelligent media processing device; and send the mixed audio and screen merging media stream to each terminal.

[0322] It should be noted that the specific processing procedures of the third communication interface 1701 and the third processor 1702 can be understood by referring to the above method, and will not be repeated here.

[0323] Of course, in practical applications, the various components in the first network element 1700 are coupled together through the third bus system 1704. It can be understood that the third bus system 1704 is used to realize the connection and communication between these components. In addition to the data bus, the third bus system 1704 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 17 The general labeled all buses as the third bus system 1704.

[0324] The third memory 1703 in this embodiment is used to store various types of data to support the operation of the first network element 1700. Examples of such data include any computer program used to operate on the first network element 1700.

[0325] The methods disclosed in the embodiments of this application can be applied to, or implemented by, the third processor 1702. The third processor 1702 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware or by instructions in the software form of the third processor 1702. The third processor 1702 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The third processor 1702 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, specifically a third memory 1703. The third processor 1702 reads information from the third memory 1703 and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0326] In an exemplary embodiment, the first network element 1700 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0327] To implement the method on the second network element side of the embodiments of this application, the embodiments of this application also provide a second network element, such as... Figure 18 As shown, the second network element 1800 includes: a fourth communication interface 1801 and a fourth processor 1802; wherein, the fourth communication interface 1801 is capable of exchanging information with the second network element, the intelligent media processing device, and the application server;

[0328] The fourth processor 1802 is connected to the fourth communication interface 1801 to enable information interaction with the second network element, the intelligent media processing device, and the application server. When running a computer program, it executes the methods provided by one or more technical solutions on the second network element side.

[0329] The fourth memory 1803 stores computer programs that can run on the fourth processor 1802.

[0330] The fourth communication interface 1801 is used to send third information to the first network element, the third information indicating the call control results of each terminal participating in the multi-party call dialing the access number; and to send each terminal's call media stream to the intelligent media processing device.

[0331] In some embodiments, the fourth communication interface 1801 is used to send a ninth message to the intelligent media processing device. The ninth message is used to notify one or more of the following events: adjusting the forwarding and mixing / screen merging relationships of the remaining media streams of the remaining terminals other than the initiating terminal, updating the screen display information of the remaining terminals, and updating the list information of the remaining terminals.

[0332] It should be noted that the specific processing procedures of the fourth communication interface 1801 and the fourth processor 1802 can be understood by referring to the above method, and will not be repeated here.

[0333] Of course, in practical applications, the various components in the second network element 1800 are coupled together through the fourth bus system 1804. It can be understood that the fourth bus system 1804 is used to realize the connection and communication between these components. In addition to the data bus, the fourth bus system 1804 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 18 The general designated all buses as the fourth bus system 1804.

[0334] The fourth memory 1803 in this embodiment is used to store various types of data to support the operation of the second network element 1800. Examples of such data include any computer program used to operate on the second network element 1800.

[0335] The methods disclosed in the embodiments of this application can be applied to, or implemented by, the fourth processor 1802. The fourth processor 1802 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware or by instructions in the form of software within the fourth processor 1802. The fourth processor 1802 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The fourth processor 1802 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, specifically a fourth memory 1803. The fourth processor 1802 reads information from the fourth memory 1803 and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0336] In an exemplary embodiment, the second network element 1800 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0337] To implement the terminal-side method of this application embodiment, this application embodiment also provides a terminal, such as... Figure 19 As shown, the terminal 1900 includes: a fifth communication interface 1901 and a fifth processor 1902; wherein, the fifth communication interface 1901 is capable of exchanging information with the first network element, the intelligent media processing device, and the application server;

[0338] The fifth processor 1902 is connected to the fifth communication interface 1901 to enable information interaction with the first network element, the intelligent media processing device, and the application server, and to execute the methods provided by one or more of the above-mentioned terminal-side technical solutions when running computer programs;

[0339] The fifth memory 1903 stores computer programs that can run on the fifth processor 1902.

[0340] The fifth communication interface 1901 is used to send a call request to the first network element to dial the access number for a multi-party call; and to receive the mixed audio and screen media stream sent by the first network element.

[0341] In some embodiments, the fifth communication interface 1901 is used to send a fifth message to the application server when the terminal is the initiating terminal participating in a multi-party call. The fifth message instructs the initiating terminal to hang up and exit the multi-party call.

[0342] In some embodiments, the fifth communication interface 1901 is used to receive updated screen display information and / or updated list information sent by the intelligent media processing device when the terminal is a participating terminal in a multi-party call.

[0343] In some embodiments, the fifth communication interface 1901 is used to send tenth information to the service acceptance server when the terminal is the initiating terminal participating in a multi-party call. The tenth information is used to activate the call service and apply for the access number corresponding to the multi-party call. The tenth information includes the terminal's main number. The terminal also receives eleventh information sent by the service acceptance server. The eleventh information indicates that the main number and the generated access number have been successfully bound. The eleventh information includes the access number.

[0344] It should be noted that the specific processing procedures of the fifth communication interface 1901 and the fifth processor 1902 can be understood by referring to the above method, and will not be repeated here.

[0345] Of course, in practical applications, the various components in terminal 1900 are coupled together via the fifth bus system 1904. It can be understood that the fifth bus system 1904 is used to implement communication between these components. In addition to the data bus, the fifth bus system 1904 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 19 The Lieutenant General labeled all buses as the Fifth Bus System 1904.

[0346] The fifth memory 1903 in this embodiment is used to store various types of data to support the operation of the terminal 1900. Examples of such data include any computer program used to operate on the terminal 1900.

[0347] The methods disclosed in the embodiments of this application can be applied to, or implemented by, the fifth processor 1902. The fifth processor 1902 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware or by instructions in the form of software within the fifth processor 1902. The fifth processor 1902 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The fifth processor 1902 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, specifically a fifth memory 1903. The fifth processor 1902 reads information from the fifth memory 1903 and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0348] In an exemplary embodiment, terminal 1900 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0349] In an exemplary embodiment, the user terminal may be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to perform the aforementioned method.

[0350] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or both. Specifically, non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Sync Link Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0351] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a first memory 1503 storing a computer program, which can be executed by the first processor 1502 of the application server 1500 to complete the steps of the method described above on the application server side. Another example is a second memory 1603 storing a computer program, which can be executed by the second processor 1602 of the intelligent media processing device 1600 to complete the steps of the method described above on the intelligent media processing device side. Yet another example is a third processor 1702 storing a computer program, which can be executed by the third processor 1702 of the first network element 1700 to complete the steps of the method described above on the first network element side. Yet another example is a fourth processor 1802 storing a computer program, which can be executed by the fourth processor 1802 of the second network element 1800 to complete the steps of the method described above on the second network element side. For example, it may include a fifth processor 1902 that stores computer programs, which can be executed by the fifth processor 1902 of the terminal 1900 to complete the steps of the aforementioned method on the terminal side. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0352] In an exemplary embodiment, this application also provides a computer product including a computer program, which can be executed by a first processor 1502 of an application server 1500 to complete the steps of the aforementioned method on the application server side. For example, it includes a second memory 1603 storing the computer program, which can be executed by a second processor 1602 of a smart media processing device 1600 to complete the steps of the aforementioned method on the smart media processing device side. Another example is a third processor 1702 storing the computer program, which can be executed by a third processor 1702 of a first network element 1700 to complete the steps of the aforementioned method on the first network element side. Yet another example is a fourth processor 1802 storing the computer program, which can be executed by a fourth processor 1802 of a second network element 1800 to complete the steps of the aforementioned method on the second network element side. And yet another example is a fifth processor 1902 storing the computer program, which can be executed by a fifth processor 1902 of a terminal 1900 to complete the steps of the aforementioned method on the terminal side.

[0353] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0354] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0355] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A method for processing multi-party calls, characterized in that, Applied to an application server, the method includes: The first network element receives a first message sent by the first network element, the first message indicating the first network element's response to the dialing of access numbers by each terminal participating in the multi-party call, wherein each terminal includes the initiating terminal and the participating terminal; Send a second message to the first network element, the second message instructing the first network element to anchor the calls of each terminal to the second network element; Receive third information sent by the first network element, wherein the third information indicates the call control result sent by the second network element to the first network element; Send a fourth message to the intelligent media processing device. The fourth message is used to notify the media anchoring result and the mixing process. The mixing process includes the intelligent media processing device mixing and merging the call media streams based on the received call media streams from each terminal forwarded by the second network element and in combination with the intelligent service type. The first network element is a VoNR+ capability network element, and the second network element is a VoNR+ media network element.

2. The method according to claim 1, characterized in that, The method further includes: The system receives a fifth message sent by the initiating terminal, which instructs the initiating terminal to hang up and exit the multi-party call. A sixth message is sent to the second network element, the sixth message instructing the second network element to release endpoint resources.

3. The method according to claim 1, characterized in that, The method further includes: When each terminal is in the multi-party call, it receives the seventh information sent by the initiating terminal, the seventh information indicating the type of smart service to be activated; Send an eighth message to the intelligent media processing device, the eighth message instructing the multi-party call to be processed in accordance with the intelligent service type.

4. The method according to claim 3, characterized in that, The intelligent service types include one or more of the following: Intelligent application business; Artificial intelligence business; Expand existing business operations.

5. A method for processing multi-party calls, characterized in that, Applied to intelligent media processing devices, the method includes: Receive a fourth message sent by the application server, the fourth message being used to notify the media anchoring result and the mixing process; Receives various call media streams from each terminal forwarded by the second network element; The media streams of each call are mixed and combined according to the intelligent service type to obtain a mixed and combined media stream. The mixed audio and screen media stream is sent to the first network element; The first network element is a VoNR+ capability network element, and the second network element is a VoNR+ media network element.

6. The method according to claim 5, characterized in that, The method further includes: The system receives a ninth message sent by the second network element. The ninth message is used to notify one or more of the following events: adjusting the forwarding and mixing / screen merging relationships of the remaining media streams of the remaining terminals other than the initiating terminal; updating the screen display information of the remaining terminals; and updating the list information of the remaining terminals. Execute one or more of the events.

7. The method according to claim 6, characterized in that, The method further includes: Send updated screen display information and / or updated list information to the remaining terminals.

8. The method according to claim 6, characterized in that, The method further includes: When each terminal is in the multi-party call, it receives the eighth information sent by the application server. The eighth information indicates that the multi-party call should be processed in combination with the intelligent service type. Accordingly, the mixing and merging of the various call media streams to obtain a mixed and merged media stream includes: The various call media streams are mixed and combined based on the intelligent service type to obtain the mixed and combined media stream.

9. A method for processing multi-party calls, characterized in that, Applied to the first network element, the method includes: Receive call requests from terminals participating in a multi-party call to dial the access number; Generate first information, which indicates the first network element's response event to each terminal dialing the access number, wherein each terminal includes an initiating terminal and a participating terminal; Send the first information to the application server; The first network element receives a second message sent by the application server, the second message instructing the first network element to anchor the calls of each terminal to the second network element; Create an endpoint to anchor the calls of each terminal to the second network element; Receive third information sent by the second network element, the third information indicating the call control result of each terminal dialing the access number; Send the third information to the application server; Receive mixed and screen-combined media streams sent by intelligent media processing devices; The mixed audio and screen merging media stream is sent to each of the terminals; The first network element is a VoNR+ capability network element, and the second network element is a VoNR+ media network element. The mixed audio and screen media stream is obtained by the intelligent media processing device combining the intelligent service type with the mixed audio and screen media streams of each terminal.

10. A method for processing multi-party calls, characterized in that, Applied to the second network element, including: Send a third message to the first network element, the third message indicating the call control result of each terminal participating in the multi-party call dialing the access number; Send each call media stream from each terminal to the intelligent media processing device; The first network element is a VoNR+ capability network element, and the second network element is a VoNR+ media network element. The various call media streams from each terminal are used by the intelligent media processing device to mix and combine the various call media streams based on the received second network element forwarded by each terminal, combined with the intelligent service type.

11. The method according to claim 10, characterized in that, The method further includes: Send a ninth message to the intelligent media processing device. The ninth message is used to notify one or more of the following events: adjust the forwarding and mixing / screen merging relationships of the remaining media streams of the remaining terminals other than the initiating terminal, update the screen display information of the remaining terminals, and update the list information of the remaining terminals.

12. A method for processing multi-party calls, characterized in that, Applied to a terminal, the method includes: Send a call request to the first network element to dial the access number for a multi-party call; Receive the mixed audio and screen combined media stream sent by the first network element; The first network element is a VoNR+ capability network element; The mixed audio and screen media stream is obtained by the intelligent media processing device by mixing and merging the various call media streams of each terminal forwarded by the received second network element, combined with the intelligent service type. The second network element is a VoNR+ media network element; The first network element is a network element that sends first information to the application server. The first information is information indicating that the first network element responds to the dialing of access numbers by each terminal participating in the multi-party call. Each terminal includes the initiating terminal and the participating terminal. The application server is a server that sends second information to the first network element. The second information is information that instructs the first network element to anchor the calls of each terminal to the second network element. The application server is a server that receives third information sent by the first network element, and the third information is information instructing the second network element to send a call control result to the first network element.

13. The method according to claim 12, characterized in that, The method further includes: When the terminal is the initiating terminal participating in the multi-party call, it sends a fifth message to the application server, the fifth message instructing the initiating terminal to hang up and exit the multi-party call.

14. The method according to claim 13, characterized in that, The method further includes: When the terminal is a participating terminal in the multi-party call, it receives updated screen display information and / or updated list information sent by the intelligent media processing device.

15. The method according to claim 12, characterized in that, The mixed audio and screen media stream is obtained by intelligent media processing equipment combining the intelligent service type to mix and screen the various call media streams of each terminal participating in the multi-party call.

16. The method according to claim 12, characterized in that, The method further includes: When the terminal is the initiating terminal participating in the multi-party call, it sends the tenth information to the service acceptance server. The tenth information is used to activate the call service and apply for the access number corresponding to the multi-party call. The tenth information includes the terminal's primary number. The system receives the eleventh message sent by the service acceptance server. The eleventh message indicates that the primary number and the generated access number have been successfully bound. The eleventh message includes the access number.

17. An application server, characterized in that, include: A first communication interface and a first processor; wherein... The first communication interface is used to receive first information sent by the first network element. The first information indicates the response event of the first network element to dialing access numbers for each terminal participating in the multi-party call. Each terminal includes the initiating terminal and the participating terminal. Send a second message to the first network element, the second message instructing the first network element to anchor the calls of each terminal to the second network element; Receive third information sent by the first network element, wherein the third information indicates the call control result sent by the second network element to the first network element; Send a fourth message to the intelligent media processing device. The fourth message is used to notify the media anchoring result and the mixing process. The mixing process includes the intelligent media processing device mixing and merging the call media streams based on the received call media streams from each terminal forwarded by the second network element and in combination with the intelligent service type. The first network element is a VoNR+ capability network element, and the second network element is a VoNR+ media network element.

18. An intelligent media processing device, characterized in that, include: A second communication interface and a second processor; wherein... The second communication interface is used to receive fourth information sent by the application server, the fourth information being used to notify the media anchoring result and the mixing process; Receives various call media streams from each terminal forwarded by the second network element; The second processor is used to combine the intelligent service type to mix and merge the various call media streams to obtain a mixed and merged media stream; The second communication interface is used to send the mixed audio and screen media stream to the first network element; The first network element is a VoNR+ capability network element, and the second network element is a VoNR+ media network element.

19. A first network element, characterized in that, include: A third communication interface and a third processor; wherein... The third communication interface is used to receive call requests from each terminal participating in a multi-party call to dial an access number; The third processor is used to generate first information, which indicates the first network element's response event to each terminal dialing the access number, wherein each terminal includes an initiating terminal and a participating terminal; The third communication interface is used to send the first information to the application server; The first network element receives a second message sent by the application server, the second message instructing the first network element to anchor the calls of each terminal to the second network element; The third processor is used to create endpoints and anchor the calls of each terminal to the second network element; The third communication interface is used to receive third information sent by the second network element, the third information indicating the call control result of each terminal dialing the access number; Send the third information to the application server; Receive mixed and screen-combined media streams sent by intelligent media processing devices; The mixed audio and screen merging media stream is sent to each of the terminals; Wherein, the first network element is a VoNR+ capability network element, and the second network element is a VoNR+ media network element; the mixed audio and screen media stream is obtained by the intelligent media processing device mixing and screening the various call media streams of each terminal in combination with the intelligent service type.

20. A second network element, characterized in that, include: The fourth communication interface and the fourth processor; among which, The fourth communication interface is used to send fourth information to the first network element, the fourth information indicating the call control result of each terminal participating in the multi-party call dialing the access number; Send each call media stream from each terminal to the intelligent media processing device; The first network element is a VoNR+ capability network element, and the second network element is a VoNR+ media network element. The various call media streams from each terminal are used by the intelligent media processing device to mix and combine the various call media streams based on the received second network element forwarded by each terminal, combined with the intelligent service type.

21. A terminal, characterized in that, include: The fifth communication interface and the fifth processor; among which, The fifth communication interface is used to send a call request to the first network element to dial an access number for a multi-party call; Receive the mixed audio and screen combined media stream sent by the first network element; The first network element is a VoNR+ capability network element; The mixed audio and screen media stream is obtained by the intelligent media processing device by mixing and merging the various call media streams of each terminal forwarded by the received second network element, combined with the intelligent service type. The second network element is a VoNR+ media network element; The first network element is a network element that sends first information to the application server. The first information is information indicating that the first network element responds to the dialing of access numbers by each terminal participating in the multi-party call. Each terminal includes the initiating terminal and the participating terminal. The application server is a server that sends second information to the first network element. The second information is information that instructs the first network element to anchor the calls of each terminal to the second network element. The application server is a server that receives third information sent by the first network element, and the third information is information instructing the second network element to send a call control result to the first network element.

22. A communication device, characterized in that, include: A processor and memory for storing computer programs that can run on the processor; wherein, When the processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 4, or 5 to 8, or 9, or 10 to 11, or 12 to 16.

23. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4, or 5 to 8, or 9, or 10 to 11, or 12 to 16.

24. A computer product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4, or 5 to 8, or 9, or 10 to 11, or 12 to 16.

Images