Audio and video call and message processing methods and system

Abstract

Provided in the embodiments of the present disclosure are audio and video call and message processing methods and system, a method comprising: during the process of an audio and video call between a first user terminal and a second user terminal, acquiring an uplink message triggered by the first user terminal or the second user terminal; and forwarding the uplink message to a message platform, the uplink message being used for instructing the message platform to send a corresponding downlink message to the first user terminal and / or the second user terminal. The present disclosure solves the problems in the prior art of poor timeliness of new calling media content, limited media resources and extremely low user usage rate of 5G messaging. Integrating new calling with 5G messaging and providing corresponding downlink messages to user terminals improve the media richness and media real-time performance of new calling, and also improve the user usage rate of 5G messaging.

Description

Audio and video call and message processing method and system

[0001] Cross-reference to Related Applications

[0002] The present disclosure is based on Chinese Patent Application No. CN2024108398063 entitled "Audio and video call and message processing method and system" filed on June 26, 2024, and claims priority to the patent application, the disclosure of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] Embodiments of the present disclosure relate to the field of communication technology, in particular to an audio and video call and message processing method and system. BACKGROUND

[0004] The new call platform-media library is highly dependent on the self-built media library of the business, and needs the user or the operator to pre-deliver the media, which has extremely low real-time performance. At the same time, the self-built media is limited by space, and the richness of the content is greatly limited. At the same time, due to the influence of Internet instant messaging applications (WeChat, Weibo, etc.), the user usage rate of 5G messages is extremely low.

[0005] In view of the problems of poor timeliness of new call media content, limited media resources, and extremely low user usage rate of 5G messages in the related art, no solutions have been proposed. SUMMARY

[0006] Embodiments of the present disclosure provide an audio and video call and message processing method and system to at least solve the problems of poor timeliness of new call media content, limited media resources, and extremely low user usage rate of 5G messages in the related art.

[0007] According to one embodiment of the present disclosure, an audio and video call and message processing method is provided, applied to a call management platform, and the method comprises:

[0008] acquiring an uplink message triggered by the first user terminal or the second user terminal in the process of audio and video call between the first user terminal and the second user terminal;

[0009] forwarding the uplink message to a message platform, wherein the uplink message is used to instruct the message platform to send a corresponding downlink message to the first user terminal and / or the second user terminal.

[0010] According to another embodiment of the present disclosure, an audio and video call and message processing method is also provided, applied to a message platform, and the method comprises:

[0011] receive an uplink message forwarded by the call management platform, wherein the uplink message is triggered by the first user terminal or the second user terminal and acquired by the call management platform during the audio / video call between the first user terminal and the second user terminal;

[0012] send a corresponding downlink message to the first user terminal and / or the second user terminal.

[0013] According to another embodiment of the present disclosure, an audio / video call and message processing system is also provided, which at least comprises a call management platform and a message platform, wherein,

[0014] the call management platform is configured to acquire an uplink message triggered by the first user terminal or the second user terminal during the audio / video call between the first user terminal and the second user terminal, and forward the uplink message to the message platform;

[0015] the message platform is configured to receive the uplink message, and send a corresponding downlink message to the first user terminal and / or the second user terminal.

[0016] According to still another embodiment of the present disclosure, a computer program product is also provided, which comprises computer program instructions, wherein the computer program instructions enable a computer to implement the steps in any of the above method embodiments.

[0017] According to still another embodiment of the present disclosure, a computer readable storage medium is also provided, which stores a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0018] According to still another embodiment of the present disclosure, an electronic device is also provided, which comprises a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above method embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0019] FIG. 1 is a hardware structure block diagram of a computer device for an audio / video call and message processing method according to an embodiment of the present disclosure;

[0020] FIG. 2 is a flowchart one of an audio / video call and message processing method according to an embodiment of the present disclosure;

[0021] FIG. 3 is a flowchart two of an audio / video call and message processing method according to an embodiment of the present disclosure;

[0022] FIG. 4 is a flowchart one of message fusion of a 5G new call platform and a 5G message platform according to an embodiment of the present disclosure;

[0023] FIG. 5 is a flowchart II of message fusion of a 5G new call platform and a 5G message platform according to an embodiment of the present disclosure;

[0024] FIG. 6 is a block diagram of an audio / video call and message processing system according to an embodiment of the present disclosure;

[0025] FIG. 7 is a structural schematic diagram of a new call platform and a 5G message platform according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings and in conjunction with embodiments.

[0027] It should be noted that the terms "first", "second", and the like in the specification and claims of the present disclosure and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence.

[0028] The method embodiments provided in the embodiments of the present disclosure can be executed in a computer device or a similar computing device. Taking a computer device as an example, FIG. 1 is a hardware structural block diagram of a computer device of an audio / video call and message processing method according to an embodiment of the present disclosure, as shown in FIG. 1, the computer device can include one or more (only one is shown in FIG. 1) processors 102 (the processor 102 can include but is not limited to a processing device such as a microprocessor MCU or programmable logic device) and a memory 104 for storing data, wherein the above-mentioned computer device can further include a transmission device 106 for communication function and an input / output device 108. Those skilled in the art can understand that the structure shown in FIG. 1 is only schematic, which does not limit the structure of the above-mentioned computer device. For example, the computer device can further include more or less components than those shown in FIG. 1, or have a different configuration from that shown in FIG. 1.

[0029] The memory 104 can be used to store computer programs, for example, software programs of application software and modules, such as a computer program corresponding to the audio / video call and message processing method in the embodiments of the present disclosure. The processor 102 executes various functional applications and single board matching by running the computer program stored in the memory 104, that is, implements the above-mentioned method. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, which can be connected to the computer device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0030] The transmission device 106 is configured to receive or send data via a network. The network can include a wireless network provided by a communication provider of the computer device. In an example, the transmission device 106 includes a network interface controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In an example, the transmission device 106 can be a radio frequency (RF) module configured to communicate with the Internet in a wireless manner.

[0031] In the embodiment, an audio / video call and message processing method running on the computer device is provided. FIG. 2 is a flowchart of an audio / video call and message processing method according to an embodiment of the present disclosure, which is applied to a call management platform and includes the following steps:

[0032] In step S202, an uplink message triggered by the first user terminal or the second user terminal is obtained during the audio / video call between the first user terminal and the second user terminal.

[0033] In step S204, the uplink message is forwarded to a message platform, and the uplink message is used to instruct the message platform to send a corresponding downlink message to the first user terminal and / or the second user terminal.

[0034] Through the above steps S202 to S204, the problems of poor timeliness of new call media content, limited media resources, and extremely low user usage rate of 5G messages in the related art can be solved. The new call and 5G message are fused, and a corresponding downlink message is provided to the user terminal, which improves the media richness and media real-time performance of the new call and improves the user usage rate of the 5G message.

[0035] In the embodiment of the present disclosure, the above step S202 can specifically include:

[0036] In S2021, semantic analysis is performed on audio / video data during the audio / video call between the first user terminal and the second user terminal to obtain a semantic analysis result, and the uplink message triggered by the first user terminal or the second user terminal is obtained according to the semantic analysis result; or

[0037] In S2022, the uplink message triggered by the first user terminal or the second user terminal through an icon or a key in an audio / video call interface is obtained during the audio / video call between the first user terminal and the second user terminal.

[0038] In step S2021, the uplink message triggered by the first user terminal or the second user terminal is obtained according to the semantic analysis result, which can include: identifying the user intention according to the semantic analysis result; sending the user intention to the first user terminal or the second user terminal, so that the first user terminal or the second user terminal initiates the uplink message according to the user intention, and receives the uplink message.

[0039] Further, the user intention is sent to the first user terminal or the second user terminal, which can include: associating the user intention with the user setting data to obtain the chatbot information corresponding to the user intention; obtaining the intention key corresponding to the user intention according to the pre-set correspondence between different user intentions and intention keys; sending the chatbot information corresponding to the user intention and the intention key to the first user terminal or the second user terminal, so that the first user terminal or the second user terminal can obtain the trigger instruction according to the intention key to trigger the uplink message, and the uplink message carries the chatbot information corresponding to the user intention.

[0040] In an embodiment, the method further includes: receiving the media message sent by the message platform, and then storing the media message, i.e. storing the media message in the call management platform; triggering the media message (audio, video, text, etc.) in the message platform through the call management platform real-time message to copy the media message to the call management platform, which can solve the problem of limited 5G new call media resources and low real-time performance. Then, the media message is sent to the first user terminal and / or the second user terminal, so that the first user terminal and / or the second user terminal can display the media message through the audio / video call interface, and the media message can be directly displayed through the audio / video call interface.

[0041] In another embodiment, the method further includes: authenticating the uplink message; and converting the format of the uplink message to a message format supported by the message platform after successful authentication, so as to facilitate information interaction between the call management platform and the message platform. The message interaction is performed through mutual authentication system. Real-time message interaction capability is provided, and the corresponding authentication key can be carried in the message header during message interaction to perform system authentication.

[0042] Based on the architecture fusion of the call management platform and the message platform, the media richness of the call management platform is improved, and the media real-time performance is improved. Meanwhile, through AI processing of the media in the audio and video call process, the semantics are analyzed, the chatbot of a specific subscription is called, and the 5G message experience is provided to the user according to the semantics. In this way, the channels of the two services of the call management platform and the message platform are connected, the problem of media resource limitation of the call management platform is solved. At the same time, the problem of user stickiness that is currently most concerned by the two applications is also solved. As a basic scene, the audio and video call has a high user usage rate, and the access point is used to connect the user side traffic of 5G new call and 5G message, thereby providing more development possibilities for subsequent operator value-added services.

[0043] The embodiment of the present disclosure further provides an audio and video call and message processing method, and Fig. 3 is a flowchart two of the audio and video call and message processing method according to the embodiment of the present disclosure. As shown in Fig. 3, the method is applied to a message platform, and the method comprises the following steps:

[0044] In step S302, an uplink message forwarded by a call management platform is received, the uplink message being triggered by a first user terminal or a second user terminal and acquired by the call management platform in an audio and video call process of the first user terminal and the second user terminal.

[0045] In step S304, a corresponding downlink message is sent to the first user terminal and / or the second user terminal.

[0046] Through the above steps S302 to S304, the problems of poor timeliness of new call media content, limited media resources, and extremely low user usage rate of 5G messages in the related art can be solved. The new call and the 5G message are fused, the corresponding downlink message is provided to the user terminal, the media richness and the media real-time performance of the new call are improved, and the user usage rate of the 5G message is also improved.

[0047] In an embodiment, after the above step S302, the method further comprises: transmitting the uplink message to a MAAP platform (Massaging as a Platform, message as a platform) in a transparent manner, so that the MAAP platform triggers a corresponding chatbot platform according to robot chatbot information carried by the uplink message; and receiving a corresponding downlink message sent by the chatbot platform and forwarded by the MAAP platform.

[0048] In the embodiment of the present disclosure, the above step S304 can specifically comprise:

[0049] S3041, in the case where the downlink message corresponding to the user intention is theme information, sending the theme information to the first user terminal and / or the second user terminal, so that the first user terminal and / or the second user terminal display the theme information through a message interface;

[0050] S3041, in the case where the downlink message corresponding to the user intention is theme information, sending the theme information to the first user terminal and / or the second user terminal, so that the first user terminal and / or the second user terminal display the theme information through a message interface;

[0051] S3043, in the case where the downlink message corresponding to the user intention includes theme information and audio / video data, sending the theme information and the audio / video data to the first user terminal and / or the second user terminal, so that the first user terminal and / or the second user terminal display the theme information through a message interface and display the audio / video data through an audio / video call interface.

[0052] The call management platform in the embodiments of the present disclosure can be a new call platform, which is used to manage 5G new calls. The message platform can be a 5G message platform, which is used to manage 5G messages. Based on the fusion of 5G new calls and 5G messages, the two hot applications of 5G new calls and 5G messages are fused, which improves user stickiness and enhances user experience, and solves the problem of limited media resources of 5G new calls. The media resources in the 5G message platform are triggered by real-time messages of 5G new calls, and the media information (including audio, video, text, etc.) is copied to the 5G new call platform, which solves the problems of limited media resources of 5G new calls and low real-time performance. Through the call basic scene as an entrance and the new call as a basis, better user increment is brought to 5G messages. Through the fusion of 5G new calls and 5G messages, the user's online and offline experience is improved for the same scene. Online users can watch 5G new call content, and offline users can obtain more rich 5G message information experience. The 5G new call platform and the 5G message platform are fused at the system level through inter-system real-time message triggering, and through the integration of the two system services, the interaction of the system internal messages is realized, and the real-time performance of the message interaction is improved, which better improves the user application experience.

[0053] The embodiments of the present disclosure can be applied to many new business scenarios in the combination of 5G new call and 5G message. In the process of 5G new call, the user's intention is recognized by AI, the uplink message is sent to the 5G message platform through the business module, and the corresponding chatbot is triggered to issue the corresponding 5G message to the user terminal. The user can obtain more rich offline experience (5G message) in the new call process. The media information of the current new call message is limited to the media distribution platform of the operator, and the diversity is insufficient. The 5G new call and 5G message fusion solution proposed in the embodiments of the present disclosure can trigger the 5G message platform to issue the chatbot media information to the 5G new call media server in real time, so as to realize online presentation in the 5G new call application. The system interaction of online (5G new call) and offline (5G message) of the user in the process of 5G new call scenario is provided, and better scene experience is brought to the user.

[0054] The 5G new call platform triggers the 5G message platform to issue the chatbot information of the corresponding scene in real time. FIG. 4 is a flowchart of message fusion of the 5G new call platform and the 5G message platform according to the embodiments of the present disclosure. As shown in FIG. 4, the process specifically includes the following steps.

[0055] In step S401, a calling user (UE-A) subscribes to a 5G new call service and triggers a new call service to trigger a 5G new call platform.

[0056] In step S402, the 5G new call platform parses the received 5G new call information, encapsulates the parsed content, provides corresponding new call scene functions (such as background replacement, translation, transcription, etc.) according to the uploaded terminal information and the user's new call setting data, and triggers a new call media processing module to provide corresponding media scenes for the terminal.

[0057] In step S403, the 5G new call platform performs semantic analysis on the media content (audio, video) through a third-party AI processing platform, identifies the intention scene, and associates the received intention scene with the user's setting data (such as data of business scene and interface operation button configuration).

[0058] In step S404, the 5G new call platform triggers the corresponding business according to the corresponding intention scene, and displays the scene operation button on the calling user terminal interface.

[0059] In step S405, the 5G new call platform receives the uplink message with the intention corresponding chatbot information triggered by the calling user according to the operation information displayed on the terminal interface.

[0060] In step S406, the corresponding message is forwarded to the 5G message platform.

[0061] Step S407, the 5G message platform transmits the corresponding uplink message to the MAAP platform, and the MAAP platform triggers the corresponding chatbot platform according to the chatbot information carried by the uplink message, and makes the chatbot platform send the downlink message of the corresponding intent scene to the terminals of the calling user and the called user (UE-B).

[0062] Step S408, the 5G message platform sends the chatbot information related to the intent scene of the new call to the terminals of the calling user and the called user through the downlink message.

[0063] Step S409, the calling user and the called user receive the corresponding chatbot downlink message and browse it, and operate the corresponding 5G message.

[0064] For example, the calling user (UE-A) and the called user (UE-B) both subscribe to the 5G new call (translation, transcription, etc.) service, and the calling user dials the new call to the called user. After the called user answers the call, the 5G new call transcribes and displays the voice during the call between the calling user and the called user to the user terminal. During the call between the calling user and the called user, the calling user tells the called user to go shopping in a certain mall. Due to the audio and video media during the call between the calling user and the called user, the third-party AI processing platform is interacted with to recognize the shopping scene. Based on the recognized shopping scene in the certain mall, the scene operation button and the chatbot information corresponding to the scene (shopping in the certain mall) set by the user are queried, and the obtained information triggers the 5G new call service to be displayed on the UE-A terminal, and the shopping in the certain mall theme chatbot message is sent to the button (5G message is sent to the called UE-B). After receiving the key control information, the uplink message (carrying chatbot information) is triggered, the uplink message is authenticated, and after the authentication is successful, the message is sent to the 5G message platform. The 5G message platform triggers the shopping in the certain mall chatbot to send the mall link 5G message to the terminals of UE-A and UE-B through the MAAP platform. The calling user and the called user receive the 5G message (mall link) corresponding to the scene (shopping in the certain mall) during the call, and the user can operate the mall link sent by the 5G message during the call to perform shopping operations, thereby achieving the effect of online and offline synchronous experience.

[0065] In addition to the above-mentioned shopping in the certain mall that can trigger the 5G message to send the mall link to the user terminal, any scene can be recognized to send the 5G message. For example, a travel scene is recognized during the call, and the corresponding chatbot can be triggered to send the 5G message of ticket purchase, accommodation, and scenic spot introduction to the user terminal. The user can obtain the corresponding information in real time and perform related operations.

[0066] 5G new call triggers 5G message platform to send chatbot media information to 5G new call platform, and the 5G new call platform plays corresponding media information on the user terminal. FIG. 5 is a flowchart II of message fusion of the 5G new call platform and the 5G message platform according to an embodiment of the present disclosure, and as shown in FIG. 5, the process specifically includes the following steps.

[0067] S501-S504, which are the same as the above steps S401-S404 respectively;

[0068] S505, the calling user (UE-A) clicks the media download button according to the operation information displayed on the terminal and notifies the new call platform;

[0069] S506, the new call platform triggers the download message (carrying the chatbot information corresponding to the intent) and uploads the downloaded message to the 5G message platform;

[0070] S507, the 5G message platform transmits the corresponding download message to the MAAP platform, the MAAP platform triggers the corresponding chatbot platform according to the chatbot information carried by the download message, and receives the media information sent by the chatbot platform through the MAAP platform;

[0071] S508, the 5G message platform stores the chatbot corresponding media information successfully, and then displays the chatbot corresponding media information to the calling user (UE-A) and the called user (UE-B) respectively.

[0072] For example, UE-A and called UE-B both subscribe to 5G new call (translation, transcription) service, and the caller dials the new call to the called. After the called answers the call, 5G new call transcribes the voice in the process of the main and called call and displays it to the user terminal. During the process of the main and called call, UE-A says happy birthday to the called. Due to the audio and video media in the process of the main and called call, the third-party AI processing platform is interacted with to recognize the scene that the called UE-B has a birthday today. According to the recognized birthday scene, the scene operation button and the chatbot information corresponding to the scene (happy birthday) set by the user are queried, and according to the obtained information, the 5G new call service is triggered to display on the UE-A terminal. The happy birthday theme chatbot media download button (the corresponding media (video, picture, etc.) in the chatbot is downloaded to the 5G new call media server). After receiving the key control information, the received uplink message (carrying chatbot information) triggered is authenticated, and after the authentication is successful, the message is sent to the 5G message platform. The 5G message platform triggers the corresponding chatbot platform through the MAAP platform, the chatbot platform receives the media download information, and the corresponding media is downloaded to the 5G new call platform. The 5G new call platform converts the happy birthday theme chatbot media (video, picture, animation, etc.) and stores it in the media server. After the media download is successful, the 5G new call service is triggered, and the chatbot corresponding to the happy birthday theme media information (video, picture, animation, etc.) is displayed on the user terminal. Through the fusion of 5G new call and 5G message, the media corresponding to the 5G message chatbot platform is played in real time in the 5G new call scene, and the user experiences a more rich experience.

[0073] In addition to the mentioned happy birthday scene real-time obtaining of chatbot platform media information for display on the terminal, all theme scenes can obtain corresponding chatbot media for display on the 5G new call platform. Fusion with the video background, special effects and other functions of the 5G new call.

[0074] The embodiments of the present disclosure also provide an audio and video call and message processing system. FIG. 6 is a block diagram of an audio and video call and message processing system according to an embodiment of the present disclosure. As shown in FIG. 6, the system at least includes a call management platform 62 and a message platform 64, wherein,

[0075] The call management platform 62 is configured to obtain an uplink message triggered by a first user terminal or a second user terminal during an audio and video call process between the first user terminal and the second user terminal, and forward the uplink message to the message platform;

[0076] The message platform 64 is configured to receive the uplink message, and send a corresponding downlink message to the first user terminal and / or the second user terminal.

[0077] In an embodiment, the call management platform 62 is further configured to perform semantic analysis on the audio and video data during the audio and video call between the first user terminal and the second user terminal to obtain a semantic analysis result, and obtain the uplink message triggered by the first user terminal or the second user terminal according to the semantic analysis result; or obtain the uplink message triggered by the first user terminal or the second user terminal through an icon or a button in an audio and video call interface during the audio and video call between the first user terminal and the second user terminal.

[0078] In an embodiment, the call management platform 62 is further configured to identify a user intention according to the semantic analysis result, send the user intention to the first user terminal or the second user terminal, and receive the uplink message initiated by the first user terminal or the second user terminal according to the user intention.

[0079] In an embodiment, the call management platform 62 is further configured to associate the user intention with user setting data to obtain robot chatbot information corresponding to the user intention, obtain an intention button corresponding to the user intention according to a pre-set correspondence between different user intentions and intention buttons, send the chatbot information and the intention button corresponding to the user intention to the first user terminal or the second user terminal, and receive a trigger instruction initiated by the first user terminal or the second user terminal based on the intention button, wherein the trigger instruction is used to trigger the uplink message, and the uplink message carries the chatbot information corresponding to the user intention.

[0080] In an embodiment, the message platform 64 is further configured to transmit the uplink message to the MAAP platform, so that the MAAP platform triggers a corresponding chatbot platform according to the robot chatbot information carried by the uplink message, and receive the downlink message sent by the chatbot platform and forwarded by the MAAP platform, wherein the downlink message carries the chatbot information corresponding to the user intention.

[0081] In an embodiment, the message platform 64 is further configured to, in a case where the downlink message corresponding to the user intention is topic information, send the topic information to the first user terminal and / or the second user terminal; in a case where the downlink message corresponding to the user intention is audio / video data, send the audio / video data to the first user terminal and / or the second user terminal; in a case where the downlink message corresponding to the user intention includes topic information and audio / video data, send the topic information and the audio / video data to the first user terminal and / or the second user terminal; so that the first user terminal and / or the second user terminal displays the topic information through a message interface and / or displays the audio / video data through an audio / video call interface.

[0082] In an embodiment, the call management platform 42 is further configured to authenticate the uplink message, wherein the uplink message carries chatbot information corresponding to the user intention; and convert a format of the uplink message into a message format supported by the message platform after successful authentication.

[0083] The following will take the call management platform 62 as a new call platform and the message platform 64 as a 5G message platform as an example to describe the embodiments of the present disclosure in detail.

[0084] FIG. 7 is a structural schematic diagram of a new call platform and a 5G message platform according to an embodiment of the present disclosure. As shown in FIG. 7, the new call platform is composed of a setting data management module, a signaling analysis module, a service capability module, a service control module, a media processing module, an AI processing module, a message conversion module, a media processor, an AI agent, and the like.

[0085] The setting data management module is configured to mainly provide user and system new call and 5G message setting functions, and provide basic data for the service control module.

[0086] The signaling analysis module is configured to be responsible for analyzing the underlying core network sip signaling, and is responsible for media playing end application and other related operations with the media processing module.

[0087] The service capability module is connected with the service control module and the signaling analysis module, and is configured to provide a capability interface of the underlying module for the upper service control module.

[0088] The service control module is configured to send a control instruction to the signaling analysis module or the media processing module through the capability module according to user settings, system settings, and third-party instructions.

[0089] The media processing module is configured to perform media port application, media synthesis, media playback, and the like according to upper instructions or media streams.

[0090] An AI processing module is configured to analyze new call audio and video media information and perform AI analysis to extract user semantics and identify user intent. The AI processing module is also configured to provide an interface of a third-party AI processing platform (such as an interface of a speech recognition system of iFLYTEK or an interface of a Baidu AI system) to connect to an external AI system.

[0091] An AI Agent module is configured to associate user intent identified by the AI processing module with user setting data, so as to trigger a 5G new call service control module to provide corresponding new call service functions (mainly to trigger the 5G new call to display operation instructions corresponding to the user intent on the user terminal interface).

[0092] A message conversion module is configured to link message interaction between the 5G new call and the 5G message platforms, process and convert messages between different platforms, and provide extensible message processing capabilities. At the same time, the module provides authentication capabilities for messages delivered by the two systems, checks incoming and outgoing messages, and prevents illegal messages from being accessed to the respective platforms.

[0093] A media storage module is configured to interface the chatbot platform and the new call platform-media library, accept external media information, and provide image, audio and video media processing and conversion capabilities to meet the 5G new call playback conditions, because the media played by the terminal needs to have a specific media format. When external media data is delivered to the 5G new call platform, media format verification and conversion are required.

[0094] A new call management platform is configured to deliver user management data to a setting data management module through real-time messages. The relevant setting data at least includes intent buttons (such as a button for recognizing a happy birthday keyword in a call process to trigger a 5G uplink message to a 5G message platform).

[0095] Through the 5G message platform, user-subscribed chatbot information is set, including chatbot ID, name, address, and chatbot scene keyword information.

[0096] A setting data management module is configured to report data to the 5G new call management platform according to the user-subscribed chatbot and its scene keyword, and the new call platform assigns corresponding control information (buttons, etc.) according to the scene.

[0097] Through the above management operations, the management data of the 5G message and the 5G new call can be associated, and basic user data for business management of the 5G new call platform can be provided.

[0098] In an embodiment, the 5G new call triggers the chatbot information of the corresponding scene under the 5G message platform in real time. The calling user (UE-A) subscribes to the 5G new call service and triggers the new call service. Through the core network, the signaling processing module of the 5G new call platform is triggered.

[0099] The signaling processing module is configured to parse the received 5G new call information, encapsulate the parsed content, and trigger the new call service control module.

[0100] The service control module is configured to provide corresponding new call scene (such as background replacement, translation, and transcription) functions according to the uploaded terminal information and the new call setting data set by the user, and trigger the new call media processing module to provide corresponding media scenes for the terminal.

[0101] The service control module is configured to control the media processing module to forward the media content (audio and video) to the AI processing module.

[0102] The AI processing module is configured to perform semantic analysis on the corresponding media content (audio and video) and identify the intent scene.

[0103] The AI processing module is configured to send the identified intent scene to the 5G new call AI Agent processing module.

[0104] The AI Agent is configured to associate the received intent scene with the user setting data (such as service scene and interface operation button configuration data), and send a message to the service control module by the AI Agent.

[0105] The service control module is configured to trigger the corresponding service according to the corresponding intent scene, and display the scene operation button on the terminal interface of the calling user (UE-A).

[0106] The calling user (UE-A) triggers the service control module to send an uplink message with the corresponding chatbot information to the message conversion module according to the operation information displayed on the terminal interface.

[0107] The message conversion module is configured to forward the corresponding message to the 5G message platform.

[0108] The 5G message platform is configured to transparently transmit the corresponding uplink message to the MAAP platform. The MAAP platform triggers the corresponding chatbot platform according to the chatbot information carried by the uplink message, and makes the chatbot platform send a downlink message of the corresponding intent scene to the terminals of the calling user (UE-A) and the called user (UE-B).

[0109] MAAP, 5G message platform, is set to send the chatbot information related to the intention scene corresponding to the new call to the terminals of user A and user B through the downlink message.

[0110] The calling user (UE-A) and the called user (UE-B) receive the corresponding chatbot downlink message and browse it, and operate the corresponding 5G message.

[0111] The following is illustrated with specific examples.

[0112] 1. Both UE-A and called UE-B subscribe to 5G new call (translation, transcription, etc.) services, and the calling party dials the new call to the called party.

[0113] 2. After the called party answers the call, the 5G new call transcribes and displays the voice during the call between the calling party and the called party to the user terminal.

[0114] 3. During the call between the calling party and the called party, UE-A tells the called party to shop at a certain mall. Due to the audio and video media during the call between the calling party and the called party, the AI processing module recognizes the shopping scene by interacting with the third-party AI processing platform.

[0115] 4. The AI processing module is set to forward the recognized shopping scene at a certain mall to the AI Agent, which queries the scene operation button and the chatbot information corresponding to the scene (shopping at a certain mall) set by the user, and forwards the obtained information to the service control module.

[0116] 5. The service control module is set to trigger the 5G new call service to display on the UE-A terminal after receiving the message from the AI Agent, and to issue a shopping theme chatbot message button (to issue a 5G message to the called UE-B).

[0117] 6. The service control module is set to trigger the message (carrying the chatbot information) to the message forwarding module after receiving the key control information, and the message forwarding module is set to authenticate the forwarded message, and send the message to the 5G message platform after successful authentication.

[0118] 7. The 5G message platform triggers the chatbot of a certain mall to issue a mall link 5G message to the terminals of the calling user (UE-A) and the called user (UE-B) through the MAAP platform according to the corresponding information.

[0119] 8. The calling user and the called user terminal receive the 5G message (mall link) corresponding to the scene (shopping at a certain mall) during the user call.

[0120] 9, the user can operate the mall link issued by the 5G message during the call to make purchases and other operations, so as to achieve the effect of online and offline synchronous experience.

[0121] 5G new call triggers 5G message platform to issue chatbot media information to 5G new call platform, and 5G new call platform plays corresponding media information on user terminal, calling user (UE-A) subscribes to 5G new call service and triggers new call service, and through core network, signaling processing module of 5G new call platform is triggered;

[0122] The signaling processing module is configured to parse the received 5G new call information, encapsulate the parsed content, and trigger the new call service control module.

[0123] The service control module is configured to provide corresponding new call scene (such as background replacement, translation, transcription, etc.) functions according to uploaded terminal information and user set new call setting data, and trigger the new call media processing module to provide corresponding media scene to the terminal.

[0124] The service control module is configured to control the media processing module to forward media content (audio, video) to the AI processing module.

[0125] The AI processing module is configured to perform semantic analysis on the corresponding media content (audio, video) and identify the intent scene.

[0126] The AI processing module is configured to send the identified intent scene to the 5G new call AI Agent processing module.

[0127] The AI Agent associates the received intent scene with user setting data (service scene and interface operation button configuration data, etc.), and sends a message to the 5G new call service control module by the AI Agent.

[0128] The service control module is configured to trigger corresponding services according to the corresponding intent scene, and display scene operation buttons on the calling user (UE-A) terminal interface.

[0129] The calling user (UE-A) clicks the media download button according to the operation information displayed on the terminal interface, and notifies the new call platform service control module.

[0130] The service control module is configured to trigger the download message (carrying the chatbot information corresponding to the intent) to the message conversion module.

[0131] The message conversion module is configured to upload the download message to the 5G message platform, and the 5G message platform transmits the corresponding download message to the MAAP platform.

[0132] The MAAP platform is configured to trigger a corresponding chatbot platform according to chatbot information carried by a download message, and trigger the platform to distribute media information to a 5G new call platform, store chatbot corresponding media information to a new call platform-media library, and send a message to a service control module after successful storage.

[0133] The service control module is configured to trigger a media processing module to display corresponding chatbot media information to a terminal of a calling user (UE-A) and a called user (UE-B) respectively.

[0134] The following is illustrated by a specific example.

[0135] 1. Both the UE-A and the called UE-B subscribe to a 5G new call (translation, transcription) service, and the calling party dials the new call to the called party.

[0136] 2. After the called party answers the call, the 5G new call transcribes and displays the voice during the process of the calling and called parties to the user terminal.

[0137] 3. During the process of the calling and called parties, the UE-A says happy birthday to the called party. Since the audio and video media during the process of the calling and called parties are copied to an AI processing module, the AI processing module recognizes the scenario that the called UE-B has a birthday today by interacting with a third-party AI processing platform.

[0138] 4. The AI processing module forwards the recognized birthday scenario to an AI Agent, the AI Agent queries a scenario operation button and chatbot information corresponding to a scenario (happy birthday) set by a user, and forwards the obtained information to a service control module.

[0139] 5. After the service control module receives the message from the AI Agent, the 5G new call service is triggered to display a happy birthday theme chatbot media download button (download corresponding media (video, picture, etc.) in the chatbot to a 5G new call media server) on the UE-A terminal.

[0140] 6. After the service receives the key control information, a message (carrying chatbot information) is triggered to be sent to a message forwarding module, the message forwarding module authenticates the forwarded message, and sends the message to a 5G message platform after successful authentication.

[0141] 7. The 5G message platform triggers a corresponding chatbot through the MAAP platform, the chatbot platform distributes corresponding media to the 5G new call platform after receiving the media download information, and the 5G new call platform converts the happy birthday theme chatbot media (video, picture, animation, etc.) through a media storage module and stores them in the media server.

[0142] 8. After the media is downloaded successfully, triggering a new call service of 5G, displaying the media information (video, picture, animation, etc.) corresponding to the happy birthday theme of the chatbot on the user terminal;

[0143] 9. Through the integration of the new call and the 5G message, playing the media corresponding to the 5G message chatbot platform in the 5G new call scene in real time, so that the user can experience a more rich experience.

[0144] The embodiment of the present disclosure also provides a computer program product, comprising computer program instructions, wherein the computer program instructions enable a computer to implement the steps in any of the above method embodiments.

[0145] The embodiment of the present disclosure also provides a computer readable storage medium, which stores a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0146] In an example embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0147] The embodiment of the present disclosure also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to perform the steps in any of the above method embodiments.

[0148] In an example embodiment, the above electronic device can further comprise a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0149] The specific examples in the embodiment can refer to the examples described in the above embodiments and example implementations, and the embodiment will not be described here again.

[0150] It is apparent that those skilled in the art should understand that the modules or steps of the present disclosure described above can be realized by general computing devices, which can be centralized on a single computing device or distributed on a network composed of multiple computing devices, and can be realized by program codes executable by the computing devices, so that they can be stored in storage devices and executed by the computing devices, and in some cases, the steps shown or described can be executed in different orders, or they can be respectively manufactured into individual integrated circuit modules, or multiple modules or steps thereof can be manufactured into a single integrated circuit module. Thus, the present disclosure is not limited to any particular combination of hardware and software.

[0151] The preferred embodiments of the present disclosure are described above, but the present disclosure is not limited to the above. Those skilled in the art can make various modifications and changes to the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An audio-video call and message processing method applied to a call management platform, the method comprising: obtaining an uplink message triggered by a first user terminal or a second user terminal during an audio-video call between the first user terminal and the second user terminal; forwarding the uplink message to a message platform, wherein the uplink message is used to instruct the message platform to send a corresponding downlink message to the first user terminal and / or the second user terminal.

2. The method of claim 1, wherein, The step of obtaining an uplink message triggered by a first user terminal or a second user terminal during an audio-video call between the first user terminal and the second user terminal comprises: performing semantic analysis on audio-video data during the audio-video call between the first user terminal and the second user terminal to obtain a semantic analysis result, and obtaining the uplink message triggered by the first user terminal or the second user terminal according to the semantic analysis result; or obtaining the uplink message triggered by the first user terminal or the second user terminal through an icon or a button in an audio-video call interface during the audio-video call between the first user terminal and the second user terminal.

3. The method of claim 2, wherein, The step of obtaining the uplink message triggered by the first user terminal or the second user terminal according to the semantic analysis result comprises: recognizing a user intent according to the semantic analysis result; sending the user intent to the first user terminal or the second user terminal; receiving the uplink message initiated by the first user terminal or the second user terminal according to the user intent.

4. The method of claim 3, wherein, The step of sending the user intent to the first user terminal or the second user terminal comprises: associating the user intent with user setting data to obtain chatbot information corresponding to the user intent; obtaining an intent button corresponding to the user intent according to a pre-set correspondence between different user intents and intent buttons; sending the chatbot information corresponding to the user intent and the intent button to the first user terminal or the second user terminal, wherein the intent button is used to obtain a trigger instruction, and the trigger instruction is used to trigger the uplink message, and the uplink message carries the chatbot information corresponding to the user intent.

5. The method of claim 1, wherein, The method further comprises: receiving a media message sent by the message platform; sending the media message to the first user terminal and / or the second user terminal, so that the first user terminal and / or the second user terminal displays the media message through the audio-video call interface.

6. The method of any one of claims 1 to 5, wherein, The method further comprises: authenticating the uplink message; converting a format of the uplink message into a message format supported by the message platform after successful authentication.

7. An audio-video call and message processing method applied to a message platform, the method comprising: receiving an uplink message forwarded by a call management platform, wherein the uplink message is triggered by a first user terminal or a second user terminal during an audio-video call between the first user terminal and the second user terminal. sending a corresponding downlink message to the first user terminal and / or the second user terminal.

8. The method of claim 7, wherein, The sending of the corresponding downlink message to the first user terminal and / or the second user terminal comprises: when the corresponding downlink message of the user intention is topic information, sending the topic information to the first user terminal and / or the second user terminal, so that the first user terminal and / or the second user terminal display the topic information through a message interface; when the corresponding downlink message of the user intention is audio / video data, sending the audio / video data to the first user terminal and / or the second user terminal, so that the first user terminal and / or the second user terminal display the audio / video data through an audio / video call interface; when the corresponding downlink message of the user intention comprises topic information and audio / video data, sending the topic information and the audio / video data to the first user terminal and / or the second user terminal, so that the first user terminal and / or the second user terminal display the topic information through a message interface and display the audio / video data through an audio / video call interface.

9. An audio video call and message processing system, the system comprising at least: The system comprises a call management platform and a message platform, wherein the call management platform is configured to acquire an uplink message triggered by the first user terminal or the second user terminal during an audio / video call between the first user terminal and the second user terminal, and forward the uplink message to the message platform; the message platform is configured to receive the uplink message, and send a corresponding downlink message to the first user terminal and / or the second user terminal.

10. The system of claim 9, wherein the call management platform is further configured to perform semantic analysis on audio / video data during the audio / video call between the first user terminal and the second user terminal, to obtain a semantic analysis result, and acquire the uplink message triggered by the first user terminal or the second user terminal according to the semantic analysis result; or acquire the uplink message triggered by the first user terminal or the second user terminal through an icon or a button in an audio / video call interface during the audio / video call between the first user terminal and the second user terminal.

11. The system of claim 10, wherein the call management platform is further configured to identify a user intention according to the semantic analysis result, send the user intention to the first user terminal or the second user terminal, and receive the uplink message initiated by the first user terminal or the second user terminal according to the user intention.

12. The system of claim 11, wherein The call management platform is further configured to associate the user intention with user setting data to obtain robot chatbot information corresponding to the user intention, obtain an intention key corresponding to the user intention according to a pre-set correspondence between different user intentions and intention keys, send the chatbot information and the intention key corresponding to the user intention to the first user terminal or the second user terminal, and receive a trigger instruction initiated by the first user terminal or the second user terminal based on the intention key, wherein the trigger instruction is used to trigger the uplink message, and the uplink message carries the chatbot information corresponding to the user intention.

13. The system of claim 11, wherein, The message platform is further configured to send the theme information to the first user terminal and / or the second user terminal when the downlink message corresponding to the user intention is theme information. When the downlink message corresponding to the user intention is audio / video data, the message platform is further configured to send the audio / video data to the first user terminal and / or the second user terminal; when the downlink message corresponding to the user intention includes theme information and audio / video data, the message platform is further configured to send the theme information and the audio / video data to the first user terminal and / or the second user terminal; so that the first user terminal and / or the second user terminal display the theme information through a message interface and / or display the audio / video data through an audio / video call interface.

14. A computer-readable storage medium having stored therein a computer program, wherein, The computer program is configured to execute the method of any one of claims 1-6, 7-8 when running.

15. An electronic device comprising a memory and a processor, the memory having a computer program stored therein, and the processor being configured to execute the computer program to perform the method of any one of claims 1-6, 7-8.

Images