Audio and video call processing method, device and computer equipment based on IVVR

Abstract

The present application relates to an audio and video call processing method, device, computer equipment, storage medium and computer program product based on IVVR. The method includes, when receiving a video call request, calling a multimedia creation server to obtain a target service menu screen from a video page server, and receiving a first playback instruction for the target service menu screen indicated by the video page server; responding to the first playback instruction, playing the prompt tone of the target service menu screen to the call requester, and sending the video of the target service menu screen to the call requester to realize a video call with the call requester; when a voice switching instruction is received during the video call, playing the prompt tone of the target service menu to the call requester to realize a voice call with the call requester. The method realizes audio and video switching based on IVVR, and the switching before and after is undertaken by the same service subject, so the continuity of the call service before and after the switching can be guaranteed.

Description

Technical Field

[0001] The present application relates to the field of mobile Internet technology, and in particular to an IVVR-based audio and video call processing method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of telecommunication technology, interactive voice response (IVR) has become the main form of voice service. Traditional IVR voice calls only support customers to interact with the system in a way that they listen and speak at the same time. Based on IVR technology, visual IVR technology has been developed. With visual IVR technology, when users call customer service, video information of the service menu can also be displayed, thus upgrading the traditional single voice broadcast mode to a video display form.

[0003] Usually, after a user initiates an IVR voice call, if it needs to be switched to a visual IVR video call, it is necessary to switch to the IVVR service to undertake the video call demand. After a user initiates a visual IVR video call, if it needs to be switched to an IVR voice call, it is necessary to switch to the IVR service to undertake the voice call demand. Since the previous and subsequent switching are undertaken by different service entities, it is difficult to ensure the continuity of the call service during the switching process. Summary of the invention

[0004] Based on this, it is necessary to provide an IVVR-based audio and video call processing method, device, computer equipment, computer-readable storage medium and computer program product that can improve the convenience of call mode switching operation during a call in response to the above technical problems.

[0005] In a first aspect, the present application provides an IVVR-based audio and video call processing method. The method comprises:

[0006] When a video call request is received, the multimedia creation server is called to obtain a target service menu screen from a video page server, and a first playback instruction for the target service menu screen is received from the video page server;

[0007] In response to the first playing instruction, playing the prompt tone of the target service menu screen to the call requesting party, and sending the video of the target service menu screen to the call requesting party to realize a video call with the call requesting party;

[0008] When a voice switching instruction is received during the video call, a prompt tone of a target service menu is played to the call requesting party to implement a voice call with the call requesting party.

[0009] In a second aspect, the present application also provides an IVVR-based audio and video call processing method. The method comprises:

[0010] When receiving the voice call instruction, keep requesting the multimedia creation server, the multimedia creation server requests the target service menu screen from the video page server, and receives the second playback instruction for the target service menu screen indicated by the video page server;

[0011] In response to the second voice playing instruction, playing the prompt tone of the target service menu screen to the call requesting party to realize a voice call with the call requesting party;

[0012] When a video switching instruction is received during a voice call, the multimedia creation server is called to request a target service menu screen from a video page server, a prompt tone corresponding to the target service menu screen is played, and a video of the target service menu screen is sent to the call requesting party for a video call.

[0013] In a third aspect, the present application also provides an IVVR-based audio and video call processing device. The device comprises:

[0014] The video processing module is used for, when receiving a video call request, calling the multimedia creation server to obtain a target service menu screen from the video page server, and receiving a first playback instruction for the target service menu screen indicated by the video page server;

[0015] A voice processing module, configured to respond to the first voice playing instruction to play a prompt tone of the target service menu screen to the call requesting party, and send a video of the target service menu screen to the call requesting party to realize a video call with the call requesting party;

[0016] The switching module is used to play a prompt tone of a target service menu to the call requesting party when a voice switching instruction is received during the video call to realize a voice call with the call requesting party.

[0017] In a fourth aspect, the present application also provides an IVVR-based audio and video call processing device. The device comprises:

[0018] A voice processing module, configured to, when receiving a voice call instruction, keep requesting the multimedia creation server, the multimedia creation server requests the target service menu screen from the video page server, and receive a second playback instruction for the target service menu screen indicated by the video page server;

[0019] A sending module, configured to respond to the second voice playing instruction and play the prompt tone of the target service menu screen to the call requesting party to realize a voice call with the call requesting party;

[0020] The switching module is used to call the multimedia creation server to request the target service menu screen from the video page server when a video switching instruction is received during a voice call, play the prompt sound corresponding to the target service menu screen, and send the video of the target service menu screen to the call requester for a video call.

[0021] In a fifth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the IVVR-based audio and video call processing of the above-mentioned embodiments when executing the computer program.

[0022] In a seventh aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the IVVR-based audio and video call processing in the above-mentioned embodiments are implemented.

[0023] In a ninth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the IVVR-based audio and video call processing in the above-mentioned embodiments are implemented.

[0024] The above-mentioned IVVR-based audio and video call processing method, device, computer equipment, storage medium and computer program product,

[0025] When a video call request is received, the IVVR server calls the multimedia creation server to obtain the target service menu screen from the video page server, and receives the first playback instruction for the target service menu screen indicated by the video page server, and plays the prompt tone of the target service menu screen to the call requester according to the first playback instruction and sends the video stream of the target service menu screen to the call requester. When a voice switching instruction is received during the video call, the prompt tone of the target service menu is played to the call requester to realize a voice call with the call requester. This method realizes audio and video switching based on IVVR, and the switching before and after is undertaken by the same service subject, so the continuity of the call service before and after the switching can be guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is an application environment diagram of an IVVR-based audio and video call processing method in an embodiment;

[0027] Figure 2It is a flowchart of an audio and video call processing method based on IVVR in one embodiment;

[0028] Figure 3 It is an interactive schematic diagram of an audio and video call processing method based on IVVR in one embodiment;

[0029] Figure 4 is a schematic diagram of an interface of a terminal in an embodiment;

[0030] Figure 5 A flowchart of steps for implementing a voice call based on a first multimedia conference in one embodiment;

[0031] Figure 6 An interactive schematic diagram of steps for implementing a voice call based on a first multimedia conference in an embodiment;

[0032] Figure 7 It is a flowchart of an audio and video call processing method based on IVVR in another embodiment;

[0033] Figure 8 It is an interactive schematic diagram of an audio and video call processing method based on IVVR in another embodiment;

[0034] Fig. 9 A flowchart of an audio and video call processing method based on IVVR in another embodiment;

[0035] Fig.10 This is an interactive schematic diagram of an audio and video call processing method based on IVVR in yet another embodiment;

[0036] Fig.11 It is a schematic diagram of the principle of implementing an IVVR-based audio and video call processing method in one embodiment;

[0037] Fig.12 It is a structural block diagram of an audio and video call processing device based on IVVR in one embodiment;

[0038] Fig.13 It is a structural block diagram of an audio and video call processing device based on IVVR in another embodiment;

[0039] Fig.14 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0040] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] The IVVR-based audio and video call processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the application environment includes SBC access module, SIP Proxy cluster, IVVR server cluster, multimedia creation server cluster and video page server cluster, etc. Based on the traditional IVVR application environment, the application environment adds SBC access module, multimedia creation server cluster, IVVR server cluster and multimedia WEB cluster.

[0043] Among them, the SBC access module is used to realize SIP docking between users and the VOLTE network of the three major operators (or the operator's video call service platform).

[0044] The multimedia server cluster is used to complete the generation of dynamic video content and connect with the business system to dynamically generate video files from business information and send them to the user side.

[0045] The IVVR server cluster is used to complete the linkage of buttons, voice playback, and video, and connect the multimedia creation server cluster, the multimedia creation server cluster, and the client as three independent access parties to the multimedia conference.

[0046] The web server cluster is responsible for multimedia WEB rendering, collecting the results of customer page keystrokes or voice interactions, calling the semantic engine, displaying business results, and scheduling instructions for IVVR.

[0047] Each server cluster may be implemented as a server cluster consisting of multiple servers.

[0048] In one embodiment, Figure 2 As shown, a method for processing audio and video calls based on IVVR is provided, and the method is applied to Figure 1 Take any server in the IVVR server cluster as an example to illustrate the process, including the following steps:

[0049] Step 202: when a video call request is received, the multimedia creation server is called to obtain a target service menu screen from a video page server, and a first playback instruction for the target service menu screen is received from the video page server.

[0050] IVVR (Interactive Voice and Video Response) is a new wireless voice and video response value-added service. Mobile phone users can get the required information or participate in interactive services by dialing a designated number.

[0051] Among them, the SBC device performs belief analysis on the video call request of the operator's IMS network, transfers the video call request to the network proxy server, and then the network proxy server load balances it to the IVVR server.

[0052] like Figure 3 As shown in the figure, the requester dials the video call request phone of the service platform through the terminal and accesses the operator's IMS network (IP Multimedia Subsystem). The IMS network uses IP technology to build a layered, open, and integrated communication core network on the operator's network. Through the IMS network, users can enjoy rich telecommunications-grade multimedia services such as high-definition audio / video calls, video ringback tone, and telephone conferences.

[0053] Specifically, the operator uses the SBC device of the video call service platform to perform signaling analysis on the video call request of the IMS network, and transfers the video call request to the network proxy server (SIPPROXY) for load balancing the incoming video call requests. The network proxy server load balances the video call request to one of the servers in the IVVR server cluster.

[0054] like Figure 3 As shown, the IVVR server calls the multimedia creation server to request the display of the target service menu screen. In this embodiment, the target service menu is the service platform homepage menu. After receiving the request from the IVVR server, the multimedia creation server calls the video page server to obtain the target service menu screen, and processes it to obtain the video stream of the target service menu screen. The IVVR server also receives the first playback instruction for the target service menu screen indicated by the video page server, and the first playback instruction instructs the prompt tone of the target service menu screen to be played to the call requesting party.

[0055] Step 204, in response to the first playing instruction, the prompt tone of the target service menu screen is played to the call requesting party, and the video of the target service menu screen is sent to the call requesting party to realize a video call with the call requesting party.

[0056] Specifically, the IVVR server responds to the first playback instruction to play the prompt tone of the target service menu screen to the call requester, and also sends the video of the target service menu screen to the call requester to implement a video call with the call requester.

[0057] The video call requester displays the target service menu screen through the terminal and listens to the prompt tone of the target service menu. If the key value of the target service menu that the user wants to jump to is received, the request continues to be made to the IVVR service cluster, thereby realizing video call interaction based on IVVR.

[0058] Step 206: When a voice switching instruction is received during the video call, a prompt tone of the target service menu is played to the call requesting party to implement a voice call with the call requesting party.

[0059] The voice switching command refers to a command to switch a video call to a voice call. The phone application interface of the terminal provides a switching control, such as Figure 4 When the voice switching is triggered, the terminal can send a voice switching instruction to the IVVR server based on the switching button.

[0060] When switching to a voice call, the IVVR server only needs to provide the target service menu prompt tone to the call requester. The caller listens to the target service menu prompt tone through the terminal. If the key value of the target service menu that the user wants to jump to is received, the request is continued to be made to the IVVR service cluster, thereby realizing voice call interaction based on IVVR.

[0061] In the above-mentioned IVVR-based audio and video call processing method, when a video call request is received, the IVVR server calls the multimedia creation server to obtain the target service menu screen from the video page server, and receives the first playback instruction for the target service menu screen indicated by the video page server, and plays the prompt tone of the target service menu screen to the call requesting party according to the first playback instruction, and sends the video stream of the target service menu screen to the call requesting party. When a voice switching instruction is received during the video call, the prompt tone of the target service menu is played to the call requesting party to realize a voice call with the call requesting party. This method realizes audio and video switching based on IVVR, and the switching before and after is undertaken by the same service subject, so the continuity of the call service before and after the switching can be guaranteed.

[0062] In another embodiment, when a video call request is received, a multimedia creation server is called to obtain a target service menu screen from a video page server, and a first playback instruction for the target service menu screen indicated by the video page server is received, including: when a video call request is received, a first multimedia conference is created with an IVVR server, a multimedia creation server and a call requesting party as access parties; based on the first multimedia conference, a multimedia creation server is called to obtain a target service menu screen from a video page server, and a first playback instruction for the target service menu screen indicated by the video page server is received.

[0063] Specifically, the IVVR cluster is used to implement IVVR access and framework control services. After the IVVR server receives the video call request, the IVVR server initializes the media resources and connects the IVVR server, the multimedia creation server, and the call requester as three independent access parties to the multimedia conference. In this multimedia conference, different conference access parties have different processing capabilities. The role of the call requester is to receive and send instructions, the role of the multimedia creation server is to provide multimedia video services, etc., and the IVVR server serves as the host of the conference.

[0064] Each access party of the multimedia conference performs processing according to its processing capability. Figure 3 As shown, the IVVR server calls the multimedia creation server to request the display of the target service menu screen. In this embodiment, the target service menu is the service platform homepage menu. After receiving the request from the IVVR server, the multimedia creation server calls the video page server to obtain the target service menu screen, and processes it to obtain the video stream of the target service menu screen. The IVVR server also receives the first playback instruction for the target service menu screen indicated by the video page server, and the first playback instruction instructs the prompt tone of the target service menu screen to be played to the call requesting party.

[0065] In this embodiment, when a video call request is received, a first multimedia conference is created based on the IVVR server, the multimedia creation server and the call requester, the multimedia creation server is called based on the first multimedia conference to obtain the target business menu screen from the video page server, and the first playback instruction for the target business menu screen indicated by the video page server is received, the prompt sound of the target business menu screen is played to the call requester according to the first playback instruction, and the video stream of the target business menu screen is sent to the call requester.

[0066] In this embodiment, a first multimedia conference may be created with the IVVR server, the multimedia creation server, and the video call requester as access parties, and a video call may be implemented based on the first multimedia conference.

[0067] In another embodiment, when a voice switching instruction is received during a video call, a prompt tone of a target service menu is played to the call requesting party to realize a voice call with the call requesting party, including: when a voice switching instruction is received during a video call, a multimedia creation server is moved out of a first multimedia conference to obtain a second multimedia conference; based on the second multimedia conference, a prompt tone of a target service menu is played to the call requesting party to realize a voice call with the call requesting party.

[0068] The advantage of multimedia conferences is that any party can join or leave the conference at will. For example, when it is applied to the switching of audio and video calls of IVVR, if it is a video call, a first multimedia conference is created based on the IVVR server, the multimedia creation server and the call requesting party. If it is switched to a voice call during the video call, the multimedia creation server is removed from the first multimedia conference to obtain a second multimedia conference.

[0069] When switching to a voice call, the participants of the second multimedia conference only include the IVVR server and the call requester. The IVVR server only needs to provide the call requester with the prompt tone of the target service menu. The caller listens to the prompt tone of the target service menu through the terminal. If the key value of the target service menu that the user wants to jump to is received, the request is continued to be made to the IVVR service cluster, thereby realizing voice call interaction based on IVVR.

[0070] In this embodiment, when a voice switching instruction is received during a video call, the multimedia creation server is moved out of the first multimedia conference to obtain a second multimedia conference, and a prompt tone of the target service menu is played to the call requester based on the second multimedia conference to realize a voice call with the call requester. Since a first multimedia conference with an IVVR server, a multimedia creation server, and a video call requester as access parties can be created, a video call is realized based on the first multimedia conference, and when a voice switching instruction is received during a video call, the multimedia creation server is moved out of the first multimedia conference to obtain a second multimedia conference, and a voice call is realized based on the second multimedia conference, the method realizes audio and video switching based on IVVR, and the switching before and after is undertaken by the same service subject, so the continuity of the call service before and after the switching can be guaranteed.

[0071] Furthermore, in order to achieve service continuity when the voice call is switched to a video call, in this embodiment, when the voice is passed, a request is maintained to the multimedia creation server, the multimedia creation server requests the target service menu screen from the video page server, and receives a second playback instruction for the target service menu screen indicated by the video page server; in response to the second playback instruction, a prompt tone of the target service menu screen is played to the call requesting party to achieve a voice call with the call requesting party.

[0072] Specifically, when a voice call is in progress, the multimedia creation server is removed from the second multimedia conference, but the IVVR server keeps requesting the multimedia creation server, and the multimedia creation server still requests the target service menu screen from the video page server. Unlike the first multimedia conference of the voice call, the multimedia creation server does not return the video stream of the target service menu screen to the IVVR server. At the same time, the video page server instructs the IVVR server to issue a second playback instruction for the target service menu screen, and the IVVR server responds to the second playback instruction and plays the prompt tone of the target service menu screen to the call requester.

[0073] During a voice call, the IVVR server keeps requesting the multimedia creation server, and the multimedia creation server still requests the target service menu screen from the video page server. However, the multimedia creation server does not return the video stream of the target service menu screen to the IVVR server. Therefore, the video stream processing capability can be maintained during the voice call. When it is necessary to switch from a voice call to a video call, the video stream of the target service menu screen can be quickly requested from the multimedia creation server based on this capability, thereby ensuring the continuity of the service during the switching from the video call to the voice call.

[0074] Specifically, based on the second multimedia conference, keep sending a second video display request to the multimedia creation server, the second video display request instructs the multimedia creation server to call the video page server, and requests the target business menu screen from the video page server through the web page server; receive the second playback instruction sent by the video page server based on the target business menu screen, the second playback instruction is used to instruct the prompt sound of the target business menu screen to be played.

[0075] Specifically, a multimedia creation server and a web server are utilized. The multimedia creation server is responsible for converting the IVVR video page content into a video stream, and is also responsible for simulating customer behavior and realizing interaction with the web server. When switching to a voice call, the IVVR server's request to the multimedia creation server and the interaction between the multimedia creation server and the web server are maintained. The web server is responsible for providing specific video pages, and performing page jumps and business logic processing based on user interactions, thereby maintaining the ability to process video streams during a voice call. When it is necessary to switch from a voice call to a video call, the video stream of the target business menu screen can be quickly requested from the multimedia creation server based on this capability, thereby implementing business continuity during the process of switching from a video call to a voice call.

[0076] Wherein, based on the first multimedia conference, the multimedia creation server is called to obtain the target service menu screen from the video page server, and the first playback instruction of the target service menu screen indicated by the video page server is received, such as Figure 5 As shown, including:

[0077] Step 502: Based on the first multimedia conference, a first video display request is sent to the multimedia creation server. The first video display request instructs the multimedia creation server to call the web page server and the video page server to return the target service menu screen.

[0078] like Figure 6 As shown, the IVVR server sends a first video display request to the multimedia creation server. After receiving the first video display request, the multimedia creation server calls the web server to start a simulated browser and requests the target service menu screen from the IVVR video page service through the browser. If it is a video display request for a call request, the target service menu is the homepage menu. If it is a target menu specified for jumping during a call, the target service menu is the specified target menu to jump to.

[0079] Step 504: Receive a video stream of a target service menu screen returned by the multimedia creation server according to the first video display request. The video stream of the target service menu screen is obtained by processing the returned target service menu screen by the multimedia creation server.

[0080] The video page server returns the target service menu screen to the multimedia creation server. The screen is a static page content. The multimedia creation server generates a silent video of the target service menu screen according to the static screen of the target service menu and feeds it back to the IVVR server.

[0081] Step 506: Receive a first playback instruction sent by the video page server based on the target service menu screen, where the first playback instruction is used to instruct to play the prompt tone of the target service menu screen.

[0082] While processing the video, the video page server also sends a first playback instruction to the IVVR server, where the first playback instruction includes information of the target service menu.

[0083] After receiving the playback command sent by the video page server, the IVVR server starts the playback process and plays the prompt tone of the target service menu, wherein the prompt tone of the target service menu specifically includes the service content of each button of the target service menu.

[0084] In this embodiment, the interaction between the multimedia creation server, the video page server and the web page server is utilized to provide the video of the service menu screen and perform video interaction operations via HTTP.

[0085] The user terminal receives the video stream of the service menu and may need to perform voice interaction. Specifically, the audio and video call processing method based on IVVR, such as Figure 7 and Figure 8As shown, including:

[0086] Step 702: Based on the first multimedia conference, a first video display request is sent to the multimedia creation server. The first video display request instructs the multimedia creation server to call the web page server and the video page server to return the target service menu screen.

[0087] like Figure 8 As shown, the IVVR server sends a first video display request to the multimedia creation server. After receiving the first video display request, the multimedia creation server calls the web server to start a simulated browser and requests the target service menu screen from the IVVR video page service through the browser. If it is a video display request for a call request, the target service menu is the homepage menu. If it is a target menu specified for jumping during a call, the target service menu is the specified target menu to jump to.

[0088] Step 704: Receive a video stream of a target service menu screen returned by the multimedia creation server according to the first video display request. The video stream of the target service menu screen is obtained by processing the returned target service menu screen by the multimedia creation server.

[0089] The video page server returns the target service menu screen to the multimedia creation server. The screen is a static page content. The multimedia creation server generates a silent video of the target service menu screen according to the static screen of the target service menu and feeds it back to the IVVR server.

[0090] Step 706: Receive a first playback instruction sent by the video page server based on the target service menu screen, where the first playback instruction is used to instruct to play the prompt tone of the target service menu screen.

[0091] While processing the video, the video page server also sends a first playback instruction to the IVVR server, where the first playback instruction includes information of the target service menu.

[0092] After receiving the playback command sent by the video page server, the IVVR server starts the playback process and plays the prompt tone of the target service menu, wherein the prompt tone of the target service menu specifically includes the service content of each button of the target service menu.

[0093] Step 708: Receive a voice instruction from the call requester, where the voice instruction includes a target service menu to be jumped to.

[0094] Specifically, the user can view the menu function that needs to be jumped while watching the video, such as Fig. 9 As shown in the figure, users can send a voice command "balance inquiry" through the video call.

[0095] Step 710, obtaining the key value corresponding to the voice command.

[0096] Specifically, the IVVR server calls the voice recognition and semantic understanding model of the contact center platform to parse the key value corresponding to the voice command, such as the key value corresponding to "balance inquiry" is 1.

[0097] Step 712, the key value is transparently transmitted to the video page server through the multimedia creation server, and the IVVR video page calls the web page server to obtain the target business menu screen to be jumped and returns it to the multimedia creation server.

[0098] Specifically, the IVVR server sends the corresponding key value to the multimedia creation server through an http request, and the multimedia creation server transmits the key value to the video page server through a simulated browser. The video page server detects the key value and realizes the page jump.

[0099] Specifically, the video page server returns the target service menu screen to the multimedia creation server. The screen is a static page content. The multimedia creation server generates a silent video of the target service menu screen according to the static screen of the target service menu and feeds it back to the IVVR server.

[0100] After step 712, the process returns to step 704 to implement video call interaction. For example, the process can receive voice commands from the user, such as "return to main menu", "reset password", etc., respond to the user's commands, jump to different target service menus, and return the video stream of the corresponding target service menu screen to the user terminal.

[0101] In this embodiment, the jump of the video screen is achieved by responding to the voice signal during the call.

[0102] In one embodiment, a method for processing audio and video calls based on IVVR is also provided, by which a voice call can be switched to a video call. Fig. 9 As shown, the method includes:

[0103] Step 902, when a voice call instruction is received, keep requesting the multimedia creation server, the multimedia creation server requests the target service menu screen from the video page server, and receives the second playback instruction for the target service menu screen indicated by the video page server.

[0104] Specifically, Fig.10As shown in the figure, the requester dials the video call request phone of the service platform through the terminal and accesses the operator's IMS network (IP Multimedia Subsystem). The IMS network uses IP technology to build a layered, open, and integrated communication core network on the operator's network. Through the IMS network, users can enjoy rich telecommunications-grade multimedia services such as high-definition audio / video calls, video ringback tone, and telephone conferences.

[0105] Specifically, the operator uses the SBC device of the video call service platform to perform signaling analysis on the voice call request of the IMS network, and transfers the video call request to the network proxy server (SIPPROXY) for load balancing the incoming video call requests. The network proxy server load balances the video call request to one of the servers in the IVVR server cluster.

[0106] The IVVR cluster is used to implement IVVR access and framework control services. After the IVVR server receives the video call request, the IVVR server initializes the media resources and connects the IVVR server, multimedia creation server, and call requester as three independent access parties to the multimedia conference. In this multimedia conference, different conference access parties have different processing capabilities. The role of the call requester is to receive and send instructions, the role of the multimedia creation server is to provide multimedia video services, etc., and the IVVR server serves as the host of the conference.

[0107] In this embodiment, for a voice call request, a second multimedia conference is created with the IVVR server and the call requester as access parties.

[0108] During the voice call, the multimedia creation server is removed from the second multimedia conference, but the IVVR server keeps requesting the multimedia creation server, and the multimedia creation server still requests the target service menu screen from the video page server. Unlike the first multimedia conference of the voice call, the multimedia creation server does not return the video stream of the target service menu screen to the IVVR server. At the same time, the video page server instructs the IVVR server to issue a second playback instruction for the target service menu screen, and the IVVR server responds to the second playback instruction and plays the prompt tone of the target service menu screen to the call requester.

[0109] Step 904: In response to the second playing instruction, a prompt tone of the target service menu screen is played to the call requesting party to implement a voice call with the call requesting party.

[0110] Specifically, Fig.10 As shown, the IVVR server responds to the second playback instruction of the video page server and plays the prompt tone of the target service menu screen to the call requester.

[0111] The user listens to the prompt tone of the target service menu screen through the terminal, and then performs voice interaction.

[0112] Step 906, when a video switching instruction is received during a voice call, the multimedia creation server is called to request the target service menu screen from the video page server, the prompt sound corresponding to the target service menu screen is played, and the video of the target service menu screen is sent to the call requesting party for a video call.

[0113] When a user needs to switch to a video call during a voice call, the user can trigger a video switching instruction through the phone application of the terminal. The video switching instruction is specifically an instruction to switch the video call to a voice call. Figure 4 As shown, a video switching instruction is sent by triggering the video switching button of the phone application.

[0114] When the IVVR server receives the video switching instruction, it can add the multimedia creation server to the second multimedia conference to obtain the first multimedia conference.

[0115] Each access party of the multimedia conference performs processing according to its processing capability. Fig.10 As shown, the IVVR server calls the multimedia creation server to request the display of the target service menu screen. In this embodiment, the target service menu is the service platform homepage menu. After receiving the request from the IVVR server, the multimedia creation server calls the video page server to obtain the target service menu screen, and processes it to obtain the video stream of the target service menu screen. The IVVR server also receives the first playback instruction for the target service menu screen indicated by the video page server, and the first playback instruction instructs the prompt tone of the target service menu screen to be played to the call requesting party.

[0116] The above-mentioned IVVR-based audio and video call processing method maintains the request to the multimedia creation server when a voice call request is received, and plays the prompt tone of the target service menu screen to the call requester to realize the voice call with the call requester. When a video switching instruction is received during the voice call, the multimedia creation server is called to request the target service menu screen from the video page server, play the prompt tone corresponding to the target service menu screen, and send the video of the target service menu screen to the call requester for the video call. Because during the voice call, the IVVR server maintains the request to the multimedia creation server, and the multimedia creation server still requests the target service menu screen from the video page server, but the multimedia creation server does not return the video stream of the target service menu screen to the IVVR server, so the video stream processing capability can still be maintained during the voice call. When it is necessary to switch from a voice call to a video call, it can quickly request the video stream of the target service menu screen from the multimedia creation server based on this capability, and implement the continuity of the service during the switching of the video call to the voice call.

[0117] Among them, the detailed process of implementing audio calls and implementing video calls has been recorded in the previous embodiments and will not be repeated here.

[0118] like Fig.11 As shown in the figure, it is a schematic diagram of the implementation of the audio and video call processing method based on IVVR. On the basis of the traditional IVVR application environment, this application environment has added an SBC access module, a multimedia creation server cluster, an IVVR server cluster and a multimedia WEB cluster. By upgrading the existing contact center platform, the SBC device is introduced to realize the SIP docking of multimedia dialogue with the operator's IMS network, and the customer's video call from the operator's IMS network is taken over and distributed to the IVVR service cluster through the contact center platform SIP Proxy. A conference call between the IVVR server, the multimedia creation server and the call requesting party is established to realize the visual IVVR service. Among them, the IVVR server accepts the audio and video switching instructions from the IMS network customer, and the multimedia creation server is connected or exited according to the instructions to realize the switching back and forth of audio and video calls.

[0119] Specifically, the requester dials the video or voice call request phone of the service platform through the terminal and accesses the operator's IMS network (IP Multimedia Subsystem). The operator performs signaling analysis on the video call request of the IMS network through the SBC device of the video call service platform, and transfers the video call request to the network proxy server (SIPPROXY) for load balancing the incoming video call requests. The network proxy server load balances the video call request to one of the servers in the IVVR server cluster.

[0120] When a video call request is received, a first multimedia conference is created with the IVVR server, the multimedia creation server and the call requester as access parties. Based on the first multimedia conference, a first video display request is sent to the multimedia creation server, the first video display request instructs the multimedia creation server to call the web server and the video page server to return the target business menu screen; the video stream of the target business menu screen returned by the multimedia creation server according to the first video display request is received, and the video stream of the target business menu screen is obtained by the multimedia creation server processing the returned target business menu screen; the first playback instruction sent by the video page server based on the target business menu screen is received, the first playback instruction is used to instruct the playback of the prompt tone of the target business menu screen, the prompt tone of the target business menu screen is played to the call requester in response to the first playback instruction, and the video of the target business menu screen is sent to the call requester to realize a video call with the call requester.

[0121] When a voice switching instruction is received during a video call, the multimedia creation server is removed from the first multimedia conference to obtain a second multimedia conference. Based on the second multimedia conference, a second video display request is continuously sent to the multimedia creation server. The second video display request instructs the multimedia creation server to call the video page server, and requests the target service menu screen from the video page server through the web page server. The second playback instruction sent by the video page server based on the target service menu screen is received. The second playback instruction is used to instruct to play the prompt tone of the target service menu screen. In response to the second playback instruction, the prompt tone of the target service menu screen is played to the call requester to realize a voice call with the call requester.

[0122] This method uses IVVR server, multimedia creation server and video page server to realize IVVR video call by linking the customer's call party, video display and IVR pure voice through conference access. When the customer chooses to switch audio and video during the call, the multimedia creation server is removed from the conference to achieve seamless switching of audio and video and maintain business continuity.

[0123] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0124] Based on the same inventive concept, the embodiment of the present application also provides an IVVR-based audio and video call processing device for implementing the IVVR-based audio and video call processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more IVVR-based audio and video call processing device embodiments provided below can refer to the limitations of the IVVR-based audio and video call processing method above, and will not be repeated here.

[0125] In one embodiment, Fig.12 As shown, an audio and video call processing device based on IVVR is provided, comprising:

[0126] The video processing module 1202 calls the multimedia creation server to obtain the target service menu screen from the video page server, and receives the first playback instruction of the target service menu screen indicated by the video page server.

[0127] The voice processing module 1204 is used to respond to the first voice playing instruction to play the prompt sound of the target service menu screen to the call requesting party, and send the video of the target service menu screen to the call requesting party to realize a video call with the call requesting party.

[0128] The switching module 1206 is used to play a prompt tone of the target service menu to the call requesting party to realize a voice call with the call requesting party when a voice switching instruction is received during the video call.

[0129] Among them, the video processing module is used to create a first multimedia conference with the IVVR server, the multimedia creation server and the call requester as access parties when a video call request is received; based on the first multimedia conference, call the multimedia creation server to obtain the target business menu screen from the video page server, and receive the first playback instruction for the target business menu screen indicated by the video page server.

[0130] Among them, the switching module is used to, when a voice switching instruction is received during the video call, move the multimedia creation server out of the first multimedia conference to obtain a second multimedia conference; based on the second multimedia conference, play the prompt tone of the target service menu to the call requesting party to realize a voice call with the call requesting party.

[0131] Among them, the voice processing module is used to maintain a request to the multimedia creation server, the multimedia creation server requests the target business menu screen from the video page server, and receives a second playback instruction for the target business menu screen indicated by the video page server; in response to the second playback instruction, the prompt sound of the target business menu screen is played to the call requesting party to realize a voice call with the call requesting party.

[0132] Among them, the video processing module is used to send a first video display request to the multimedia creation server, the first video display request instructs the multimedia creation server to call the web server and the video page server to return the target business menu screen; receive the video stream of the target business menu screen returned by the multimedia creation server according to the first video display request, the video stream of the target business menu screen is obtained by the multimedia creation server processing the returned target business menu screen; receive the first playback instruction sent by the video page server based on the target business menu screen.

[0133] Wherein, the device further comprises:

[0134] A voice analysis module, used to receive a voice instruction from the call requester, wherein the voice instruction includes a target service menu to be jumped to, and obtain a key value corresponding to the voice instruction;

[0135] The video processing module is also used to transmit the key value to the video page server through the multimedia creation server, and the IVVR video page calls the web page server to obtain the target business menu screen to be jumped and returns it to the multimedia creation server.

[0136] Among them, the video processing module is also used to keep sending a second video display request to the multimedia creation server, the second video display request instructs the multimedia creation server to call the video page server, request the target business menu screen from the video page server through the web page server, and receive the second playback instruction sent by the video page server based on the target business menu screen.

[0137] like Fig.13 As shown, an IVVR-based audio and video call processing device is also provided, the device comprising:

[0138] The voice processing module 1302 is used to keep requesting the multimedia creation server when receiving the voice call instruction, and the multimedia creation server requests the target service menu screen from the video page server, and receives the second playback instruction for the target service menu screen indicated by the video page server.

[0139] The sending module 1304 is configured to respond to the second voice playing instruction and play the prompt tone of the target service menu screen to the call requesting party to implement a voice call with the call requesting party.

[0140] The switching module 1306 is used to call the multimedia creation server to request the target business menu screen from the video page server when a video switching instruction is received during a voice call, play the prompt sound corresponding to the target business menu screen, and send the video of the target business menu screen to the call requester for a video call.

[0141] Each module in the above-mentioned IVVR-based audio and video call processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0142] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.14As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store voice and picture data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for processing audio and video calls based on IVVR is implemented.

[0143] Those skilled in the art will understand that Fig.14 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0144] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the IVVR-based audio and video call processing method of the above-mentioned embodiments when executing the computer program.

[0145] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the IVVR-based audio and video call processing method of the above-mentioned embodiments are implemented.

[0146] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the IVVR-based audio and video call processing method of the above-mentioned embodiments.

[0147] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0148] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0149] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An IVVR-based audio and video call processing method, applied to an IVVR server, characterized in that: The method comprises: When a video call request is received, a first multimedia conference is created with the IVVR server, the multimedia creation server and the call requesting party as access parties; Based on the first multimedia conference, calling the multimedia creation server to obtain a target service menu screen from a video page server, and receiving a first playback instruction for the target service menu screen indicated by the video page server; the target service menu is a service platform homepage menu; In response to the first playing instruction, playing the prompt tone of the target service menu screen to the call requesting party, and sending the video of the target service menu screen to the call requesting party to realize a video call with the call requesting party; When a voice switching instruction is received during the video call, a prompt tone of a target service menu screen is played to the call requesting party to implement a voice call with the call requesting party; When a voice switching instruction is received during the video call, playing a prompt tone of a target service menu to the call requesting party to implement a voice call with the call requesting party includes: When a voice switching instruction is received during the video call, the multimedia creation server is removed from the first multimedia conference to obtain a second multimedia conference; based on the second multimedia conference, a prompt tone of a target service menu screen is played to the call requesting party to realize a voice call with the call requesting party; Alternatively, the request to the multimedia creation server is maintained, the multimedia creation server requests the target business menu screen from the video page server, and receives a second playback instruction for the target business menu screen indicated by the video page server, and the multimedia creation server does not return the video stream of the target business menu screen to the IVVR server; in response to the second playback instruction, the prompt tone of the target business menu screen is played to the call requester to realize a voice call with the call requester.

2. The method according to claim 1, characterized in that: The calling of the multimedia creation server to obtain a target service menu screen from a video page server, and receiving a first playback instruction for the target service menu screen indicated by the video page server, comprises: Sending a first video display request to the multimedia creation server, wherein the first video display request instructs the multimedia creation server to call a web page server and a video page server to return a target service menu screen; Receiving a video stream of a target service menu screen returned by the multimedia creation server according to the first video display request, wherein the video stream of the target service menu screen is obtained by the multimedia creation server processing the returned target service menu screen; A first playback instruction is received from the video page server based on the target service menu screen.

3. The method according to claim 2, characterized in that After the step of playing a prompt tone of the target service menu to the call requesting party to realize a voice call with the call requesting party, the method further includes: receiving a voice instruction from the call requester, wherein the voice instruction includes a target service menu to be jumped to; Obtaining the key value corresponding to the voice command; The key value is transparently transmitted to the video page server through the multimedia creation server, and the IVVR video page calls the web page server to obtain the target business menu screen to be jumped and returns it to the multimedia creation server; Return to the step of receiving the video stream of the target service menu screen returned by the multimedia creation server according to the first video display request.

4. The method according to claim 1, characterized in that The step of maintaining the request to the multimedia creation server, requesting the target service menu screen from the video page server by the multimedia creation server, and receiving the second playback instruction for the target service menu screen indicated by the video page server includes: Keep sending a second video display request to the multimedia creation server, wherein the second video display request instructs the multimedia creation server to call the video page server and request a target service menu screen from the video page server through the web page server; A second playback instruction is received from the video page server based on the target service menu screen.

5. An IVVR-based audio and video call processing method, applied to an IVVR server, characterized in that: The method comprises: When receiving a voice call instruction, a second multimedia conference is created with the IVVR server and the call requester as access parties, and the request is kept to the multimedia creation server, and the multimedia creation server requests the target service menu screen from the video page server, and receives a second playback instruction for the target service menu screen indicated by the video page server, and the multimedia creation server does not return the video stream of the target service menu screen to the IVVR server; In response to the second playback instruction, playing the prompt tone of the target service menu screen to the call requesting party to realize a voice call with the call requesting party; or, based on the second multimedia conference, playing the prompt tone of the target service menu screen to the call requesting party to realize a voice call with the call requesting party; When a video switching instruction is received during a voice call, the multimedia creation server is added to the second multimedia conference to obtain the first multimedia conference; based on the first multimedia conference, the multimedia creation server is called to request the target service menu screen from the video page server, and the first playback instruction for the target service menu screen indicated by the video page server is accepted, and the prompt sound corresponding to the target service menu screen is played to the call requesting party in response to the first playback instruction, and the video of the target service menu screen is sent to the call requesting party to realize a video call with the call requesting party; the target service menu is the service platform homepage menu.

6. An audio and video call processing device based on IVVR, characterized in that: The device comprises: The video processing module is used to create a first multimedia conference with the IVVR server, the multimedia creation server and the call requester as access parties when receiving a video call request; based on the first multimedia conference, call the multimedia creation server to obtain a target service menu screen from the video page server, and receive a first playback instruction for the target service menu screen indicated by the video page server; the target service menu is a service platform homepage menu; A voice processing module, configured to respond to the first voice playing instruction to play a prompt tone of the target service menu screen to the call requesting party, and send a video of the target service menu screen to the call requesting party to realize a video call with the call requesting party; A switching module is used to play the prompt sound of the target service menu screen to the call requester to realize the voice call with the call requester when a voice switching instruction is received during the video call; when a voice switching instruction is received during the video call, the multimedia creation server is moved out of the first multimedia conference to obtain a second multimedia conference; based on the second multimedia conference, the prompt sound of the target service menu screen is played to the call requester to realize the voice call with the call requester; or, the voice processing module is also used to maintain the request to the multimedia creation server, the multimedia creation server requests the target service menu screen from the video page server, and receives the second playback instruction for the target service menu screen indicated by the video page server; in response to the second playback instruction, the prompt sound of the target service menu screen is played to the call requester to realize the voice call with the call requester, and the multimedia creation server does not return the video stream of the target service menu screen to the IVVR server.

7. The device according to claim 6, characterized in that The video processing module is used to send a first video display request to the multimedia creation server, wherein the first video display request instructs the multimedia creation server to call a web page server and a video page server to return a target service menu screen; Receive a video stream of a target business menu screen returned by the multimedia creation server according to the first video display request, wherein the video stream of the target business menu screen is obtained by the multimedia creation server processing the returned target business menu screen; and receive a first playback instruction sent by the video page server based on the target business menu screen.

8. The device according to claim 7, characterized in that The device also includes: A voice analysis module, used to receive a voice instruction from the call requester, wherein the voice instruction includes a target service menu to be jumped to, and obtain a key value corresponding to the voice instruction; The video processing module is also used to transmit the key value to the video page server through the multimedia creation server, and the IVVR video page calls the web page server to obtain the target business menu screen to be jumped and returns it to the multimedia creation server.

9. The device according to claim 6, characterized in that The video processing module is also used to keep sending a second video display request to the multimedia creation server, wherein the second video display request instructs the multimedia creation server to call the video page server, request a target business menu screen from the video page server through the web page server, and receive a second playback instruction sent by the video page server based on the target business menu screen.

10. An audio and video call processing device based on IVVR, characterized in that: The device comprises: A voice processing module, configured to, when receiving a voice call instruction, create a second multimedia conference with the IVVR server and the call requester as access parties, maintain a request to the multimedia creation server, the multimedia creation server requests the target service menu screen from the video page server, and receive a second playback instruction for the target service menu screen indicated by the video page server, and the multimedia creation server does not return a video stream of the target service menu screen to the IVVR server; a sending module, configured to respond to the second playback instruction and play the prompt tone of the target service menu screen to the call requesting party to realize a voice call with the call requesting party; or, based on the second multimedia conference, play the prompt tone of the target service menu screen to the call requesting party to realize a voice call with the call requesting party; The switching module is used to, when receiving a video switching instruction during a voice call, add the multimedia creation server to the second multimedia conference to obtain the first multimedia conference; based on the first multimedia conference, call the multimedia creation server to request the target service menu screen from the video page server, and accept the first playback instruction for the target service menu screen indicated by the video page server, respond to the first playback instruction to play the prompt sound corresponding to the target service menu screen to the call requesting party, and send the video of the target service menu screen to the call requesting party to realize a video call with the call requesting party; the target service menu is the service platform homepage menu.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images