Interaction method for call interface, and related apparatus

Abstract

An interaction method for a call interface. The method is applied to an electronic device, on which an AI assistant is mounted and which has a display screen. The method comprises: detecting that an AI call mode has been enabled, wherein the AI call mode is a mode in which an AI assistant is used to make a call with a calling object; and detecting a first operation, wherein the first operation is used for instructing switching of the AI call mode to a human call mode, and the human call mode is a mode in which a user makes a call to the calling object. The method supports switching between a user call mode and an AI call mode, and supports interaction with an AI assistant via an interface in the AI call mode, so as to better guide the AI assistant to make a call to a calling object, thereby improving the user usage experience and working efficiency.

Description

Interaction method and related device of call interface Technical Field

[0001] The present application relates to the technical field of terminal equipment, and in particular to an interactive method and related device for a call interface. Background Art

[0002] With the development of society and the revolution in manufacturing technology, the world is accelerating into a digital, intelligent society where everything senses, everything is connected, and everything is intelligent. The digitization of electronic devices has gradually become mainstream. In this digital age, people have higher expectations for efficient work and life. For example, in multitasking scenarios such as holding an online meeting while taking an important call, users can use AI assistants to help answer the call and record important information during the call to improve efficiency.

[0003] At present, using AI assistants to answer calls has become a mainstream scenario. However, due to the complexity of call scenarios and the diversity of call types, AI assistants find it difficult to handle complex and changeable call scenarios well. Therefore, a call method that supports user interaction with AI assistants is essential.

[0004] Summary of the Invention

[0005] This application provides a method for call interface interaction, which can help users answer calls and generate key information summary cards, thereby improving user experience and work efficiency.

[0006] The first aspect of the present application provides an interactive method for a call interface. The method is applied to an electronic device with an AI assistant application, and the electronic device is not limited to a smart terminal electronic device such as a mobile phone, a computer, a tablet, or a watch. The method comprises: detecting that an AI call mode is enabled, wherein the AI ​​call mode refers to a mode in which an AI assistant is used to make a call with an incoming caller; and detecting a first operation, wherein the first operation is used to indicate that the AI ​​call mode is switched to a manual call mode, wherein the manual call mode refers to a mode in which a user makes a call with an incoming caller.

[0007] The method helps users receive some key call information by using an AI assistant to answer calls, realizes interaction during the call process through the interaction between the AI ​​call mode and the manual mode, reduces the AI ​​assistant's incorrect response to the call content, and solves the problem that the AI ​​assistant has difficulty in dealing with complex call scenarios, thereby improving user experience and work efficiency.

[0008] In a possible implementation, the method further includes: detecting input of a prompt word, generating a call voice signal based on the prompt word, and sending the call voice signal to the caller.

[0009] In this solution, users can set the AI ​​assistant's response content and method based on the prompt words that pop up on the interface. This allows the AI ​​assistant to adjust its status and content in a timely manner according to the needs of the actual scenario, thereby providing a better user experience.

[0010] In a possible implementation, the method further includes: displaying an information card and a summary card on the call interface, wherein the information card is used to indicate key content text in the call information, and the summary card is used to indicate summary content text in the call information.

[0011] In this solution, the call interface will display information cards and summary cards to record the content of the call, thereby improving the user experience and work efficiency.

[0012] In one possible implementation, the method further includes: displaying an AI assistant icon on the call interface, wherein the AI ​​assistant icon is used to interact with the user to implement mode switching.

[0013] In this solution, an interface interaction scenario is proposed, in which users switch call modes by operating the AI ​​assistant icon.

[0014] In one possible implementation, the prompt words specifically include fixed recommendation words, and the fixed recommendation words include tone recommendation words and control recommendation words; the tone recommendation words are used to indicate the tone of the output voice text; the control recommendation words are used to output voice text to control the progress and duration of the call.

[0015] In this solution, prompt words include tone-related recommended words and control-related recommended words. Tone-related recommended words can help the AI ​​assistant adapt to the current communication context by adjusting the tone by selecting the tone type, including cute tone, lively tone, serious tone, gentle tone, angry tone, etc., and better deal with expressing user emotions in complex scenarios; control-related recommended words are mainly used to control and remind call duration through call language selection.

[0016] In a possible implementation, the prompt word specifically includes a real-time recommendation word, which is text data generated according to real-time call content and is used to reply to the caller.

[0017] This solution proposes real-time word recommendations, which are primarily generated based on the call content and provide key information prompts for the caller. For example, if the caller asks which restaurant to go to, Restaurant A or Restaurant B, the call interface may present two options, "Restaurant A" and "Restaurant B." Or, if the caller requests a time to book a meeting room, the call interface will present options based on the user's selection. This approach effectively allows users to participate in AI call interaction, helping the AI ​​assistant better express the user's thoughts and wishes, thereby effectively improving the user experience and work efficiency.

[0018] In a possible implementation, the prompt word specifically includes user input text, where the user input text is text data manually input by the user and used to express the user's wishes and ideas.

[0019] In this solution, a method is proposed that supports users to manually input text and express their ideas in real time to interact with the caller. This can help AI assistants better deal with complex scenarios, better express the user's wishes, and in some scenarios can be more tactful and humble, which is more conducive to maintaining interpersonal relationships.

[0020] In one possible implementation, the prompt word specifically includes user input voice, and the method includes: detecting a second operation, where the second operation is used to indicate receiving the user input voice; converting the user input voice into an AI assistant voice; and sending the AI ​​assistant voice to the caller.

[0021] In this solution, an AI assistant is proposed to answer calls, and humans help the AI ​​assistant better respond to the caller's questions by inputting voice.

[0022] In one possible implementation, the method further includes: upon detecting a third operation, adjusting the conversation speed of the AI ​​assistant.

[0023] In this solution, a method for adjusting the speech speed of an AI assistant is proposed. The method can improve the user experience of using the AI ​​assistant, allowing the AI ​​assistant to better simulate human behavior and language rhythm.

[0024] In a possible implementation, the call interface displays an information card and a summary card, wherein the information card is used to indicate key content text in the call information, and the summary card is used to indicate summary content text in the call information.

[0025] In one possible implementation, the call interface displays information cards and summary cards specifically including: detecting an audio signal, identifying information of a preset type in the audio signal to obtain key content text and summary text; generating an information card based on the key content text; and generating a summary card based on the summary text.

[0026] In this solution, users can identify key information, preset information, and other information based on the audio information of the caller. Based on this information, they can extract important summary information and generate summary cards; and they can summarize the information in the call content to generate summary text for recording and reminding users.

[0027] In a possible implementation, the preset types in the audio signal include but are not limited to telephone numbers, addresses, and schedules.

[0028] In a possible implementation, the summary text is text content that summarizes the call content within a preset duration.

[0029] In this solution, the AI ​​assistant can generate content summary text based on the large language model according to the audio call content within a preset duration.

[0030] A second aspect of the present application provides an electronic device for an AI assistant, the device comprising: a detection module for detecting whether an AI call mode is enabled, wherein the AI ​​call mode refers to a mode in which an AI assistant is used to talk to a callee; the detection module is also used to detect a first operation, wherein the first operation is used to indicate that the AI ​​call mode is switched to a manual call mode, wherein the manual call mode refers to a mode in which a user talks to a callee.

[0031] In a possible implementation, the device further includes: an input module, configured to input a prompt word, generate a call voice signal based on the prompt word, and send the call voice signal to the callee.

[0032] In a possible implementation, the device further includes: a display module, configured to display information cards and summary cards, wherein the information cards are configured to indicate key content text in the call information, and the summary cards are configured to indicate summary content text in the call information.

[0033] In one possible implementation, the display module is further used to display an AI assistant icon, and the AI ​​assistant icon is used to interact with the user to implement mode switching.

[0034] In one possible implementation, the prompt words specifically include fixed recommendation words, and the fixed recommendation words include tone recommendation words and control recommendation words; the tone recommendation words are used to indicate the tone of the output voice text; the control recommendation words are used to output voice text to control the progress and duration of the call.

[0035] In a possible implementation, the prompt word specifically includes a real-time recommendation word, which is text data generated according to real-time call content and is used to reply to the caller.

[0036] In a possible implementation, the prompt word specifically includes user input text, where the user input text is text data manually input by the user and used to express the user's wishes and ideas.

[0037] In one possible implementation, the prompt word specifically includes user input voice, and the method includes: detecting a second operation, where the second operation is used to indicate receiving the user input voice; converting the user input voice into an AI assistant voice; and sending the AI ​​assistant voice to the caller.

[0038] In one possible implementation, the detection module is further used to detect a third operation, where the third operation is used to instruct adjustment of the conversation speed of the AI ​​assistant.

[0039] In a possible implementation, the display module is configured to display information cards and summary cards, wherein the information cards are configured to indicate key content text in the call information, and the summary cards are configured to indicate summary content text in the call information.

[0040] In one possible implementation, the device also includes a text generation module, which is used to generate text data from the detected audio signal, specifically including: the detection module detects the audio signal, identifies the preset type of information in the audio signal to obtain key content text and summary text; the text generation module generates an information card based on the key content text; the text generation module generates a summary card based on the summary text.

[0041] In a possible implementation, the preset types in the audio signal include but are not limited to telephone numbers, addresses, and schedules.

[0042] In a possible implementation, the summary text is text content that summarizes the call content within a preset duration.

[0043] In a third aspect, the present application provides an electronic device for an AI assistant, which may include a processor coupled to a memory, wherein the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method of the first aspect or any implementation of the first aspect is implemented. For details of the steps in each possible implementation of the first aspect executed by the processor, please refer to the first aspect and will not be repeated here.

[0044] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method of any implementation of the first aspect.

[0045] The fifth aspect of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method of any implementation manner of the first aspect.

[0046] The beneficial effects of the second to fifth aspects mentioned above can be referred to the introduction of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] FIG1 is a schematic diagram of an architecture of an application scenario provided by an embodiment of the present application;

[0048] FIG2 is a schematic diagram of a call answering interface provided in an embodiment of the present application, wherein FIG2(a) and FIG2(b) are schematic diagrams of a call application interface for manual answering, and FIG2(c) is a schematic diagram of a call application interface for AI answering;

[0049] FIG3 is a schematic diagram of starting an AI call mode according to an embodiment of the present application;

[0050] FIG4 is a schematic diagram of generating and typing a call voice signal based on a prompt word according to an embodiment of the present application, wherein FIG4 (a) and FIG4 (b) respectively illustrate two different presentation modes;

[0051] FIG5 is a schematic diagram of a call interface after calling out a virtual keyboard according to an embodiment of the present application;

[0052] FIG6 is a schematic diagram of a call interface after calling out a virtual keyboard according to an embodiment of the present application;

[0053] FIG7 is a schematic diagram of generating a summary card according to an embodiment of the present application;

[0054] FIG8 is a schematic diagram of entering a call card interface from an answering interface according to an embodiment of the present application;

[0055] FIG9 is a schematic diagram of an electronic device 100 carrying an AI assistant provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of this application more clear, the embodiments of this application are described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only embodiments of a part of this application, rather than all embodiments. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0057] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the descriptions used in this way can be interchangeable where appropriate so that the embodiments can be implemented in a sequence other than that illustrated or described in this application. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or that are inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The named or numbered process steps can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0058] The division of units in this application is a logical division. In actual application, there may be other division methods. For example, multiple units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection between units can be electrical or other similar forms, which are not limited in this application. Moreover, the units or sub-units described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed into multiple circuit units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this application.

[0059] To facilitate understanding, some technical terms involved in the embodiments of this application are first introduced below.

[0060] (1) AI Assistant

[0061] AI Assistant is an artificial intelligence assistant with functions such as AI question-and-answer assistant, AI drawing assistant, AI analysis assistant, and intelligent voice translation robot, providing users with an efficient and convenient artificial intelligence system platform.

[0062] (2) Message Card

[0063] Message cards can be categorized in a variety of ways. For example, in WeChat, an instant messaging app, unread messages can be categorized by content into text messages, picture messages, video messages, voice messages, and voice / video call messages. They can also be categorized into personal messages and group messages based on the recipient. These different types can all serve as message card categories.

[0064] (3) Large Language Model

[0065] Large language models are an application of deep learning, particularly in natural language processing (NLP). These models are trained on large amounts of text data to learn the various patterns and structures of language. Their goal is to understand and generate human language for effective conversations and to answer various questions.

[0066] With the advancement of technology and the development of society, daily conference calls have become overwhelming, and important calls are often missed due to the multitude of tasks. To make people's lives more convenient and work more efficient, answering calls with the help of AI assistants is an important means. AI assistants can communicate with callers according to a preset tone and generate a call summary based on key information in the conversation, so that even if the user is not present, they can still clearly understand the content of the call.

[0067] This method can effectively avoid missing important calls by using an AI assistant to answer calls; by supporting switching between AI mode and manual mode, it can effectively avoid the problem that the AI ​​assistant has difficulty in coping with complex scenarios and processing complex call content; this method also supports users to guide the tone and content of AI calls in AI call mode, so that the AI ​​assistant can better handle the call content and answer questions in the call according to the user's wishes; in addition, the summary information generated by the large language model based on the call audio data can be used to obtain key information of the conference call and summary information can be used to record the call content within the preset call duration, which can effectively improve work efficiency and user experience.

[0068] In view of this, an embodiment of the present application provides an interactive method for a call interface. Based on the connection settings for important calls, an AI assistant is allowed to answer important calls. Within a period of time after the missed call rings, the AI ​​assistant automatically answers the call and generates a key information summary card based on the call content. In addition, during the AI ​​assistant's answering process, the user can intervene at any time to interact with the AI ​​assistant and guide the AI ​​assistant to better conduct the call. During the call, the AI ​​assistant function can also be turned off at any time to continue the call using manual answering.

[0069] Please refer to Figure 1, which is a schematic diagram of the architecture of an application scenario provided by an embodiment of the present application. As shown in Figure 1, the application scenario includes user 001 and electronic device 002. Among them, electronic device 002 can be a smart terminal electronic device such as a mobile phone, tablet, laptop computer, watch, etc. Specifically, user 001: is usually the owner of electronic device 002, who obtains network information and communicates with others through electronic device 002. Electronic device 002: is equipped with an operating system and has installed a variety of software, such as call applications, text messages, instant messaging software, email applications, etc. The electronic device obtains satellite / network call signals through communication modules such as antennas, enables call applications, and displays a call interface.

[0070] In one possible implementation, user 001 sets the AI ​​assistant in electronic device 002 to automatically answer the call after the phone rings for a period of time (the time can be preset, assuming it is set to 40s). For example, assuming that the electronic device rings for 40 seconds and the user does not answer the call, the AI ​​assistant automatically answers the call.

[0071] In one possible implementation, user 001 receives a call from "Zhang San". At this time, the answer and hang up buttons are displayed on the interface of electronic device 002, and the "Swipe up to answer by AI assistant" icon is displayed at the bottom. Swipe up on the AI ​​assistant button to enable the AI ​​assistant to answer the call.

[0072] The above solution is explained using the example of an AI assistant answering a call. In other implementations of the present application, the AI ​​assistant can also help answer voice calls, etc., and can use voice communication with other users.

[0073] Please refer to FIG2 , which is a schematic diagram of a call answering interface provided in an embodiment of the present application.

[0074] S101: The AI ​​assistant icon is displayed on the incoming call interface

[0075] The electronic device detects an incoming call signal and displays an incoming call interface with an AI assistant icon. A calling application is installed on the electronic device, which can be a built-in satellite calling application or an Internet calling application. The calling application receives the incoming call signal through a communication module, such as a satellite signal received through an antenna. After receiving the satellite signal, the calling application starts and displays the calling interface on the front display.

[0076] In this solution, the incoming call interface includes the pre-answering interface and the answering interface. The pre-answering interface is the call application interface before the user performs a preset action. The answering interface is the interface after the user performs a preset action and the call is connected. The preset action is a system-defined action to connect the call, for example, dragging the answer button to answer the call.

[0077] The AI ​​assistant icon is displayed on both the pre-answering interface and the answering interface. In this solution, the preset operation can also be an interactive operation with the AI ​​assistant, such as dragging the AI ​​assistant icon to move, interacting with the AI ​​assistant through voice input, etc.

[0078] Exemplarily, Figure 2 shows the pre-answering interface and the answering interface respectively. Among them, Figure 2 (a) and Figure 2 (b) are schematic diagrams of the call application interface for manual answering, and Figure 2 (c) is a schematic diagram of the call application interface for AI answering. Figure 2 (a) is the pre-answering interface, and Figure 2 (b) and Figure 2 (c) are the answering interfaces. In the pre-answering interface of Figure 2 (a), in addition to the answer button and the hang-up button, and the answering interface of Figure 2 (b), in addition to the hang-up button, an AI assistant icon is also displayed at the bottom of the screen. Before the user interacts with the AI ​​assistant icon, the AI ​​assistant icon is only partially displayed on the screen, and part of it is outside the screen, that is, the AI ​​assistant is not fully displayed on the pre-answering interface. The user drags the AI ​​assistant icon to move it, and the AI ​​assistant icon can be fully displayed on the screen. In (c) of Figure 2, in AI answering mode, the AI ​​assistant icon is fully displayed at the preset position on the screen, and the AI ​​assistant application will synchronously receive the audio signal, understand the audio, and then combine the user's input and the AI ​​assistant's semantic understanding of audio-to-text to output AI voice and send it to the caller.

[0079] S102: Enable AI call mode

[0080] When the AI ​​assistant icon is detected to move from the bottom of the screen to the middle of the screen, the AI ​​call mode is enabled and the call audio is synchronously transmitted to the AI ​​assistant application; in the interface before answering the call, the user can interact with any of the answer, hang up, and AI assistant icons to handle the call. The interaction of the answer and hang up buttons is consistent with the existing technology. The user can drag the AI ​​assistant icon to move, drag the AI ​​assistant icon from the bottom of the screen to the preset area in the middle of the screen, or drag a distance greater than the preset value to enable the AI ​​call mode (the opposite of the AI ​​call mode is the manual answering mode).

[0081] Please refer to Figure 3, which is a schematic diagram of starting the AI ​​call mode provided by an embodiment of the present application. For example, Figure 3 shows the process of enabling the AI ​​call mode. After the user selects the AI ​​assistant icon, he drags the AI ​​assistant to move on the screen. When it moves to the preset area of ​​the screen (such as the dotted box area in the middle figure), if the gesture is released at this time, it is considered that the AI ​​call answering condition is met and the AI ​​call is enabled. After the release, the AI ​​assistant icon will move from the gesture release position to the preset position of the answering interface.

[0082] In one possible implementation, in AI call mode, in addition to sending the received audio data to the call application, the electronic device will also send it to the AI ​​assistant application simultaneously. After the AI ​​assistant calls the language model based on the received audio data for understanding, it outputs the reply text corresponding to the audio, and converts the reply text into a voice signal and sends it to the call application, which is then sent to the caller by the call application. At the same time, in AI call mode, when the call application sends a voice signal to the caller, it will also synchronously send the voice signal to the audio output module for output, such as directly through the speaker, or through the Bluetooth module to the Bluetooth headset, so that the user can obtain all conversations between the AI ​​assistant and the caller in real time as a third party.

[0083] In one possible implementation, the electronic device receives input based on a prompt word and generates and types a call voice signal. Specifically, the prompt word is detected, a call voice signal is generated based on the prompt word, and the call voice signal is sent to the caller. In AI call answering mode, the user can enter the prompt word, and the AI ​​assistant generates a reply text based on the user's prompt word and its understanding of the received audio information. The reply text is converted into a call voice, which is sent to the call application, which then sends it to the caller / audio output module.

[0084] In one possible implementation, the prompt word can be input as a recommended word selected by the user on the call interface, or as an input statement entered by the user on the virtual keyboard in the call interface. Specifically, after enabling the AI ​​call answering mode, the AI ​​assistant can display recommended words on the call interface, and the user can influence the call statement output by the AI ​​by selecting the recommended words. In addition, the call interface in the AI ​​call mode can also display a virtual keyboard icon. The user clicks the virtual keyboard icon to enable the virtual keyboard for input and directly enters a statement as the prompt word.

[0085] In this solution, there are several ways to generate recommendation words:

[0086] Recommended words can include fixed and real-time recommendations. Fixed recommendations are fixed words that apply to all AI calls. Real-time recommendations are generated by the AI ​​assistant based on its understanding of the received voice signal and are directly associated with the caller's audio sentence. In other words, real-time recommendations are generated based on real-time recognition of the current call content and are generated based on the semantics of the current caller.

[0087] 1) Fixed recommendation words

[0088] Fixed recommended words include tone-related words and process control-related words. Tone-related words influence the output of the spoken text. For example, choosing a formal or lively tone will result in different output text: a stern tone results in a more formal text, while a lively tone results in a more colloquial text.

[0089] Progress control recommendations are used to control the progress and duration of a call, including options for quickly ending or maintaining the call. For example, selecting "Quick End" will cause the AI ​​assistant to add a statement indicating the intended end of the call after completing / while executing the current output, thereby conveying the caller's intention to end the conversation (rather than simply hanging up).

[0090] 2) Real-time recommended words

[0091] Real-time recommendation words can be generated by the AI ​​assistant application based on the received audio signal, or generated by the AI ​​assistant application based on the expected reply content output for the received audio signal. The expected reply content is the text generated by the AI ​​assistant based on the received audio signal and has not yet been sent to the call application.

[0092] Specifically, real-time recommendation words can be keywords extracted from the text obtained after converting the received audio signal into text. For example, if the text obtained by voice conversion is "Where should we eat tonight, restaurant A or restaurant B?", the keywords "restaurant A" and "restaurant B" will be extracted as recommendation words. In addition, derivative words of the keywords can be obtained according to preset rules based on the keywords and context. For example, if the aforementioned voice is a question sentence and two options are given - restaurant A and restaurant B, in addition to choosing an answer from restaurant A and restaurant B, the other party can also decide. Therefore, the derivative word "it is decided by the other party" can be obtained.

[0093] Alternatively, real-time recommendations can be keywords extracted from the expected reply, or derived from keywords extracted from the expected reply based on pre-set rules. For example, if the text received by voice-redirected is "Do you want to go to dinner tonight?", the AI ​​can output the response words "Agree" and "Reject" as well as "Watch it later," thus generating the derived word "Watch it later."

[0094] 3) Presentation of recommended words

[0095] Recommended words can be presented by category or by individual word. Each recommended word corresponds to a bubble, which is displayed in a preset area on the call interface. The recommended word bubble can be static or dynamic. For example, the recommended word bubble can move within the preset area, such as displaying a dynamic bubble moving from inside the screen to outside the screen. A single recommended word can be displayed for a period of time and then displayed for a period of time.

[0096] All recommended words can be displayed on the call interface, or only preset types of recommended words can be displayed on the call interface. Non-preset types of recommended words are collapsed, and only after the user performs the preset operation will the recommended words of that type be expanded and displayed on the call interface.

[0097] Real-time recommendations have a direct impact on the output results of AI calls, while tone recommendations only modify the output results. Process control recommendations are only called when the user needs them. Therefore, all real-time recommendations are displayed in the call interface in the form of recommendation bubbles. Tone recommendations and process control recommendations only display two type controls. Click the type control to display the recommendation of that type on the call interface.

[0098] Please refer to Figure 4, which is a schematic diagram of a method for generating and typing a call voice signal based on a prompt word according to an embodiment of the present application. For example, Figure 4 (a) and Figure 4 (b) respectively show two different presentation methods. In Figure 4 (a), two rows of recommendation words are displayed in a static manner. One row of real-time recommendation words, such as "politely decline", "accept the other party's invitation", "give a conclusion later" and so on, are related to the current call content and need to be recommended in real time based on the current call content; one row of fixed recommendation words, such as "relaxed tone", "lively tone", "formal tone", which are guidance and suggestion choices for the tone of the AI ​​assistant's reply. All types of recommendation words are collapsed and are directly displayed in the form of recommendation word bubbles (limited by the screen area, some bubbles are displayed outside the screen). Swiping left and right can display more recommendation word bubbles. In (b) of Figure 4, real-time recommended words are displayed around the icon of the AI ​​assistant and change their position dynamically over time, with dynamic effects (for example, bubbles move from inside the screen to outside the screen, gradually appear, and their transparency decreases). Fixed recommended words only display category controls in fixed positions - "tone selection" and "call control". Click the corresponding category control to display the recommended word of that type in the call interface in the form of a recommended word bubble. The displayed result refers to the fixed recommended word in (a) of Figure 4.

[0099] In one implementation, if the displayed recommended words do not meet the user's needs, the user can also click the virtual keyboard icon in the lower left corner to call out the virtual keyboard and manually enter a sentence as the recommended word.

[0100] Please refer to Figure 5, which is a schematic diagram of a call interface after calling up the virtual keyboard according to an embodiment of the present application. For example, as shown in Figure 5, after the user enters custom text and clicks Send, the virtual keyboard transmits the user-entered text to the AI ​​assistant application. After receiving the input text, the AI ​​assistant application uses the input text as a recommended word to generate a reply text. For example, the input text is directly used as the reply text.

[0101] After receiving the recommendation word, the AI ​​assistant application inputs the text converted from the recommendation word and the received audio into the built-in / cloud-based language model to obtain the reply text, and converts the reply text into audio and outputs it to the call application.

[0102] When the user selects a tone recommendation word, he or she can also select different voices based on the tone recommendation word to synthesize the audio when converting the reply text into audio. The correspondence between the tone recommendation word and the voice can be pre-defined.

[0103] For example, different tone recommendations can correspond to different voice models, with the corresponding voice model selected based on the tone recommendation; or different tone recommendations can correspond to different voice tags, with the voice model selected based on the tags. It is understood that to ensure uniformity in the output audio, different voice models can be fine-tuned based on the same baseline voice to achieve different tones.

[0104] When the call application sends the received audio, it will also synchronously call the audio output module to play the received audio, so that the user can receive the voice conversation between the AI ​​assistant and the caller in real time.

[0105] In the AI ​​call answering interface, in addition to the recommended word interaction, the following interactions can also be included:

[0106] ① Long press the AI ​​assistant to send voice: This can be achieved in two ways. One is to long press to trigger the AI ​​assistant to receive the voice signal and send the voice signal to the call application, which then sends it to the other end; the other is to long press the AI ​​assistant to trigger the interruption of the AI ​​call mode and temporarily switch to the manual answering mode. The call application directly receives and sends the user's voice. When it detects that the long press gesture is released, it switches back to the AI ​​call mode.

[0107] ② Drag the AI ​​assistant to move, and adjust the speaking speed of the AI ​​assistant by controlling the direction of movement and giving examples. For example, drag the AI ​​assistant left / right to adjust the speaking speed. Of course, it can also be moved up and down. You can adjust the speaking speed by directly dragging the AI ​​assistant to move, or you can activate the speaking speed adjustment function through preset gestures (such as long pressing for more than the preset value), and then adjust the speaking speed by dragging the AI ​​assistant to move.

[0108] In one possible implementation, the electronic device can switch the AI ​​call mode to the manual answering mode. Specifically, it is detected that the AI ​​assistant icon moves from the middle of the screen to the bottom of the screen, and the AI ​​call mode is switched to the manual answering mode. During the AI ​​call, the user can switch the AI ​​call mode to the manual answering mode by moving the AI ​​assistant icon to the bottom of the screen. Specifically, if the electronic device detects the movement of the AI ​​assistant icon again, the electronic device obtains the position of the AI ​​assistant icon in real time according to the operation gesture. If the AI ​​assistant icon falls at least partially in a preset area (such as outside the screen display area), the electronic device ends the AI ​​call mode, and the call application no longer sends the audio data to the AI ​​assistant application, but switches to the manual answering mode and outputs it only through the audio output module.

[0109] Please refer to Figure 6, which is a schematic diagram of a call interface after calling up the virtual keyboard according to an embodiment of the present application. For example, Figure 6 illustrates the interactive process of switching from AI call mode to manual answering mode. After selecting the AI ​​assistant icon, drag the AI ​​assistant icon downward to move it at least partially outside the screen display area to switch to manual answering mode.

[0110] In one possible implementation, the AI ​​assistant application can also generate summary cards and information cards during a call, where the summary card is used to record key information during the call; the information card is used to summarize the call information during the call.

[0111] In one possible implementation, the electronic device can generate a summary card. Specifically, the answering interface is displayed, the AI ​​assistant application obtains the audio signal of the incoming call in real time, identifies the preset type of information in the audio signal to obtain key content text, and generates a summary text based on the audio signal. The call application generates an information card based on the key content text, and generates a summary card based on the summary text, which is displayed on the answering interface.

[0112] After the call is connected (whether in AI answering mode or manual answering mode), the call application enters the answering interface. After entering the answering interface, the AI ​​assistant application obtains the audio signal received by the call application in real time and analyzes the audio signal to identify the preset type of information (key content text) and obtain summary information within the preset time (summary text). For example, after the audio signal is converted from speech to text, the obtained text is input into the large language model to obtain the key content text and summary text.

[0113] In this solution, the preset types of information include: phone numbers, addresses, schedules, etc. When the AI ​​assistant application detects that the audio contains information of the preset type, it extracts the preset information separately and obtains the key content text. Specifically, the large language model can be trained to recognize phone number formats in various countries and address descriptions. Therefore, when the text is input into the large language model, the text content that matches the phone number format / address description is extracted separately.

[0114] In one implementation, the AI ​​assistant usually also has the ability to summarize and generalize, and can summarize the caller's speech to obtain a summary text. The summary text can be generated in units of time. For example, a summary text is generated every preset time (such as 1 minute). It is also possible to segment the audio sentences based on the pause time of the caller in the audio, with a pause time exceeding a preset time (such as 5s), and generate a summary text according to the audio sentences. Of course, the AI ​​assistant can also break the sentences by itself and generate a summary text in units of sentences. Alternatively, the AI ​​assistant makes topic judgments based on the audio content and generates summary text by topic.

[0115] In one implementation, the AI ​​assistant app sends the extracted key content and summary text to the call app. After receiving the key content and summary text, the call app generates different cards based on the content type and displays them on the call interface. Specifically, an information card corresponding to the key content and a summary card corresponding to the summary text are generated, and the information and summary cards are then displayed on the answering interface.

[0116] In one implementation, due to limited screen area, the content presented in the answering interface is limited, and not all summary cards and information cards can be displayed. Since information cards usually have less content and are important information, while summary cards are relatively unimportant information, only the information cards can be displayed in full, and the summary cards can be displayed in stacked form. The stacked summary cards only display the latest summary card in full.

[0117] Please refer to Figure 7, which is a schematic diagram of generating a summary card provided in an embodiment of the present application. For example, Figure 7 shows an answering interface, where an information card A01, "Location: Restaurant B, BB Road, AA District," is displayed above the caller's avatar. In the middle of the screen, a summary card B01, "Department is organizing a dinner party tonight at 7:00 PM, location: Restaurant B, BB Road, AA District," is displayed above the AI ​​assistant icon.

[0118] In one possible implementation, the electronic device can also generate summary cards and information cards. Specifically, upon detecting a preset action, the call card interface is entered, where all information cards and summary cards for the call are displayed. If the user wishes to view all summary cards, a preset action can be used to display both the information cards and summary cards for the entire call. For example, swiping right enters the call card interface, where all information cards and summary cards are displayed.

[0119] Please refer to Figure 8, which is a schematic diagram of entering the call card interface from the answering interface provided in an embodiment of the present application. For example, Figure 8 shows a schematic diagram of entering the call card interface from the answering interface. Swiping a preset distance to the right from the left side of the answering interface screen (or any position) enters the call card interface. The call card interface displays all information cards and summary cards, where key information corresponds to the information card and the call summary corresponds to the summary card.

[0120] After the information card and summary card are generated, even if you hang up the call, the information card and summary card will still exist until you exit the call application. After exiting the call application, the information card / summary card will be destroyed. Of course, you can also assign a folder to each call and store the information card and summary card in the folder.

[0121] Information cards and summary cards can also support interaction. For example, long press to add an information card / summary card to a note / memo / contact. Specifically: after long pressing the card, a secondary menu appears at the corresponding card position. According to the secondary menu option selected, the content of the information card / summary card is added as a new note / contact, etc.

[0122] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of method. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the modules and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0123] The embodiment of the present application can divide the functional modules of the electronic device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0124] The electronic device in the embodiment of the present application is described in detail below:

[0125] Please refer to Figure 9, which is a schematic diagram of an electronic device 100 carrying an AI assistant provided in an embodiment of the present application, including an input module 101, a detection module 102, a text generation module 103 and a display module 104.

[0126] The detection module 101 is used to detect whether the AI ​​call mode is enabled, which refers to a mode in which an AI assistant is used to talk to the callee; the detection module is also used to detect a first operation, which is used to indicate that the AI ​​call mode is switched to a manual call mode, which refers to a mode in which a user talks to the callee.

[0127] The input module 102 is used to input a prompt word, generate a call voice signal based on the prompt word, and send the call voice signal to the caller.

[0128] The text generation module 103 is used to generate prompt words. The text generation module 103 generates text information based on the received audio signal, identifies the preset type of audio data, and generates summary text and summary text based on the large prediction model. The text generation module generates information cards based on the summary text, and the text generation module generates summary cards based on the summary text.

[0129] Display module 104 is used to display information cards and summary cards. The information cards are used to indicate the key content text in the call information, and the summary cards are used to indicate the summary content text in the call information. This display module is also used to display the AI ​​assistant icon, which is used to interact with the user to switch modes.

[0130] In one possible implementation, the detection module 102 detects that the AI ​​call mode is enabled, and the AI ​​assistant conducts a voice call with the caller. During the call, the input module 101 allows the user to input text data, voice data, and select the content of the control according to actual needs and the content of the call. The detection module 102 detects the input data in real time and adjusts the call content and call status of the AI ​​assistant according to the input data, thereby enabling the user to interact with the AI ​​assistant in real time during the call between the AI ​​and the caller. In addition, the display interface will display the AI ​​assistant icon and prompt words in real time. The prompt words can be dynamically generated based on the audio data, or can be input by the user in real time, or can be of a fixed type. The above prompt words can control the call process and call time by controlling the content and tone of the AI ​​assistant's call response, etc.

[0131] In one possible implementation, the electronic device supports switching of call modes through the AI ​​assistant icon, for example, switching from AI call mode to manual call mode; or switching from manual call mode to AI call mode.

[0132] These electronic devices typically feature an AI assistant, allowing the AI ​​assistant to answer calls and interact with the user in AI call mode. This ensures the AI ​​assistant conducts calls according to the user's wishes and effectively guarantees call quality. Furthermore, the electronic device can generate corresponding summary text and a summary based on the audio data of the call, making it easier for users to review and record meeting content.

[0133] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0134] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, training equipment or data center to another website, computer, training equipment or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training equipment, data center, etc. that includes one or more available media integrations. Available media can be magnetic media, (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive (SSD)), etc.

Claims

1. A call interface interaction method, characterized in that: Applied to electronic devices with AI assistant applications, including: Detecting that the AI ​​call mode is enabled, where the AI ​​call mode refers to a mode in which the AI ​​assistant is used to talk to a caller; A first operation is detected, where the first operation is used to instruct to switch the AI ​​call mode to the manual call mode, where the manual call mode refers to a mode in which the user has a call with the callee.

2. The method according to claim 1, characterized in that The method further includes: detecting input of a prompt word, generating a call voice signal based on the prompt word, and sending the call voice signal to the caller.

3. The method according to claim 1, characterized in that The method further comprises: The call interface displays information cards and summary cards, wherein the information card is used to indicate key content text in the call information, and the summary card is used to indicate summary content text in the call information.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: The call interface displays an AI assistant icon, which is used to interact with the user to switch modes.

5. The method according to claim 2, characterized in that: The prompt words specifically include fixed recommendation words, and the fixed recommendation words include tone recommendation words and control recommendation words; The tone recommendation words are used to indicate the tone of the output voice text; The control-type recommended words are used to output voice text to control the progress and duration of the call.

6. The method according to claim 2, characterized in that The prompt words specifically include real-time recommendation words, which are text data generated according to real-time call content and are used to reply to the caller.

7. The method according to claim 2, characterized in that The prompt word specifically includes user input text, which is manually input by the user and is text data used to express the user's wishes and ideas.

8. The method according to claim 2, characterized in that: The prompt word specifically includes a user input voice, and the method includes: A second operation is detected, where the second operation is used to indicate receiving the user input voice; Convert the user input voice into AI assistant voice; And send the AI ​​assistant voice to the caller.

9. The method according to claim 1, characterized in that: The method further comprises: A third operation is detected, and the conversation speed of the AI ​​assistant is adjusted.

10. The method according to claim 3, characterized in that: The call interface displays an information card and a summary card, wherein the information card is used to indicate key content text in the call information, and the summary card is used to indicate summary content text in the call information.

11. The method according to claim 10, characterized in that The call interface displays information cards and summary cards specifically including: An audio signal is detected, and information of a preset type in the audio signal is identified to obtain a key content text and a summary text; Generate an information card based on the key content text; A summary card is generated based on the summary text.

12. The method according to claim 10, characterized in that The preset types in the audio signal include but are not limited to telephone numbers, addresses, and schedules.

13. The method according to claim 10, characterized in that The summary text is a text content that summarizes the call content within a preset time period.

14. An electronic device carrying an AI assistant, characterized in that: The device comprises: A detection module, used to detect the activation of an AI call mode, where the AI ​​call mode refers to a mode in which the AI ​​assistant is used to talk to a caller; The detection module is further used to detect a first operation, where the first operation is used to instruct to switch the AI ​​call mode to a manual call mode, where the manual call mode refers to a mode in which the user has a call with the callee.

15. The electronic device according to claim 14, characterized in that: The device also includes: The input module is used to input a prompt word, generate a call voice signal based on the prompt word, and send the call voice signal to the caller.

16. The electronic device according to claim 14, characterized in that: The device also includes: The display module is used to display information cards and summary cards, wherein the information cards are used to indicate key content texts in the call information, and the summary cards are used to indicate summary content texts in the call information.

17. The electronic device according to any one of claims 14 to 16, characterized in that: The display module is also used to display an AI assistant icon, and the AI ​​assistant icon is used to interact with the user to achieve mode switching.

18. The electronic device according to claim 15, characterized in that: The prompt words specifically include fixed recommendation words, and the fixed recommendation words include tone recommendation words and control recommendation words; The tone recommendation words are used to indicate the tone of the output voice text; The control-type recommended words are used to output voice text to control the progress and duration of the call.

19. The electronic device according to claim 15, characterized in that: The prompt words specifically include real-time recommendation words, which are text data generated according to real-time call content and are used to reply to the caller.

20. The electronic device according to claim 15, characterized in that: The prompt word specifically includes user input text, which is manually input by the user and is text data used to express the user's wishes and ideas.

21. The electronic device according to claim 15, characterized in that: The prompt word specifically includes a user input voice, and the method includes: A second operation is detected, where the second operation is used to indicate receiving the user input voice; Convert the user input voice into AI assistant voice; And send the AI ​​assistant voice to the caller.

22. The electronic device according to claim 14, characterized in that: The detection module is also used to detect a third operation and adjust the conversation speed of the AI ​​assistant.

23. The electronic device according to claim 14, characterized in that: The display module is used to display information cards and summary cards, wherein the information cards are used to indicate key content text in the call information, and the summary cards are used to indicate summary content text in the call information.

24. The electronic device according to claim 23, characterized in that: The device also includes a text generation module, which is used to generate text information from the detected audio information; specifically including: The audio signal is detected, and information of a preset type in the audio signal is identified to obtain key content text and summary text; The text generation module generates an information card based on the key content text; The text generation module generates a summary card based on the summary text.

25. The electronic device according to claim 23, characterized in that: The preset types in the audio signal include but are not limited to telephone numbers, addresses, and schedules.

26. The electronic device according to claim 23, characterized in that: The summary text is a text content that summarizes the call content within a preset time period.

27. An electronic device carrying an AI assistant, characterized in that: The device comprises a memory and a processor; the memory stores codes, the processor is configured to execute the codes, and when the codes are executed, the device executes the method according to any one of claims 1 to 13.

28. A computer storage medium, characterized in that The computer storage medium stores instructions, which, when executed by a computer, cause the computer to implement the method of any one of claims 1 to 13.

29. A computer program product, characterized in that The computer program product stores instructions, which, when executed by a computer, cause the computer to implement the method according to any one of claims 1 to 13.

Images