Interaction method, terminal and readable storage medium

Abstract

The present application discloses an interaction method, a terminal and a readable storage medium. The interaction method comprises: obtaining and identifying first operation information; detecting whether the first operation information satisfies a first preset condition; if so, obtaining and executing second operation information. Based on this, the present application can control the voice assistant through gestures to reduce the probability of misoperation, and can still facilitate other operations after turning off this function, such as answering calls within a predetermined distance.

Description

Technical Field

[0001] The present application relates to the technical field of terminal and human-computer interaction, and in particular to an interaction method, a terminal and a readable storage medium. Background Art

[0002] At present, human-computer interaction technology has achieved many research results, such as tactile interaction technology used in virtual reality, remote-controlled robots and telemedicine, eye tracking technology used in advertising, websites, product catalogs, magazine utility tests, and brain wave-based human-computer interface technology used in "mind wheelchairs" developed for people with language and movement disorders. Among them, voice-based human-computer interaction has become a hot technology in the field of human-computer interaction due to its low cost, high popularity and strong practicality, and has great application potential. Take smart speakers as an example, they can convert voice into text and perform functions such as music playback according to text instructions.

[0003] However, the existing human-computer interaction technology based on voice has a high probability of misoperation. For example, during a conversation, the terminal can easily capture the content of the conversation and perform corresponding functions based on it, which is obviously against the user's original intention. In addition, after turning off the human-computer interaction function based on voice on the existing terminal, the user will have some inconvenience in other terminal operations.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the invention

[0005] In view of this, the present application provides an interaction method, a terminal and a readable storage medium to solve the problem of high probability of misoperation of existing voice interaction functions.

[0006] The present application provides an interaction method, which is applied to a terminal and includes:

[0007] S11, acquiring and identifying first operation information;

[0008] S12, detecting whether the first operation information meets a first preset condition;

[0009] S13: If yes, obtain and execute the second operation information.

[0010] Optionally, if the first operation information does not meet the first preset condition, the method further includes:

[0011] If an incoming call is detected, determining whether the current distance between the terminal and the user is less than or equal to a first preset distance;

[0012] If yes, answer the call; and / or,

[0013] If not, hang up the call and / or start the voice assistant to output a prompt message.

[0014] Optionally, after the step of outputting the prompt information, the method further includes:

[0015] Hang up or answer the call according to the received voice command.

[0016] Optionally, the step S13 includes:

[0017] Detecting whether the terminal meets a second preset condition;

[0018] If so, the received second operation information is obtained and executed.

[0019] Optionally, the second preset condition includes at least one of the following:

[0020] The terminal detects an incoming call event;

[0021] The distance between the terminal and the user is greater than or equal to a second preset distance.

[0022] Optionally, when the terminal detects an incoming call event, the step S13 includes:

[0023] Check whether the incoming call number is stored in the address book and start the voice assistant;

[0024] If yes, then the incoming call number is voice broadcasted according to the preset information, and the incoming call is answered or hung up according to the received voice command; and / or,

[0025] If not, the incoming call number is classified and recorded.

[0026] Optionally, when the distance between the terminal and the user is greater than or equal to a second preset distance, the step S13 includes:

[0027] Start the voice assistant and output prompt information to remind you to perform voice operations;

[0028] Make calls and / or open applications based on received voice commands.

[0029] Optionally, after the step of starting the application according to the received voice instruction, the method further includes:

[0030] If the control instruction is not received within the first predetermined time, a prompt message is output;

[0031] If no control instruction is received within a second preset time after the prompt message, a frequently used contact is selected for contact according to the attribute of the application.

[0032] Optionally, the first operation information includes at least one of the following: operation gesture, operation duration, operation frequency, operation position; and / or,

[0033] The first preset condition includes at least one of the following:

[0034] The operation gesture is a preset gesture;

[0035] The operation duration meets the preset duration;

[0036] The operating frequency reaches a preset frequency;

[0037] The operating position is a preset position.

[0038] The present application provides a human-computer interaction method based on speech, comprising:

[0039] Obtain and recognize user gesture information;

[0040] Detecting whether the user's gesture information matches preset gesture information;

[0041] When matching, the corresponding operation is performed according to the voice command issued by the user;

[0042] When there is no match, if an incoming call is detected, detecting whether the current distance between the user and the user is less than or equal to a first preset threshold; and

[0043] When the current distance is less than or equal to the first preset threshold, answering the incoming call;

[0044] When the current distance is greater than the first preset threshold, hang up the call.

[0045] Optionally, performing a corresponding operation according to a voice command issued by a user includes:

[0046] When an incoming call is detected, check whether the incoming call number has been recorded in the address book;

[0047] If the incoming call number has been recorded in the address book, a voice broadcast is performed according to the preset information of the incoming call number in the address book, and a prompt is given as to whether to answer the call; and,

[0048] Answer or hang up the call according to the received user instructions;

[0049] If the incoming call number is not recorded in the address book, the incoming call number is classified and recorded.

[0050] Optionally, performing a corresponding operation according to a voice command issued by a user includes:

[0051] When it is detected that the distance between the user is less than or equal to a second preset threshold, a call is made or an application is started according to a voice command issued by the user.

[0052] Optionally, performing a corresponding operation according to a voice command issued by a user includes:

[0053] Launching the corresponding application according to the voice command issued by the user, and prompting the user at least once if no command issued by the user is received within a predetermined time;

[0054] If no instruction from the user is received after prompting at least once, a frequently used contact is selected for contact according to the attributes of the application.

[0055] Optionally, when the current distance is greater than the preset threshold, after hanging up the call, the human-computer interaction method includes:

[0056] Record information about missed calls;

[0057] When it is detected that the user's gesture information matches the preset gesture information, the recorded relevant information of the missed call is broadcasted.

[0058] The present application provides a terminal including a processor and a memory; the memory stores computer-executable instructions, and when the computer-executable instructions are executed by the processor, any one of the above-mentioned interaction methods is implemented.

[0059] The present application provides a readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, any of the above-mentioned interaction methods is implemented.

[0060] A terminal provided in the present application includes a processor, a memory, a gesture sensing module, a distance sensing module, and a communication bus, wherein the communication bus is used to realize the connection between the processor, the memory, the gesture sensing module, and the distance sensing module;

[0061] The memory is used to store preset gesture information and a first preset threshold;

[0062] The gesture sensing module is used to obtain and recognize the user's gesture information;

[0063] The processor is used to detect whether the user's gesture information matches the preset gesture information; and, when matched, to perform a corresponding operation according to the voice command issued by the user; when not matched, if an incoming call is detected, to control the distance sensing module to detect the current distance between the user and the distance sensing module;

[0064] The processor is also used to detect whether the current distance is less than or equal to a first preset threshold; and, when the current distance is less than or equal to the first preset threshold, answer the call; when the current distance is greater than the first preset threshold, hang up the call.

[0065] Optionally, the terminal further includes a speaker, and when detecting that the gesture information of the user matches the preset gesture information,

[0066] The processor is used to detect whether the incoming call number has been recorded in the address book when a call comes in;

[0067] If the incoming call number has been recorded in the address book, the speaker is controlled to perform voice broadcast according to the preset information of the incoming call number in the address book, and prompts whether to answer the call; and,

[0068] Answer or hang up the call according to the received user instructions;

[0069] If the incoming call number is not recorded in the address book, the incoming call number is classified and recorded in the memory.

[0070] Optionally, the memory is used to store preset gesture information and a second preset threshold, and when it is detected that the gesture information of the user matches the preset gesture information,

[0071] The distance sensing module is used to detect the distance between the user and the processor, when detecting that the distance between the user is less than or equal to the second preset threshold, makes a call or opens an application according to a voice command issued by the user.

[0072] Optionally, when detecting that the user's gesture information matches preset gesture information,

[0073] The processor is used to start the corresponding application according to the voice command issued by the user, and prompt the user at least once when the command issued by the user is not received within a predetermined time;

[0074] If no instruction from the user is received after prompting at least once, a frequently used contact is selected for contact according to the attributes of the application.

[0075] The present application performs corresponding operations according to operation information that meets preset conditions, can control the voice interaction function through gesture information, and reduce the probability of misoperation. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0077] Figure 1 A schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application;

[0078] Figure 2 A communication network system architecture diagram provided for an embodiment of the present application;

[0079] Figure 3 is a flowchart of the interaction method of the first embodiment of the present application;

[0080] Figure 4 is a flowchart of an interactive method according to a second embodiment of the present application;

[0081] Figure 5 is a flowchart of an interactive method according to a third embodiment of the present application;

[0082] Figure 6 is a flowchart of an interaction method according to a fourth embodiment of the present application;

[0083] Figure 7 This is a schematic diagram of an interface for implementing a voice interaction function according to an embodiment of the present application;

[0084] Figure 8 is a flowchart of an interactive method according to a fifth embodiment of the present application;

[0085] Fig. 9 It is a schematic diagram of the structure of a terminal according to an embodiment of the present application.

[0086] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0087] The exemplary embodiments will be described in detail below, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0088] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0089] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0090] The terminal may be implemented in various forms. For example, the terminal described in the present application may include mobile terminals such as mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0091] The following description will be made by taking a mobile terminal as an example, and those skilled in the art will understand that, in addition to components specifically used for mobile purposes, the construction according to the embodiments of the present application can also be applied to fixed-type terminals.

[0092] See also Figure 1 , which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application, the mobile terminal 100 may include: RF (Radio Frequency, radio frequency) unit 101, WiFi module 102, audio output unit 103, A / V (audio / video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111 and other components. Those skilled in the art can understand that Figure 1 The structure of the mobile terminal shown in the figure does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0093] Combine the following Figure 1 The following is a detailed introduction to the various components of the mobile terminal:

[0094] The radio frequency unit 101 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving the downlink information of the base station, it is sent to the processor 110 for processing; in addition, the uplink data is sent to the base station. Generally, the radio frequency unit 101 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 101 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution) and TDD-LTE (Time Division Duplexing-Long Term Evolution), etc.

[0095] WiFi is a short-range wireless transmission technology. Mobile terminals can help users send and receive emails, browse web pages, and access streaming media through WiFi module 102. It provides users with wireless broadband Internet access. Figure 1 The WiFi module 102 is shown, but it is understandable that it is not an essential component of the mobile terminal and can be omitted as needed without changing the essence of the invention.

[0096] The audio output unit 103 can convert the audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, etc. Moreover, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0097] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processor (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 106. The image frame processed by the graphics processor 1041 can be stored in the memory 109 (or other storage medium) or sent via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in operating modes such as a telephone call mode, a recording mode, and a voice recognition mode, and can process such sound into audio data. The processed audio (voice) data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 can implement various types of noise elimination (or suppression) algorithms to eliminate (or suppress) noise or interference generated in the process of receiving and sending audio signals.

[0098] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can also be configured on the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0099] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0100] The user input unit 107 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile terminal. Specifically, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1071 or near the touch panel 1071 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 110, and can receive and execute the command sent by the processor 110. In addition, the touch panel 1071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Specifically, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc., which are not specifically limited here.

[0101] Furthermore, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of the touch event. Then, the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of the touch event. Figure 1 In the figure, the touch panel 1071 and the display panel 1061 are used as two independent components to implement the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to implement the input and output functions of the mobile terminal, which is not limited here.

[0102] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

[0103] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 109 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0104] The processor 110 is the control center of the mobile terminal. It uses various interfaces and lines to connect various parts of the entire mobile terminal. It executes various functions of the mobile terminal and processes data by running or executing software programs and / or modules stored in the memory 109, and calling data stored in the memory 109, so as to monitor the mobile terminal as a whole. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 110.

[0105] The mobile terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0106] although Figure 1 Not shown, the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0107] In order to facilitate understanding of the embodiments of the present application, the communication network system on which the mobile terminal of the present application is based is described below.

[0108] See also Figure 2 , Figure 2 A communication network system architecture diagram is provided for an embodiment of the present application, wherein the communication network system is an LTE system of universal mobile communication technology, and the LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are connected in sequence for communication.

[0109] Specifically, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.

[0110] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 , etc. Among them, eNodeB 2021 can be connected to other eNodeBs 2022 through a backhaul (eg, an X2 interface), and eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 can provide UE 201 with access to EPC 203 .

[0111] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, PGW (PDN Gate Way) 2035 and PCRF (Policy and Charging Rules Function) 2036, etc. Among them, MME2031 is a control node that processes signaling between UE201 and EPC203, and provides bearer and connection management. HSS2032 is used to provide some registers to manage functions such as home location register (not shown in the figure), and save some user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, which selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0112] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem) or other IP services.

[0113] Although the above introduction takes the LTE system as an example, those skilled in the art should know that the present application is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA and future new network systems, etc., which are not limited here.

[0114] Based on the above-mentioned mobile terminal hardware structure and communication network system, various embodiments of the present application are proposed.

[0115] The existing voice interaction technology has the problem of high probability of misoperation of the interactive function, resulting in unsatisfactory voice interaction effect. In view of this, the embodiment of the present application provides an interaction method, which controls the voice interaction function (such as voice assistant) through gesture information, and reduces the probability of misoperation while realizing voice control, thereby improving the voice interaction and control effect.

[0116] The technical solutions in the embodiments of the present application are clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application. In the absence of conflict, the following embodiments and their technical features can be combined with each other.

[0117] Figure 3 It is a flowchart of the interaction method of the first embodiment of the present application. Figure 3 The execution subject of the described method can be a terminal, such as a mobile terminal such as a smart phone, a PDA (Personal Digital Assistant, or a tablet computer), and a wearable device with adaptive functions worn on a limb or embedded in clothing, jewelry, or accessories.

[0118] Figure 3 is a flowchart of the interaction method of the first embodiment of the present application. Figure 3 As shown, the interaction method includes the following steps S11 to S13.

[0119] S11. Acquire and identify first operation information.

[0120] S12: Detect whether the first operation information satisfies a first preset condition.

[0121] If yes, execute step S13.

[0122] S13: Obtain and execute second operation information.

[0123] This embodiment controls the execution of a corresponding operation (second operation information) according to first operation information that meets a preset condition, wherein the specific meanings of the first operation information, the first preset condition and the second operation information have different adaptability corresponding to different application scenarios.

[0124] For example, the first operation information includes at least one of the following: operation gesture, operation duration, operation frequency, and operation position. The operation gesture can be understood as an action made by a user's finger or arm, such as the gestures "√" and "×". It should be understood that the operation gesture can be a gesture in the air or on the terminal screen. Specifically, the operation gesture can be classified into at least one of the following: air gesture, operation track, sliding track, click, hard press, long press, etc.

[0125] The main body of the operation duration, operation frequency and operation position can be a gesture action or other operations other than a gesture action. The operation position includes but is not limited to at least one of the following: the position of the gesture action on the terminal screen, the preset distance from the terminal screen, and the position of the person making the gesture action.

[0126] The first preset condition is a basis for judging whether the first operation information can obtain and execute the second operation information. Corresponding to the example of the aforementioned first operation information, the first preset condition includes at least one of the following: the operation gesture is a preset gesture, the operation duration meets the preset duration, the operation frequency reaches the preset frequency, and the operation position is a preset position.

[0127] In the step S12, taking the first operation information as an operation gesture as an example, when it is detected that the operation gesture is a preset gesture, for example, when the gestures of the two are the same, both are the "√" gesture, it is determined that the first operation information meets the first preset condition, otherwise it does not meet the first preset condition.

[0128] The second operation information can be understood as a predetermined operation to be implemented, such as answering or hanging up a call, automatic recording, starting music playback, sending and receiving emails, starting a chat application, etc. For ease of understanding and reflecting the improvement over the prior art, the following takes the second operation information as a voice command to a voice assistant as an example to describe the above embodiments of the present application in detail.

[0129] In a specific scenario, for example, in a scenario where a voice assistant is used to handle an incoming call event, when it is detected that the user's operation gesture matches the preset gesture, it indicates that the verification is passed. In step S13, the voice assistant is allowed to obtain and execute the voice command issued by the user, for example, if the voice command is "answer", the call is answered, or if it is "hang up", the call is hung up. In this regard, this embodiment can control the voice interaction function through the operation gesture to reduce the probability of misoperation.

[0130] If the user's operation gesture does not match the preset gesture, it means that the verification fails. At this time, the user cannot control the incoming call event through the voice assistant. In this regard, this embodiment can perform corresponding operations according to specific scenarios (such as incoming call events). Figure 3 In the described embodiment, steps S11 to S12 are continued until the first operation information satisfies the first preset condition. Figure 4 As shown, in the step S12, when the first operation information does not meet the first preset condition, the interaction method further includes steps S14 to S16.

[0131] S14: If an incoming call is detected, determine whether the current distance between the terminal and the user is less than or equal to a first preset distance.

[0132] If yes, execute step S15; if no, execute step S16.

[0133] S15: Answer the call.

[0134] S16: Hang up the call, and / or start the voice assistant to output a prompt message.

[0135] In step S14, when an incoming call is detected, the first preset distance can be understood as the distance at which the user can clearly hear the voice of the incoming caller and whether his own voice can be clearly transmitted to the incoming caller. That is, when the distance between the terminal and the user is less than or equal to the first preset distance, the voice of the incoming caller can be clearly played to the user through the terminal, and the user's voice can be collected by the terminal and clearly transmitted to the incoming caller. The specific value of the first preset distance can be set by different users according to their own circumstances.

[0136] When the distance between the terminal and the user is greater than the first preset distance, the call can be hung up, or the voice assistant can be started to output a prompt message, or both operations can be performed.

[0137] Among them, the output prompt information is used to prompt the user to answer or hang up the call. Here, after starting the voice assistant to output the prompt information, the voice assistant can receive the voice command issued by the user and answer or hang up the call according to the received voice command.

[0138] This embodiment controls the answering or hanging up of incoming calls according to the distance, so that users can answer calls at a certain distance and avoid missing important information. On the other hand, the voice assistant will consume power to a certain extent and affect the battery life of the terminal. Therefore, when it is turned off, if the user is busy with other things and the mobile phone or other terminal is far away from him, the phone rings and he chooses to automatically answer the call when the distance is appropriate, so that the call can be connected as soon as possible.

[0139] Figure 5 This is a flowchart of the interactive method of the third embodiment of this application. Figure 3 and Figure 5As shown, the step S13 in this embodiment includes two sub-steps S131 and S132. In the step S131, detecting whether the terminal meets the second preset condition can be regarded as a trigger condition for executing the step S132. Only when it is detected that the terminal meets the second preset condition, the received second operation information is obtained and executed.

[0140] Still taking the incoming call event scenario as an example, the second preset condition includes at least one of the following: the terminal detects an incoming call event; the distance between the terminal and the user is greater than or equal to the second preset distance.

[0141] In a specific scenario, for example, the terminal detects an incoming call event. In step S13, the terminal detects whether the incoming call number has been stored in the address book and starts the voice assistant.

[0142] This embodiment performs different operations depending on whether the incoming call number has been pre-recorded in the address book. The address book generally manually stores the numbers of multiple contacts, and these contacts can be grouped, such as "relatives", "friends", "colleagues", "alumni", etc. The number and names of these groups can be customized by the user or can be defaulted by the terminal.

[0143] When a call comes in, if the caller number has been recorded in the address book and the voice assistant has been started, the terminal will make a voice announcement based on the preset information of the caller number in the address book, which includes at least one of the following: caller number, contact name, and group information. Then it will prompt whether to answer the call, and answer or hang up the call according to the received voice command.

[0144] And / or, if the incoming call number is not recorded in the address book, the incoming call number is classified and recorded. Specifically, if the incoming call number is an unfamiliar call that is not recorded in the address book, the terminal can voice broadcast the incoming call number, and then prompt whether to answer the call, and choose whether to answer the call according to the user's voice command. Further, the terminal can classify the incoming call number into a corresponding group according to the user's voice command and set the contact name.

[0145] In one implementation, for example, when the distance between the terminal and the user is greater than or equal to a second preset distance, in the S13 step, the terminal starts the voice assistant and outputs a prompt message to remind the user to perform voice operations, and then performs corresponding operations according to the received voice instructions, such as making a call and / or opening an application.

[0146] The second preset distance can be understood as a critical distance at which the user cannot manually operate the terminal. When the distance between the terminal and the user is greater than or equal to the second preset distance, the user cannot contact the terminal and can perform the operation by calling a voice assistant and combining voice commands.

[0147] Different voice commands issued by the user will result in different operations performed by the terminal. For example, the user issues a voice command of "Please call XXX", and the voice assistant automatically calls the phone number of XXX stored in the address book and makes the call. For another example, the user issues a voice command of "make a call", and the voice assistant prompts that the call interface has been opened, and prompts the user to select the call object. The user speaks the name or phone number of the call contact and other identity information, and the terminal automatically matches and calls. For another example, the user issues a voice command of "open X application", the voice assistant responds and opens the corresponding application. For chat applications, the voice assistant can further prompt the user to select the object to be contacted. After selecting the object, it receives the voice command from the user to select video call or voice call, and executes the chat call.

[0148] After launching the application according to the received voice command, the voice assistant can also assist the user in completing the scheduled event. Specifically, the corresponding application is launched according to the voice command issued by the user. If the control command issued by the user (including but not limited to the control command in the form of voice) is not received within the first predetermined time, a prompt message is output to prompt the user at least once; if the control command is still not received within the second preset time after the prompt message, a frequently used contact is selected for contact according to the properties of the application.

[0149] Taking making a phone call as an example, when the user calls up the address book interface through voice commands, if the user does not respond to select the contact person within the specified time, the terminal will make a voice reminder three times in a row. If there is no response for three times, the terminal will automatically select the frequently used contacts among relatives to initiate an event and intelligently notify the other party of what happened.

[0150] Based on this, the voice assistant of this embodiment can assist users in completing scheduled events. For example, in one scenario, when an elderly person feels unwell at home alone, the first thing he thinks of is to call his children. Once the call event has occurred but the call has not been made, the terminal will remind him. If the number of reminders exceeds, the mobile phone will automatically call the elderly person's children to avoid accidents.

[0151] Figure 6 is a flowchart of an interactive method according to an embodiment of the present application. Figure 6 As shown, the interaction method includes the following steps S41 to S46.

[0152] S41: Acquire and recognize user gesture information.

[0153] Gesture information can be understood as information about the user's arm movements or finger movements.

[0154] In the embodiment of the present application, a camera can be used to capture a user's gesture to obtain an image, and gesture information corresponding to the gesture can be identified through image recognition technology.

[0155] S42: Detect whether the user's gesture information matches the preset gesture information.

[0156] The preset gesture information can be regarded as the switch of the voice-based human-computer interaction function (such as voice assistant). Only when the gesture action is consistent with the preset gesture action, the user's gesture information matches the preset gesture information, and the voice-based human-computer interaction function is turned on.

[0157] It can be seen that the embodiment of the present application is equivalent to adding a function of enabling voice human-computer interaction through gestures to the terminal. In real application scenarios, the implementation of this function includes but is not limited to: taking a smartphone as an example, pre-programming a script or application and installing it in the operating system of the smartphone, such as Figure 7 As shown, a "gesture control voice human-computer interaction" option is added to the setting interface of the smartphone; then, by sliding the slider to turn on or off the option, the function can be turned on or off accordingly. After turning on the option, the smartphone can automatically pop up a dialog box to ask the user to set the preset gesture information.

[0158] The methods for users to set preset gesture information include but are not limited to the following two:

[0159] 1. The user's gesture is captured by a camera to obtain an image, and gesture information corresponding to the gesture is identified by image recognition technology as the preset gesture information.

[0160] 2. A page or dialog box is popped up, displaying images of multiple gesture actions, and the preset gesture information is set according to one or more gesture actions selected by the user.

[0161] In order to distinguish the gestures used in traditional gesture-based human-computer interaction, the embodiments of the present application can be set differently in actual application scenarios. For example, the prior art uses gesture "C" to start the camera to shoot, and uses gesture "M" to start the music player to play music. In order to distinguish from this, the preset gesture information can be: gesture "√" and "×". The gesture "√" is used to turn on the voice-based human-computer interaction function, and accordingly, the gesture "×" is used to turn off the voice human-computer interaction function.

[0162] When it is detected that the user's gesture information matches the preset gesture information, it means that the verification is passed, and the current switch state of the voice human-computer interaction function is allowed to be changed.

[0163] When the detected user gesture information does not match the preset gesture information, it means that the verification fails and the current switch state of the voice human-computer interaction function is not allowed to be changed.

[0164] Among them, the current switch state of the voice human-computer interaction function indicates whether the function is currently turned on or off. When the current switch state is the off state, if the detected user's gesture information matches the preset gesture information, step S43 is executed; if the detected user's gesture information does not match the preset gesture information, step S44 is executed. When the current switch state is the on state, if the detected user's gesture information matches the preset gesture information, the voice human-computer interaction function can be turned off; if the detected user's gesture information does not match the preset gesture information, the voice human-computer interaction function can continue to be kept in the on state.

[0165] S43: Execute corresponding operations according to the voice command issued by the user.

[0166] The corresponding operations include but are not limited to: making calls, automatic recording, starting music playback, email, chat applications (Application, APP), etc.

[0167] S44: When an incoming call is detected, detecting whether the current distance between the user and the user is less than or equal to a first preset threshold.

[0168] The first preset distance can be understood as the distance at which the user can clearly hear the voice of the incoming caller and whether his own voice can be clearly transmitted to the incoming caller. That is, when the distance between the terminal and the user is less than or equal to the first preset distance, the voice of the incoming caller can be clearly played to the user through the terminal, and the user's voice can be collected by the terminal and clearly transmitted to the incoming caller.

[0169] The specific value of the first preset distance can be set by different users according to their own circumstances.

[0170] When the current distance is less than or equal to the first preset threshold, step S45 is executed. When the current distance is greater than the first preset threshold, step S46 is executed.

[0171] S45: Answer the call.

[0172] S46: Hang up the call.

[0173] Based on the above, it can be known that the embodiment of the present application controls the switch of the voice human-computer interaction function through gesture information, and allows the corresponding operation to be executed only when matching gesture information is detected, thereby reducing the probability of misoperation while realizing voice control. Moreover, when the user's gesture information does not match the preset gesture information, that is, when the voice human-computer interaction function is turned off, the terminal will control the answering and hanging up of incoming calls based on the distance between the terminal and the user, which is convenient for user operation.

[0174] That is to say, after turning off the voice human-computer interaction function, on the basis of allowing normal use of the terminal, the embodiment of the present application further controls the answering or hanging up of incoming calls according to the distance, so that users can reasonably answer the phone at a certain distance and avoid missing important information. On the other hand, the voice human-computer interaction function will consume power to a certain extent and affect the battery life of the terminal. Therefore, when this function is turned off, if the user is busy with other things and the mobile phone or other terminal is far away from him, the phone rings at this time, and he can choose to automatically answer the call when the distance is appropriate, so that the call can be connected at the first time.

[0175] When the current distance between the terminal and the user is greater than the first preset threshold, after hanging up the call in step S46, the embodiment of the present application can record the relevant information of the missed call, such as the caller number, the group of the caller contact and the name in the address book, and when it is detected that the user's gesture information matches the preset gesture information, the recorded relevant information of the missed call is voice broadcast. Based on this, the embodiment can inform the user of the current call when the user is far away from the terminal and misses the call, so as to avoid missing important information.

[0176] Of course, another embodiment of the present application may not notify the user of the incoming call information until it is detected that the user's gesture information matches the preset gesture information, but instead directly notify the user of the incoming call information via text message or other means after hanging up the call.

[0177] In step S43, the terminal performs different operations depending on the voice command issued by the user. For example, if the user issues a voice command of "please call XXX", the terminal automatically calls the phone number of XXX stored in the address book and makes a call. For another example, if the user issues a voice command of "make a call", the terminal voice prompts that the call interface has been opened and prompts the user to select the call partner. The user speaks the name or phone number of the call contact and other identity information, and the terminal automatically matches and calls.

[0178] Of course, considering that the sound needs to be within an appropriate distance to be collected and accurately recognized by the terminal, before receiving the user's voice command, the terminal also needs to detect whether the distance between the terminal and the user is less than or equal to the second preset threshold. Only when the distance condition is met, the terminal will perform the corresponding operation according to the voice command issued by the user.

[0179] For different types of operations, the value of the second preset threshold may be different. For example, when making a call, the second preset threshold needs to ensure that the user can clearly hear the voice of the incoming caller and whether his own voice can be clearly transmitted to the incoming caller. At this time, the second preset threshold can be equal to the first preset threshold. For another example, when opening a video image application, the user issues a voice command to call the video application, the terminal responds and opens the corresponding application, and then prompts the user to select the object to be contacted. After the object is selected, the terminal prompts that the object has been determined, and the user then issues a voice command to select a video call or a voice call, and finally completes the event. In this example, the second preset threshold also needs to further ensure that the user can clearly see the image displayed on the terminal. At this time, the second preset threshold can generally be smaller than the first preset threshold.

[0180] See also Figure 8 , Figure 8 1 is a flow chart of a human-computer interaction method according to another embodiment of the present application. The human-computer interaction method according to this embodiment may include the following steps S51 to S56.

[0181] S51: Acquire and recognize user gesture information.

[0182] S52: Detect whether the user's gesture information matches the preset gesture information.

[0183] When it is detected that the user's gesture information matches the preset gesture information, steps S531 to S534 are executed. When it is detected that the user's gesture information does not match the preset gesture information, step S54 is executed.

[0184] S531: When an incoming call is detected, check whether the incoming call number has been recorded in the address book.

[0185] If the incoming call number has been recorded in the address book, steps S532 to S533 are executed.

[0186] If the incoming call number is not recorded in the address book, step S534 is executed.

[0187] S532: Perform voice broadcast according to the preset information of the incoming call number in the address book, and prompt whether to answer the call.

[0188] S533: Answer the incoming call or hang up the incoming call according to the received user instruction.

[0189] S534: Categorize and record the incoming call number.

[0190] S54: When an incoming call is detected, detecting whether the current distance between the user and the user is less than or equal to a first preset threshold.

[0191] When the current distance is less than or equal to the first preset threshold, step S55 is executed. When the current distance is greater than the first preset threshold, step S56 is executed.

[0192] S55: Answer the call.

[0193] S56: Hang up the call.

[0194] Based on the description of the aforementioned embodiment, but different therefrom, for the application scenario of answering a call through voice recognition, this embodiment performs different operations depending on whether the incoming call number has been pre-recorded in the address book.

[0195] Before the call comes in, the address book of the terminal in the embodiment of the present application generally has manually stored the numbers of multiple contacts, and the multiple contacts can be grouped, such as "relatives", "friends", "colleagues", "alumni", etc. The number and names of these groups can be customized by the user or can be provided by the terminal by default.

[0196] When a call comes in, if the caller number has been recorded in the address book, the terminal will make a voice announcement based on the preset information of the caller number in the address book, such as which group the caller number belongs to, the name of the corresponding contact in the address book, etc., and then prompt whether to answer the call, and choose whether to answer the call based on the user's voice command.

[0197] If the incoming call number is an unfamiliar call that is not recorded in the address book, the terminal can voice broadcast the incoming call number, then prompt whether to answer the call, and choose whether to answer the call according to the user's voice command. Further, the terminal can classify the incoming call number into a corresponding group according to the user's voice command and set a name.

[0198] In this embodiment, the terminal can automatically record incoming calls or outgoing calls, including: unfamiliar calls for the first time, frequent incoming calls in a day, missed calls in a day, frequent outgoing calls in a day, etc., and make simple records, for example, the records include: whether the call is connected, the time of the call, the duration of the call, etc. Furthermore, the user is reminded by SMS at an appropriate time.

[0199] Please continue reading Figure 6 In step S43, the voice human-computer interaction function can assist the user to complete the scheduled event. Specifically, the corresponding application is started according to the voice command issued by the user, and if the command issued by the user is not received within the predetermined time, the user is prompted at least once; if the user's command is still not received after at least one prompt, a common contact is selected for contact according to the attributes of the application.

[0200] Take making a phone call as an example. After the user calls the address book interface through voice commands, if the user does not respond to the contact person within the specified time, the terminal will remind the user by voice for three consecutive times. If there is no response for three times, the terminal will automatically select the frequently used contacts among relatives to initiate the event and intelligently notify the other party of what happened. Based on this, when the elderly feel unwell at home alone, they will first think of calling their children. Once the phone event has occurred and the call has not been made, the terminal will remind them. After the reminder times exceed, the mobile phone will automatically call the elderly's children to avoid accidents.

[0201] Fig. 9 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Fig. 9 As shown, the terminal 70 can be regarded as the aforementioned electronic device, including a processor 71 and a memory 72 , and the two can be connected via a communication bus 73 to achieve data or signal transmission.

[0202] The processor 71 is the control center of the terminal 70. It uses various interfaces and lines to connect the various parts of the entire terminal 70. It executes various functions of the terminal 70 and processes data by running or loading programs stored in the memory 72 and calling data stored in the memory 72, thereby monitoring the terminal 70 as a whole.

[0203] The processor 71 will load instructions corresponding to the processes of one or more programs into the memory 72 according to predetermined steps, and the processor 71 will run the programs stored in the memory 72 to implement the steps in the method of any of the above embodiments.

[0204] For the specific implementation method of each step, that is, the specific content of the steps executed by the processor 71 calling the program, please refer to the aforementioned embodiment, which will not be described one by one here.

[0205] It should be understood that, when implemented in an actual application scenario, depending on the device type to which the terminal 70 belongs, the execution body of the above steps may not be the processor 71 and the memory 72, but may be implemented by other modules and units respectively.

[0206] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling related hardware through instructions, and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present application provides a readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded and executed by a processor to implement the steps in any of the interactive methods provided in the embodiments of the present application.

[0207] The storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0208] Since the instructions stored in the storage medium can execute the steps in any interaction method provided in the embodiments of the present application, the beneficial effects that can be achieved by any interaction method can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0209] The present application also provides a device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the method described above when executed by the processor.

[0210] The present application also provides a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0211] An embodiment of the present application further provides a computer program product, which includes a computer program code. When the computer program code is executed on a computer, the computer executes the methods described in the various possible embodiments above.

[0212] An embodiment of the present application also provides a chip, including a memory and a processor, wherein the memory is used to store computer programs, and the processor is used to call and run the computer programs from the memory, so that a device equipped with the chip executes the methods in the various possible embodiments as described above.

[0213] It should be noted that the terms "include", "comprises" or any other variants thereof herein are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0214] In addition, although the terms "first, second, third" and the like are used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information, depending on the context. The term "if" may be interpreted as "at the time of" or "when" or "in response to determination". Furthermore, as used herein, the singular forms "one", "an" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. The terms "or" and "and / or" are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Only when the combination of elements, functions, steps or operations is inherently mutually exclusive in some way, will there be an exception to this definition.

[0215] Furthermore, although the steps in the flowcharts herein are shown in sequence as indicated by the arrows, they are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the steps are not strictly executed in sequence and may be executed in other sequences. Moreover, at least a portion of the steps in the diagram may include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but may be executed at different times, and their execution order is not necessarily sequential, but may be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0216] It should be noted that, in this article, step codes such as S11, S12, etc. are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the sequence. When implementing it specifically, those skilled in the art may execute S12 first and then S11, etc., but these should all be within the scope of protection of this application.

[0217] Although the present application has been shown and described with respect to one or more implementations, those skilled in the art will think of equivalent variations and modifications based on reading and understanding of this specification and the drawings. The present application includes all such modifications and variations, and is supported by the technical solutions of the aforementioned embodiments. That is, the above is only a partial embodiment of the present application, and does not limit the patent scope of the present application. All equivalent structural transformations made using the contents of this specification and the drawings, such as the combination of technical features between the embodiments, or direct or indirect application in other related technical fields, are similarly included in the patent protection scope of the present application.

Claims

1. An interaction method, applied to a terminal, characterized in that: The method comprises: S11, acquiring and identifying first operation information; S12, detecting whether the first operation information meets a first preset condition; S13. If so, detect whether the terminal satisfies a second preset condition, the second preset condition including that the distance between the terminal and the user is greater than or equal to the second preset distance, or the distance between the terminal and the user is greater than or equal to the second preset distance and the terminal detects an incoming call event; and, upon detecting that the terminal satisfies the second preset condition, obtain and execute second operation information, including: starting a voice assistant, and outputting a prompt message to remind the user to perform voice operation, and making a call and / or opening an application according to the received voice command, and, if no control command is received within a first preset time, outputting a prompt message; if no control command is received within a second preset time after the prompt message, selecting a frequently used contact for contact according to the attributes of the application.

2. The method according to claim 1, characterized in that If the first operation information does not meet the first preset condition, the method further includes: If an incoming call is detected, determining whether the current distance between the terminal and the user is less than or equal to a first preset distance; If yes, answer the call; and / or, If not, hang up the call and / or start the voice assistant to output a prompt message.

3. The method according to claim 2, characterized in that After the step of outputting the prompt information, the method further includes: Hang up or answer the call according to the received voice command.

4. The method according to claim 1, characterized in that: The second preset condition also includes: the terminal detects an incoming call event.

5. The method according to claim 4, characterized in that When the terminal detects an incoming call event, the step S13 further includes: Check whether the incoming call number is stored in the address book and start the voice assistant; If yes, then the incoming call number is voice broadcasted according to the preset information, and the incoming call is answered or hung up according to the received voice command; and / or, If not, the incoming call number is classified and recorded.

6. The method according to any one of claims 1 to 5, characterized in that The first operation information includes at least one of the following: operation gesture, operation duration, operation frequency, and operation position; and / or, The first preset condition includes at least one of the following: The operation gesture is a preset gesture; The operation duration meets the preset duration; The operating frequency reaches a preset frequency; The operating position is a preset position.

7. A terminal, characterized in that: The terminal includes: a processor and a memory; The memory stores computer-executable instructions; When the computer-executable instructions are executed by the processor, the interactive method according to any one of claims 1 to 6 is implemented.

8. A readable storage medium, characterized in that: The readable storage medium stores computer-executable instructions, which are used to implement the interaction method according to any one of claims 1 to 6 when executed by a processor.

Images