Human-computer interaction method, terminal equipment, readable storage medium and chip

Abstract

The invention provides a man-machine interaction method, terminal equipment, a readable storage medium and a chip, and relates to the technical field of artificial intelligence AI. The method is applied to terminal equipment provided with an AI assistant, and specifically comprises the steps that a first interface is displayed, and the first interface comprises a target object; in response to a dragging operation on the target object, starting an AI assistant, and displaying a first AI control at the bottom of the first interface; and after the target object is dragged to the area where the first AI control is located and the dragging operation is released, controlling an AI assistant to store the target object. In the embodiment of the application, the AI assistant can provide richer man-machine interaction modes, including helping the user to store the target object for subsequent use by the user, so that better user experience is achieved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a human-computer interaction method, a terminal device, a readable storage medium and a chip. Background Art

[0002] With the development of computer technology, AI assistants such as "Xiaoyi", "YOYO", and "Siri" have gradually appeared on mobile phones, tablet computers and other terminal devices. Users can interact with terminal devices through AI assistants. For example, users can control terminal devices to make calls, play music, or plan navigation through AI assistants. However, the current AI assistants mainly interact with users in the form of dialogue, and the form of interaction is relatively single, which cannot meet the diverse interaction needs of users. Summary of the invention

[0003] The present application provides a human-computer interaction method, a terminal device, a readable storage medium and a chip, which are used to solve the problem that the interaction method between AI assistants and users in the prior art is relatively single and cannot meet the diverse interaction needs of users.

[0004] In a first aspect, an embodiment of the present application provides a human-computer interaction method, which is applied to a terminal device provided with an AI assistant, and the method includes: displaying a first interface, the first interface including a target object; in response to a drag operation on the target object, starting the AI ​​assistant and displaying a first AI control at the bottom of the first interface; after the target object is dragged to the area where the first AI control is located and the drag operation is released, controlling the AI ​​assistant to store the target object.

[0005] Exemplarily, the AI ​​assistant may be "Xiaoyi", "YOYO" or "Siri", etc. The AI ​​assistant can determine the user intent based on the interaction with the user, and perform corresponding actions based on the determined user intent.

[0006] Through the method provided in the embodiment of the present application, the terminal device can help the user store the target object through the AI ​​assistant for subsequent use by the user. Compared with the existing AI assistant, the AI ​​assistant provided in this embodiment can provide a richer human-computer interaction mode and a better user experience.

[0007] In some embodiments, after the AI ​​assistant is started, a second AI control is also displayed at the bottom of the first interface; the second AI control is used to control the AI ​​assistant to communicate with the user based on the target object after the target object is dragged to the area where the second AI control is located and the drag operation is released.

[0008] Through the method provided in the embodiment of the present application, the terminal device can conduct human-computer interaction with the user based on the target object through the AI ​​assistant, and the interactive content is more specific, detailed and clear, with a better user experience. For example, the user can discuss topics with the AI ​​assistant based on the target object, or control the AI ​​assistant to preprocess the target object. The preprocessing can be image recognition, similar picture search, etc.

[0009] In addition, in this embodiment, the user can control the AI ​​assistant to implement two different AI interaction functions, "talk to AI" and "let AI keep" based on the target object through a control operation, that is, dragging the target object to the target control (i.e., the first AI control or the second AI control). Not only is the control operation simple, but it can also better enrich and cover various scenarios of interaction between people and AI assistants.

[0010] In some embodiments, while the target object is being dragged, the method further includes: while the target object is approaching the first AI control or the second AI control, reducing the display size of the target object.

[0011] Since the target object displayed in the first interface is usually large in size, reducing the size of the target object can accurately drag it to the area where the first AI control or the second AI control is located, thereby improving control accuracy.

[0012] In some embodiments, while the target object is being dragged, the method further includes: determining the positional relationship between the dragged target object and the first AI control and the second AI control; when the dragged target object is near the first AI control, increasing the display size of the first AI control and reducing the display size of the second AI control; or, when the dragged target object is near the second AI control, increasing the display size of the second AI control and reducing the display size of the first AI control.

[0013] Through this method, the user can accurately drag the target object to the area where the target control (ie, the first AI control or the second AI control) is located, thereby improving control accuracy and user experience.

[0014] In some embodiments, the target object includes at least one of a picture, text, file, audio or video, and a web link.

[0015] In a second aspect, an embodiment of the present application provides a human-computer interaction method, which is applied to a terminal device provided with an AI assistant, and the method includes: displaying a second interface; in response to an operation of starting the AI ​​assistant, displaying an object stored by the AI ​​assistant on the upper layer of the second interface; in response to an operation of dragging a target object to the second interface, adding the target object to the second interface; wherein the target object is any one or more objects stored by the AI ​​assistant.

[0016] Through the method provided in this embodiment, for the content stored by the user in the AI ​​assistant, the user can quickly drag and use the target object from the AI ​​assistant as needed. For example, the user can drag the previously stored picture A from the AI ​​assistant to the dialog box of the chat application to quickly send the picture A to a friend.

[0017] In some embodiments, in response to the operation of starting the AI ​​assistant, objects stored by the AI ​​assistant are displayed on the upper layer of the second interface, including: in response to the operation of starting the AI ​​assistant, an AI comprehensive interface is displayed on the upper layer of the second interface, and the AI ​​comprehensive interface includes AI dialogue controls and objects stored by the AI ​​assistant.

[0018] In this embodiment, the AI ​​conversation function and the AI ​​storage function are configured with the same startup entry. In other words, the user can simultaneously start two different AI functions through one control operation.

[0019] In some embodiments, in response to the operation of starting the AI ​​assistant, objects stored by the AI ​​assistant are displayed on the upper layer of the second interface, including: in response to the operation of starting the AI ​​assistant, an AI dialogue interface is displayed on the upper layer of the second interface, and the AI ​​dialogue interface includes AI dialogue controls and AI auxiliary controls; in response to the operation of the AI ​​auxiliary control, an AI storage interface is displayed on the upper layer of the second interface, and the storage interface includes objects stored by the AI ​​assistant.

[0020] In this embodiment, the startup entry of the AI ​​storage function is embedded in the AI ​​dialogue function. The terminal device needs to start the AI ​​dialogue function first to display the AI ​​dialogue interface, and further control the display of the objects stored by the AI ​​assistant in the AI ​​dialogue interface. The use frequency of the AI ​​dialogue function is usually higher than that of the AI ​​storage function. Therefore, through the method provided by this embodiment, the terminal device can give priority to starting the AI ​​dialogue function commonly used by the user, ensuring that the display interface corresponding to the commonly used function is simple.

[0021] In some embodiments, the operation of starting the AI ​​assistant includes: the operation of waking up the AI ​​assistant according to a wake-up word; or, the control operation of the AI ​​startup control; or, the pressing operation of a preset physical button.

[0022] In a third aspect, an embodiment of the present application provides a human-computer interaction method, which is applied to a terminal device provided with an AI assistant, and the method includes: displaying a second interface; in response to a first control operation at the bottom of the second interface, displaying an object stored by the AI ​​assistant on the upper layer of the second interface; in response to an operation of dragging a target object to the second interface, adding the target object to the second interface; wherein the target object is any one or more objects stored by the AI ​​assistant.

[0023] In some embodiments, the method further includes: in response to a second control operation at the bottom of the second interface, displaying an AI dialogue interface on an upper layer of the second interface; wherein the first control operation and the second control operation are different.

[0024] In this embodiment, the terminal device sets different startup methods for the AI ​​storage function and the AI ​​dialogue function. Through this method, the user can quickly start the target AI function according to the needs without starting other AI functions. For example, when it is necessary to talk to the AI ​​assistant, only the AI ​​dialogue function is started, and the AI ​​storage function is not started; or, when it is necessary to retrieve the target object from the AI ​​assistant, only the AI ​​storage function is started, and the AI ​​dialogue function is not started.

[0025] In some embodiments, the first control operation is: a control operation acting on a first control area at the bottom of the second interface; the second control operation is: a control operation acting on a second control area at the bottom of the second interface; wherein the first control area and the second control area do not overlap.

[0026] For example, a prompt bar is displayed at the bottom of the second interface, the first control area is the area where the prompt bar is located, and the second control area is the left area and / or right area of ​​the prompt bar. Alternatively, the first control area is the left area of ​​the prompt bar, and the second control area is the right area of ​​the prompt bar. Alternatively, the first control area is the right area of ​​the prompt bar, and the second control area is the left area of ​​the prompt bar.

[0027] In a fourth aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method shown in the first, second or third aspect above is implemented.

[0028] In a fifth aspect, an embodiment of the present application provides a human-computer interaction device, which includes at least one module, and the at least one module is used to execute the method shown in the first aspect, the second aspect or the third aspect above.

[0029] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method shown in the first aspect, the second aspect or the third aspect above.

[0030] In the seventh aspect, an embodiment of the present application provides a chip, which includes a processor and a memory, in which a computer program is stored. When the computer program is executed by the processor, the method shown in the first aspect, the second aspect or the third aspect is implemented.

[0031] In an eighth aspect, an embodiment of the present application provides a computer program product, in which a computer program is stored. When the computer program is executed by a processor, the method shown in the first aspect, the second aspect or the third aspect can be implemented.

[0032] It can be understood that the beneficial effects of the fourth to eighth aspects mentioned above can be found in the relevant descriptions in the first, second or third aspects mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a schematic diagram of a scenario for waking up an AI assistant provided in an embodiment of the present application;

[0034] Figure 2 This is a schematic diagram of a scenario in which a user controls a terminal device through an AI assistant provided in an embodiment of the present application;

[0035] Figure 3 is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0036] Figure 4 is a schematic flow chart of a human-computer interaction method provided by an embodiment of the present application;

[0037] Figure 5 is a schematic diagram of a first interface provided in an embodiment of the present application;

[0038] Figure 6 is a schematic diagram of a first AI control and a second AI control provided in an embodiment of the present application;

[0039] Figure 7 is a schematic diagram of an AI assistant storing a target object according to a user operation provided by an embodiment of the present application;

[0040] Figure 8 is a schematic diagram of a conversation between an AI assistant and a user based on a target object provided in an embodiment of the present application;

[0041] Fig. 9 is a schematic flow chart of a human-computer interaction method provided by another embodiment of the present application;

[0042] Fig.10 is a schematic diagram of an AI assistant outputting and using a target object according to a user operation provided by an embodiment of the present application;

[0043] Fig.11 is a schematic flow chart of a human-computer interaction method provided by another embodiment of the present application;

[0044] Fig.12 is a schematic diagram of an AI assistant outputting and using a target object according to a user operation provided by another embodiment of the present application;

[0045] Fig.13 is a schematic diagram of a first control area and a second control area provided in an embodiment of the present application;

[0046] Fig.14 is a schematic flow chart of a human-computer interaction method provided by yet another embodiment of the present application;

[0047] Fig.15 is a schematic diagram of an AI assistant provided in an embodiment of the present application outputting and using picture A according to user operation;

[0048] Fig.16 This is a schematic diagram of the AI ​​assistant provided in an embodiment of the present application starting an AI dialogue function according to a user operation display;

[0049] Fig.17 is a schematic diagram of an AI assistant provided in an embodiment of the present application intelligently classifying storage objects;

[0050] Fig.18 is a schematic diagram of an AI storage interface provided in an embodiment of the present application;

[0051] Figures 19A to 19C is a schematic diagram of the AI ​​assistant preprocessing a target object provided by different embodiments of the present application;

[0052] Fig. 20 is a schematic diagram of a human-computer interaction device provided in an embodiment of the present application;

[0053] Fig.21 It is a schematic diagram of the structure of the chip provided in the embodiment of the present application. DETAILED DESCRIPTION

[0054] It should be understood that in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a way to describe the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0055] In this embodiment, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0056] The technical solution provided in the embodiments of the present application is described below with reference to the accompanying drawings.

[0057] With the development of computer technology, AI assistants such as "Xiaoyi", "YOYO" and "Siri" have gradually appeared on mobile phones, tablet computers and other terminal devices to assist users in human-computer interaction.

[0058] In this embodiment, the AI ​​assistant may also be referred to as a voice assistant, intelligent voice, digital assistant, virtual assistant, etc. It can be understood as an information processing system that can recognize natural language input in the form of voice and / or text to infer user intent, and perform corresponding actions based on the inferred user intent. In addition, the system can respond to user input through audible (such as voice) and visual (such as text, pictures) output forms.

[0059] In order to save power consumption of terminal devices, AI assistants are usually in a dormant state. Users can start AI assistants through specific control operations. Starting AI assistants can also be called waking up AI assistants. For example Figure 1 As shown, the user can start the AI ​​assistant by saying the wake-up words such as "Xiaoyi Xiaoyi" or "Hello YOYO"; or by clicking the AI ​​assistant icon to start the AI ​​assistant; or by long pressing the power button to start the AI ​​assistant. This embodiment does not limit the way to start the AI ​​assistant.

[0060] The AI ​​assistant can interact with the user after it is started. For example Figure 2 As shown in the figure, the AI ​​assistant can control the terminal device to play song A according to the voice input of the user "play song A". Or, according to the voice input of the user "navigate home", the route home can be planned and navigation can be performed. Or, according to the voice input of the user "call Lucy", the contact Lucy's phone number can be called. It can be seen that the AI ​​assistant significantly improves the convenience of user interaction with terminal devices and brings users a new human-computer interaction experience.

[0061] At present, AI assistants usually directly communicate with users, and the forms of dialogue include: voice dialogue and text dialogue. That is, the AI ​​assistant receives the voice or text input by the user, and immediately responds with the corresponding voice, text or control. However, as the services of terminal devices gradually become richer, users may have other needs in addition to the need to communicate with AI assistants. For example, in order to synchronize the context and knowledge background of each other with chat friends and enhance communication, users may need to share some content with chat friends. Therefore, users may also have the need to let AI assistants temporarily store text, pictures, files, audio and video, and web links waiting for the use of content; in addition, users may also have the need to communicate with AI assistants based on specific content (such as pictures, text, audio). It can be seen that the current form of human-computer interaction between AI assistants and users is relatively single. For example, some objects that do not need to be processed immediately by AI assistants do not have a carrying position, which cannot meet the diverse human-computer interaction needs of users.

[0062] To this end, an embodiment of the present application provides a human-computer interaction method, which is applied to a terminal device. Through this method, the AI ​​assistant in the terminal device can provide a variety of human-computer interaction methods to meet the user's diverse interaction needs, such as helping users to store content, and having conversations with users based on specific content, etc., and has a better user experience.

[0063] In the embodiment of the present application, the terminal device may be a mobile phone, a tablet computer, a computer with wireless transceiver function, a smart TV, a projector, a wearable device (such as a smart watch), a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and a vehicle computer and other electronic devices equipped with an AI assistant. The embodiment of the present application does not limit the specific type of the terminal device.

[0064] Figure 3 3 is a schematic diagram of the structure of the terminal device 300 provided in the embodiment of the present application. Figure 3As shown, the terminal device 300 includes a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, an earphone interface 370D, a sensor module 380, a button 390, a motor 391, an indicator 392, a camera 393, a display screen 394, and a subscriber identification module (SIM) card interface 395, etc.

[0065] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 300. In other embodiments of the present application, the terminal device 300 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0066] For example, when the terminal device 300 is a mobile phone or a tablet computer, it may include all the components shown in the figure, or may include only some of the components shown in the figure.

[0067] As an example, when the terminal device 300 is a large-screen device, it can include the processor 310 shown in the figure, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, a camera 393, and a display screen 394.

[0068] The processor 310 may include one or more processing units, for example, the processor 310 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated into one or more processors. Among them, the controller may be the nerve center and command center of the terminal device 300. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0069] The external memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 200. The external memory card communicates with the processor 310 through the external memory interface 320 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0070] The internal memory 321 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 310 executes various functional applications and data processing of the electronic device 200 by running the instructions stored in the internal memory 321. The internal memory 321 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and an application required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the electronic device 200 (such as audio data, a phone book, etc.).

[0071] The USB interface 330 is an interface that complies with the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 330 can be used to connect a charger to charge the electronic device 200, and can also be used to transfer data between the electronic device 200 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0072] The charging management module 340 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 340 can receive charging input from the wired charger through the USB interface 330. In some wireless charging embodiments, the charging management module 340 can receive wireless charging input through the wireless charging coil of the terminal device 300. While the charging management module 340 is charging the battery 342, it can also power the electronic device through the power management module 341.

[0073] The power management module 341 is used to connect the battery 342, the charging management module 340 and the processor 310. The power management module 341 receives input from the battery 342 and / or the charging management module 340, and supplies power to the processor 310, the internal memory 321, the external memory, the display screen 394, the camera 393, and the wireless communication module 360. The power management module 341 can also be used to monitor parameters such as battery capacity, battery cycle number, and battery health status (leakage, impedance).

[0074] The wireless communication function of the terminal device 300 can be implemented through antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, modem processor and baseband processor.

[0075] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 300 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0076] The mobile communication module 350 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the terminal device 300. The mobile communication module 350 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 350 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 350 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1.

[0077] In some embodiments, at least some functional modules of the mobile communication module 350 may be disposed in the processor 310. In some embodiments, at least some functional modules of the mobile communication module 350 and at least some modules of the processor 310 may be disposed in the same device.

[0078] The wireless communication module 360 ​​can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the terminal device 300. The wireless communication module 360 ​​can be one or more devices integrating at least one communication processing module. The wireless communication module 360 ​​receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 310. The wireless communication module 360 ​​can also receive the signal to be sent from the processor 310, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0079] The terminal device 300 implements the display function through a GPU, a display screen 394, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 310 may include one or more GPUs, which execute program instructions to generate or change display information.

[0080] The display screen 394 is used to display images, videos, etc. The display screen 394 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 300 may include 1 or N display screens 394, where N is a positive integer greater than 1.

[0081] The terminal device 300 can implement audio functions through an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, an earphone interface 370D, and an application processor.

[0082] The audio module 370 is used to convert digital audio signals into analog audio signals for output, and is also used to convert analog audio inputs into digital audio signals. The audio module 370 can also be used to encode and decode audio signals. In some embodiments, the audio module 370 can be arranged in the processor 310, or some functional modules of the audio module 370 can be arranged in the processor 310.

[0083] The speaker 370A, also called a "speaker", is used to convert audio electrical signals into sound signals. The terminal device 300 can listen to music or listen to hands-free calls through the speaker 370A. For example, the speaker can play the comparison analysis results provided in the embodiment of the present application.

[0084] The receiver 370B, also called a "handset", is used to convert audio electrical signals into sound signals. When the terminal device 300 receives a call or voice message, the voice can be received by placing the receiver 370B close to the human ear.

[0085] Microphone 370C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by putting his mouth close to the microphone 370C to input the sound signal into the microphone 370C. The terminal device 300 can be provided with at least one microphone 370C. In other embodiments, the terminal device 300 can be provided with two microphones 370C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the terminal device 300 can also be provided with three, four or more microphones 370C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0086] The earphone interface 370D is used to connect a wired earphone and can be a USB interface 330 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0087] The sensor module 380 may include a pressure sensor 280A, a gyroscope sensor 280B, an air pressure sensor 280C, a magnetic sensor 280D, an acceleration sensor 280E, a distance sensor 280F, a proximity light sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, a bone conduction sensor 280M, etc.

[0088] Among them, the pressure sensor 280A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 280A can be set on the display screen 394. There are many types of pressure sensors 280A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 280A, the capacitance between the electrodes changes. The terminal device 300 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 394, the terminal device 300 detects the intensity of the touch operation based on the pressure sensor 280A. The terminal device 300 can also calculate the position of the touch based on the detection signal of the pressure sensor 280A.

[0089] In some embodiments, touch operations acting on the same touch position but with different touch operation strengths may correspond to different operation instructions. For example, when a touch operation with a touch operation strength less than a first pressure threshold acts on a short message application icon, an instruction to view a short message is executed. When a touch operation with a touch operation strength greater than or equal to the first pressure threshold acts on a short message application icon, an instruction to create a new short message is executed.

[0090] The touch sensor 280K is also called a "touch panel". The touch sensor 280K can be set on the display screen 394, and the touch sensor 280K and the display screen 394 form a touch screen, also called a "touch screen". The touch sensor 280K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 394. In other embodiments, the touch sensor 280K can also be set on the surface of the terminal device 300, which is different from the position of the display screen 394.

[0091] The key 390 includes a power key, a volume key, etc. The key 390 may be a mechanical key or a touch key. The terminal device 300 may receive key input and generate key signal input related to user settings and function control of the terminal device 300.

[0092] Motor 391 can generate vibration prompts. Motor 391 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 394, motor 391 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0093] Indicator 392 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0094] The SIM card interface 395 is used to connect a SIM card. The SIM card can be connected to or disconnected from the terminal device 300 by inserting the SIM card interface 395 or pulling the SIM card interface 395 out. The terminal device 300 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 395 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0095] Based on the above terminal device, the human-computer interaction method provided by this embodiment is specifically described below.

[0096] In this embodiment, in addition to being able to have a basic conversation with the user based on the user's natural language input, the AI ​​assistant can also perform the following operations: (a) The AI ​​assistant receives and processes the content input by the user. For example, it helps the user store the content, or has a conversation with the user based on the content. (b) The AI ​​assistant outputs the content stored by the user. In other words, when the user needs to use the content stored by the AI ​​assistant, he can quickly retrieve and use the content from the AI ​​assistant. The following are detailed descriptions of each.

[0097] (I) AI assistant receives and processes user input

[0098] Figure 4 It is a schematic flowchart of a human-computer interaction method provided by an embodiment of the present application, which specifically includes steps S401 to S404.

[0099] S401, the terminal device displays a first interface, where the first interface includes a target object.

[0100] In this embodiment, the first interface includes various web pages and application interfaces, such as a browser interface, a gallery interface, a contact interface, a text message interface, an email interface, a word document interface, a memo interface, a chat interface, a desktop, etc. The target object can be text, pictures, files, audio and video, web links, etc. in the first interface, and this embodiment does not limit this.

[0101] When the terminal device displays the first interface, the user's interaction needs with the AI ​​assistant may include: (1) asking the AI ​​assistant to temporarily keep the target object; (2) discussing topics with the AI ​​assistant based on the target object.

[0102] The first interface is Figure 5 Take the gallery interface shown as an example, which displays various pictures and videos stored by the user. In some scenarios, the user needs to discuss a topic with the AI ​​assistant based on a picture in the interface, such as asking the AI ​​assistant where picture A was taken. In other scenarios, the user needs the AI ​​assistant to temporarily keep some pictures so that they can be added to other application interfaces later. For example, the user needs to temporarily keep picture A in the gallery interface to the AI ​​assistant, and when switching to the memo interface, quickly obtain picture A from the AI ​​assistant and add it to the memo interface.

[0103] The implementation process of the above interaction requirements (1) and (2) is described in detail below.

[0104] S402, in response to the user's dragging operation on the target object, the terminal device starts the AI ​​assistant and displays the first AI control and / or the second AI control at the bottom of the first interface.

[0105] Since the AI ​​assistant is usually in a dormant state, the terminal device needs to respond to a specific control operation to start the AI ​​assistant. Exemplarily, in this embodiment, the terminal device starts the AI ​​assistant in response to the user's drag operation on the target object in the first interface.

[0106] After the AI ​​assistant is started, the terminal device displays the first AI control and / or the second AI control at the bottom of the first interface. Among them, the first AI control is used to control the AI ​​assistant to temporarily store the target object so that the user can quickly use the target object later. The second AI control is used to control the AI ​​assistant to communicate with the user based on the target object. The user can choose to drag the target object to the first AI control or the second AI control according to actual needs, so as to control the AI ​​assistant to implement the corresponding function.

[0107] See also Figure 6As shown, taking the first interface as the gallery interface and the target object as picture A as an example, in response to the user's drag operation on picture A, the terminal device starts the AI ​​assistant and displays the first AI control "Let AI keep" and the second AI control "Talk to AI" at the bottom of the first interface. If the user's requirement is to let the AI ​​assistant temporarily store picture A, drag picture A to the area where the first AI control "Let AI keep" is located, and release the drag operation. If the user's requirement is to have a conversation with the AI ​​assistant based on picture A, drag picture A to the area where the second AI control "Talk to AI" is located, and release the drag operation.

[0108] In this embodiment, the drag operation can be interpreted as: the user selects the target object by mouse click, touch or other control methods, and controls the process of moving the target object on the display interface. The drag operation can visualize the user behavior, improve the user experience, and make the human-computer interaction process more natural. Accordingly, the release of the drag operation can be interpreted as: the user stops performing the drag operation. After the drag operation is released at a certain position on the display interface, the target object selected by the user is also released at that position.

[0109] S403, after the target object is dragged to the area where the first AI control is located and the drag operation is released, the terminal device controls the AI ​​assistant to store the target object.

[0110] In this embodiment, in addition to being able to control the movement of the display position of the target object, the drag operation is also extended to define a copy function, that is, after the target object is dragged to the area where the first AI control (such as the "Let AI Keep" control) is located and the drag operation is released, the target object is copied and stored in the AI ​​assistant, and the target object is retained in the original position.

[0111] Optionally, the AI ​​assistant enters a dormant state after storing the target object. Accordingly, the terminal device returns to displaying the first interface without displaying the first AI control, the second AI control, and other contents.

[0112] See also Figure 7 As shown in (a) to (c) in the figure, taking the first interface as the gallery interface and the target object as picture A as an example, in response to the user dragging picture A to the area where the "Let AI keep" control is located and releasing the drag operation, the AI ​​assistant stores picture A. After picture A is stored, the terminal device resumes displaying the first interface.

[0113] Through step S403, the terminal device can temporarily store the target object through the AI ​​assistant. Compared with the existing AI assistants, the AI ​​assistant provided in this embodiment can provide richer human-computer interaction methods and have a better user experience.

[0114] S404, after the target object is dragged to the area where the second AI control is located and the drag operation is released, the terminal device controls the AI ​​assistant to communicate with the user according to the target object.

[0115] In this embodiment, after the target object is dragged to the area where the second AI control (such as the "Talk to AI" control) is located and the drag operation is released, the terminal device displays an AI dialogue interface, which includes an AI dialogue control and a thumbnail of the target object. Based on this, the user can trigger the dialogue function of the AI ​​assistant through the AI ​​dialogue control and have a dialogue with the AI ​​assistant for the target object.

[0116] See also Figure 8 As shown in (a) to (c) in the figure, taking the first interface as the gallery interface and the target object as picture A as an example, in response to the user dragging picture A to the area where the "talk to AI" control is located and releasing the drag operation, the terminal device displays the AI ​​dialogue interface, which includes the AI ​​dialogue control (such as "voice ball") and the thumbnail of picture A. The user can trigger the dialogue function of the AI ​​assistant through the "voice ball" control and have a dialogue with the AI ​​assistant based on the picture A.

[0117] For example, the user can instruct the AI ​​assistant to recognize the content in picture A through voice interaction; or, instruct the AI ​​assistant to perform specific processing on picture A, such as recognizing text in picture A, adjusting the contrast of picture A, etc.; or, inquire about the basic information of picture A, such as the size of the memory occupied, the picture format, the shooting time, the shooting location, etc.

[0118] It should be noted that, in this embodiment, the dialogue between the AI ​​assistant and the user based on the target object should not be narrowly understood as voice communication, but may also include controlling the AI ​​assistant to process the target object through preset controls.

[0119] For example Figure 8 As shown in (c) in the figure, the AI ​​dialogue interface also includes some preset controls, such as "Smart Image Recognition" and "Similar Images". Among them, "Smart Image Recognition" is used to control the AI ​​assistant to recognize and process the information in the image, such as text extraction, text translation, table extraction, QR code recognition, object recognition, etc. for the image. "Similar Images" is used to control the AI ​​assistant to find other images similar to the image locally, on a cloud server, or on a third-party server.

[0120] Through step S404, the terminal device can perform human-computer interaction with the user based on the target object through the AI ​​assistant, and the interaction content is more specific, detailed and clear, with a better user experience.

[0121] Optionally, while the target object is moving toward the first AI control or the second AI control according to the drag operation, the terminal device may also dynamically adjust the shape of the target object, the first AI control, and the second AI control to present better visual effects and control effects to the user and improve the user experience.

[0122] In some embodiments, see Figure 7 (b) or Figure 8 As shown in (b), when the target object (e.g., picture A) moves toward the first AI widget or the second AI widget according to the drag operation, its size can gradually decrease to a preset size, so that the target object can be accurately dragged to the area where the first AI widget or the second AI widget is located. This embodiment does not limit the specific value of the preset size.

[0123] In other embodiments, see Figure 7 (b) or Figure 8 As shown in (b) in the figure, the terminal device can also detect the relative position relationship between the target object and the first AI control and the second AI control. When the target object is near the first AI control, the terminal device increases the display size of the first AI control and reduces the display size of the second AI control. Alternatively, when the target object is near the second AI control, the terminal device increases the display size of the second AI control and reduces the display size of the first AI control. Through this method, the user can accurately drag the target object to the area where the target control (i.e., the first AI control or the second AI control) is located.

[0124] To sum up, through the human-computer interaction method provided by this application, the user can control the AI ​​assistant to implement two different AI interaction functions of "talking to AI" and "letting AI keep" based on the target object through a control operation, that is, dragging the target object to the target control. Not only is the control operation simple, but it can also better enrich and cover various scenarios of interaction between people and AI assistants.

[0125] (II) AI assistant outputs user stored content

[0126] For the content stored by the user in the AI ​​assistant, the user can quickly drag and drop the target object from the AI ​​assistant according to the needs. For example, the user can drag and drop the previously stored picture A from the AI ​​assistant to the dialog box of the chat application to quickly send the picture A to a friend.

[0127] Since the AI ​​assistant is configured with different functions such as AI dialogue and AI storage, the terminal device may set different forms of startup entrances for these functions. The following is an exemplary description of the process of users calling and using the target object from the AI ​​assistant in combination with different startup entrance configurations.

[0128] (1) AI storage and AI dialogue functions are configured with the same startup entry

[0129] Fig. 9 is a schematic flow chart of a human-computer interaction method provided by another embodiment of the present application. Fig. 9 As shown, the method involves a process in which a terminal device drags out and uses a target object from an AI assistant according to a user operation, specifically including steps S901 to S903.

[0130] S901, the terminal device displays a second interface, wherein the second interface is the same as or different from the first interface shown above.

[0131] S902, in response to the user's operation of starting the AI ​​assistant, the terminal device displays an AI comprehensive interface on the upper layer of the second interface, and the AI ​​comprehensive interface includes AI dialogue controls and objects stored by the AI ​​assistant.

[0132] In this embodiment, when the target object is not dragged, the user's control operation to start the AI ​​assistant includes: saying wake-up words such as "Xiaoyi Xiaoyi", "Hello YOYO", "hey Siri", etc.; or clicking the AI ​​assistant icon; or long pressing the power button of the terminal device; or long pressing the prompt bar at the bottom of the second interface, etc. The prompt bar can be found in Figure 5 As shown, it is used to prompt the control function of the terminal device, such as the function of starting the AI ​​assistant in this embodiment. After the AI ​​assistant is started, the terminal device can display the objects stored by the AI ​​assistant.

[0133] In this embodiment, the AI ​​comprehensive interface is an AI control interface that integrates "AI dialogue" and "AI storage". The AI ​​comprehensive interface includes AI dialogue controls and objects stored by the AI ​​assistant.

[0134] It should be noted that this embodiment does not limit the way in which the AI ​​assistant displays the stored content. In addition to displaying it in the form of thumbnails, it can also be displayed in the form of folders, lists, etc. In addition, when the AI ​​comprehensive interface displays some storage objects, the user can control the terminal device to display other storage objects by sliding left and right or sliding up and down the target object.

[0135] S903: In response to the user's operation of dragging the target object to the second interface, the terminal device adds the target object to the second interface.

[0136] In this embodiment, the target object dragged by the user can be any one of the objects stored in the AI ​​assistant, or any multiple ones, which is not limited in this embodiment.

[0137] For S901 to S903, see Fig.10As shown in (a) to (d), taking the second interface as a memo interface as an example, in response to the user's long press operation on the prompt bar at the bottom of the memo interface, the terminal device displays the AI ​​comprehensive interface. Exemplarily, the AI ​​comprehensive interface is a control panel located at the bottom of the second interface, and the control panel includes: an AI dialogue control "voice ball", which is used to control the AI ​​assistant to communicate with the user. In addition, the AI ​​comprehensive interface also includes objects stored by the AI ​​assistant. Among them, the objects stored by the AI ​​assistant are presented in full or in part in the control panel in the form of thumbnails for user selection. In response to the user dragging picture A to the memo interface, the terminal device inserts picture A into the memo interface.

[0138] (2) The startup entry of the AI ​​storage function is embedded in the AI ​​dialogue function

[0139] Fig.11 is a schematic flow chart of a human-computer interaction method provided by another embodiment of the present application. Fig.11 As shown, the method involves a process in which a terminal device drags out and uses a target object from an AI assistant according to a user operation, specifically including steps S1101 to S1104.

[0140] S1101, the terminal device displays a second interface, wherein the second interface is the same as or different from the first interface shown above.

[0141] S1102, in response to the user's operation of starting the AI ​​assistant, the terminal device displays an AI dialogue interface on the upper layer of the second interface, and the AI ​​dialogue interface includes an AI dialogue control and an AI auxiliary control.

[0142] Among them, the operation of starting the AI ​​assistant is specifically referred to in S902 and will not be repeated here.

[0143] S1103, in response to the user's operation on the AI ​​auxiliary control, the terminal device displays an AI storage interface on the upper layer of the second interface, and the AI ​​storage interface includes objects stored by the AI ​​assistant.

[0144] S1104: In response to the user's operation of dragging the target object to the second interface, the terminal device adds the target object to the second interface.

[0145] For S1101 to S1104, see Fig.12As shown in (a) to (d) in the figure, taking the second interface as a memo interface as an example, in response to the user's long press operation on the prompt bar at the bottom of the second interface, the terminal device displays an AI dialogue interface, which includes an AI dialogue control "Voice Ball" and an AI auxiliary control "Xiao Yi Bang Ji". Among them, the "Voice Ball" is used to control the AI ​​assistant to start a conversation with the user, and the "Xiao Yi Bang Ji" is used to control the terminal device to display the AI ​​storage interface. In response to the user's control operation on the "Xiao Yi Bang Ji" control, the terminal device displays the AI ​​storage interface, which displays all or part of the content stored by the AI ​​assistant in the form of thumbnails. In response to the user's operation of dragging picture A to the memo interface, the terminal device inserts picture A into the memo interface.

[0146] (3) Different startup entries are set for the AI ​​storage function and the AI ​​dialogue function

[0147] In this embodiment, the terminal device activates different functions of the AI ​​assistant according to different control methods. For example, the terminal device activates the AI ​​dialogue function according to the first control operation and displays the AI ​​dialogue interface; and activates the AI ​​storage function according to the second control operation and displays the AI ​​storage interface.

[0148] In some embodiments, the first control operation is a control operation acting on the first control area of ​​the second interface, and the second control operation is a control operation acting on the second control area of ​​the second interface, and the control operation may be a click operation, a long press operation, etc. Among them, the first control area and the second control area may be two different hot areas on the screen of the terminal device, that is, areas that the user frequently touches to control the terminal device. The first control area and the second control area do not overlap, and this embodiment does not limit the positions of the first control area and the second control area.

[0149] For example, see Fig.13 As shown, the first control area is the middle area at the bottom of the second interface, such as the area where the prompt bar is located; the second control area is the left area and / or right area at the bottom of the second interface, such as the left area and / or right area of ​​the prompt bar. Alternatively, the first control area is the left area at the bottom of the second interface, such as the left area of ​​the prompt bar; the second control area is the right area at the bottom of the second interface, such as the right area of ​​the prompt bar.

[0150] In other embodiments, the control gestures of the first control operation and the second control operation are different. For example, the first control operation is a touch sliding operation from left to right at the bottom of the display interface, and the second control operation is a touch sliding operation from right to left at the bottom of the display interface. Of course, other control gestures may also be used, and this embodiment does not limit this.

[0151] The method provided in this embodiment is described below with reference to specific examples.

[0152] Fig.14 1401 to 1404.

[0153] S1401: The terminal device displays a second interface. The second interface is the same as or different from the first interface shown above.

[0154] S1402, in response to the user's first control operation at the bottom of the second interface, the terminal device displays an AI storage interface on the upper layer of the second interface, and the AI ​​storage interface includes objects stored by the AI ​​assistant.

[0155] S1403: In response to the user's operation of dragging the target object to the second interface, the terminal device adds the target object to the second interface.

[0156] For S1402 to S1403, see Fig.15 As shown in (a) to (c) in the figure, taking the target object as picture A and the second interface as the memo interface as an example, in response to the user's long press operation on the first control area at the bottom of the second interface, the terminal device displays the AI ​​storage interface. The AI ​​storage interface displays all or part of the objects stored by the AI ​​assistant, including picture A. In response to the user's operation of dragging picture A from the AI ​​storage interface to the memo interface, the terminal device displays picture A on the memo interface.

[0157] S1404, in response to the user's second control operation at the bottom of the second interface, the terminal device displays an AI dialogue interface.

[0158] For S1404, see Fig.16 As shown in (a) to (b), in response to the user's long press operation on the second control area in the second interface, the terminal device displays an AI dialogue interface. The AI ​​dialogue interface includes an AI dialogue control (such as a "voice ball"), which is used to control the AI ​​assistant to start a voice dialogue with the user.

[0159] It should be noted that the process of dragging the target object from the AI ​​assistant shown in S1402 to S1403 and the process of communicating with the assistant shown in S1404 are independent of each other, have no order requirement, and have no dependency relationship with each other.

[0160] In summary, through the human-computer interaction method provided in this embodiment, the AI ​​assistant can not only perform human-computer interaction with the user based on the target object, but also help the user temporarily store content for subsequent use by the user. It can be seen that the human-computer interaction function of the AI ​​assistant is richer, which can meet the diverse usage needs of users and provide a better user experience.

[0161] Optionally, in each of the above embodiments, in response to the user's operation of dragging the target object from the AI ​​assistant, the terminal device may copy the target object to the current display interface and continue to retain the target object stored in the AI ​​assistant; in addition, the terminal device may also move the target object to the current display interface and delete the target object stored in the AI ​​assistant, which is not limited in this embodiment.

[0162] In addition, the content stored in the AI ​​assistant can be used not only across applications, but also across devices. For example, for the first terminal device and the second terminal device logged in to the same device account, the AI ​​assistants used are essentially the same. After the AI ​​assistant of the first terminal device obtains the target object stored by the user, it can synchronize the target object to the AI ​​assistant of the second terminal device through wireless communication methods such as a server, Bluetooth, or wireless fidelity (Wi-Fi). Based on this, the AI ​​assistant of the second terminal device can control the use of the target object according to the user's drag operation. Please refer to the previous text for details, which will not be repeated here.

[0163] To further improve the user experience, the AI ​​assistant can also analyze the objects stored by the user and intelligently classify them based on the analysis results to facilitate user search.

[0164] In some embodiments, the AI ​​assistant can intelligently classify its stored objects according to different dimensions such as time, type and topic, so that users can search for content. It is worth noting that after the AI ​​assistant classifies the stored objects according to topics, it can better perform human-computer interaction in the later stage, for example, it can quickly retrieve the stored content required by the user, or can analyze the user's intention more deeply based on the objects stored by the user.

[0165] See also Fig.17As shown in (a) to (d) in the figure, taking the AI ​​storage interface as the "Xiao Yi Bang Ji" interface as an example, the AI ​​storage interface displays objects stored by the AI ​​assistant and a switch control. In response to the user's operation on the switch control, the terminal device displays different classification controls such as "classify by time", "classify by type" and "classify by topic" on the AI ​​storage interface. Among them, different classification controls are used to control the AI ​​assistant to display its stored objects in different classification methods. For example, in response to the user's operation on the "classify by topic" control, the AI ​​assistant classifies the stored objects according to the topic, stores the objects of the same topic in the same folder, and displays the number of objects stored in the folder. Exemplarily, the topic category can be "cooking", "fitness", "travel", "games" and "music", etc. In response to the user's selection operation on the target topic, the terminal device displays the objects belonging to the topic on the AI ​​storage interface. For example, in response to the user's selection operation on the "cooking" topic, the terminal device displays cooking-related objects stored by the AI ​​assistant, such as text, pictures, audio and video related to cooking.

[0166] In addition, the AI ​​assistant can also perform content preprocessing on the target object to improve the user experience. Exemplarily, the preprocessing includes extracting key information of the target object, such as extracting keywords, phone numbers, ID numbers, summaries, event information, etc. from the text; extracting text information, character information, location information, scene information and keywords from pictures and videos, and generating video cover images and core video clips. This embodiment does not limit the preprocessing method.

[0167] Take the AI ​​storage interface of the terminal device as an example, which is the "Xiaoyi Bangji" interface. Fig.18 As shown in the figure, the AI ​​storage interface displays objects stored by the AI ​​assistant and pre-processing controls. Fig.18 The "Intelligent Processing" control in the pre-processing control can control the AI ​​assistant to pre-process the target object selected by the user.

[0168] For example Fig.19A As shown, after the AI ​​assistant detects that the user has selected the target text and clicks the "Intelligent Processing" control, the AI ​​assistant processes the text, extracts the phone number in the text, and displays relevant controls for adding the phone number to the address book.

[0169] For example Fig.19B As shown, after the AI ​​assistant detects that the user has selected the target text and clicks the "Intelligent Processing" control, the AI ​​assistant processes the text, extracts the ID number from the text, and displays the relevant controls that will copy the ID number.

[0170] For example Fig.19CAs shown, after the AI ​​assistant detects that the user has selected Attachment 1 and clicks the "Intelligent Processing" control, the AI ​​assistant can pre-process Attachment 1 and generate a summary of Attachment 1, etc., so that the user can quickly understand the main information of Attachment 1.

[0171] In some embodiments, the AI ​​assistant can also store the pre-processing results of each object to form an AI memory, so as to better interact with the user in the future. For example, the AI ​​assistant can intelligently generate chat topics based on the AI ​​memory and have a conversation with the user.

[0172] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0173] Based on the same concept, as an implementation of the above method, an embodiment of the present application provides a human-computer interaction device, and the device embodiment corresponds to the aforementioned method embodiment. For ease of reading, the present device embodiment will no longer repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspond to and implement all the contents in the aforementioned method embodiment.

[0174] Fig. 20 2 is a schematic diagram of a human-computer interaction device 2000 provided in an embodiment of the present application. The human-computer interaction device 2000 includes: an input module 2001 , a display module 2002 and a processing module 2003 .

[0175] The input module 2001 is used to receive user input on the display interface of the terminal device, such as touch input, voice input, gesture input, etc. The input module 2001 is used to support the terminal device to perform the steps of receiving user operations in the above embodiment and / or other processes for the technology described herein. The input module can be a touch screen or other hardware or a combination of hardware and software.

[0176] The display module 2002 is used to control the terminal device to execute the interface display operations involved in the above embodiments and / or other processes for the technology described herein. The display module can be a touch screen or other hardware or a combination of hardware and software.

[0177] The processing module 2003 is used to control the terminal device to perform the control operations involved in the above embodiments. For example, the AI ​​assistant is started according to the user operation. Alternatively, after the target object is dragged to the area where the first AI control is located and the drag operation is released, the AI ​​assistant is controlled to store the target object. Alternatively, the AI ​​assistant is controlled to communicate with the user according to the target object.

[0178] An embodiment of the present application also provides a terminal device, which is configured to execute the human-computer interaction method shown in the above embodiments.

[0179] The present application also provides a chip, see Fig.21 As shown, the chip includes a processor and a memory, in which a computer program is stored. When the computer program is executed by the processor, the human-computer interaction method in the above embodiments is implemented.

[0180] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the human-computer interaction method provided in the above embodiments is implemented.

[0181] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by an electronic device, the electronic device implements the human-computer interaction method provided in the above embodiments.

[0182] It should be understood that the processor mentioned in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0183] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (doubledatarate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM), and direct rambus RAM (DR RAM).

[0184] In the embodiments provided in the present application, the division of each framework or module is only a logical function division. There may be other division methods in actual implementation. For example, multiple frameworks or modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0185] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0186] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0187] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0188] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A human-computer interaction method, characterized in that: Applied to a terminal device provided with an artificial intelligence (AI) assistant, the method comprises: Displaying a first interface, wherein the first interface includes a target object; In response to a drag operation on the target object, starting the AI ​​assistant and displaying a first AI control at the bottom of the first interface; After the target object is dragged to the area where the first AI control is located and the drag operation is released, the AI ​​assistant is controlled to store the target object.

2. The method according to claim 1, characterized in that After the AI ​​assistant is started, a second AI control is also displayed at the bottom of the first interface; the second AI control is used to control the AI ​​assistant to communicate with the user based on the target object after the target object is dragged to the area where the second AI control is located and the drag operation is released.

3. The method according to claim 2, characterized in that During the process of dragging the target object, the method further includes: When the target object moves toward the first AI control or the second AI control, the display size of the target object is reduced.

4. The method according to claim 2 or 3, characterized in that: During the process of dragging the target object, the method further includes: Determine a positional relationship between the dragged target object and the first AI control and the second AI control; When the dragged target object is located near the first AI control, increasing the display size of the first AI control and reducing the display size of the second AI control; and / or, When the dragged target object is located near the second AI control, the display size of the second AI control is increased, and the display size of the first AI control is reduced.

5. The method according to any one of claims 1 to 4, characterized in that: The target object includes: at least one of a picture, text, file, audio or video, and a web page link.

6. A human-computer interaction method, characterized in that: Applied to a terminal device provided with an artificial intelligence (AI) assistant, the method comprises: Display the second interface; In response to the operation of starting the AI ​​assistant, displaying the objects stored by the AI ​​assistant on the upper layer of the second interface; In response to an operation of dragging a target object to the second interface, the target object is added to the second interface; wherein the target object is any one or more objects stored in the AI ​​assistant.

7. The method according to claim 6, characterized in that In response to the operation of starting the AI ​​assistant, displaying objects stored by the AI ​​assistant on the upper layer of the second interface includes: In response to the operation of starting the AI ​​assistant, an AI comprehensive interface is displayed on the upper layer of the second interface, and the AI ​​comprehensive interface includes AI dialogue controls and objects stored by the AI ​​assistant.

8. The method according to claim 6, characterized in that In response to the operation of starting the AI ​​assistant, displaying objects stored by the AI ​​assistant on the upper layer of the second interface includes: In response to the operation of starting the AI ​​assistant, an AI dialogue interface is displayed on the upper layer of the second interface, wherein the AI ​​dialogue interface includes an AI dialogue control and an AI auxiliary control; In response to an operation on the AI ​​auxiliary control, an AI storage interface is displayed on the upper layer of the second interface, and the storage interface includes objects stored by the AI ​​assistant.

9. The method according to any one of claims 6 to 8, characterized in that: The operation of starting the AI ​​assistant includes: The operation of waking up the AI ​​assistant according to the wake-up word; or Control operations on AI-initiated controls; or, Pressing operation on a preset physical button.

10. A human-computer interaction method, characterized in that: Applied to a terminal device provided with an artificial intelligence (AI) assistant, the method comprises: Display the second interface; In response to a first control operation at the bottom of the second interface, displaying an object stored by the AI ​​assistant on an upper layer of the second interface; In response to an operation of dragging a target object to the second interface, the target object is added to the second interface; wherein the target object is any one or more objects stored in the AI ​​assistant.

11. The method according to claim 10, characterized in that The method further comprises: In response to a second control operation at the bottom of the second interface, an AI dialogue interface is displayed on an upper layer of the second interface; wherein the first control operation and the second control operation are different.

12. The method according to claim 11, characterized in that The first control operation is: a control operation acting on a first control area at the bottom of the second interface; The second control operation is: a control operation acting on a second control area at the bottom of the second interface; The first control area does not overlap with the second control area.

13. The method according to claim 12, characterized in that A prompt bar is displayed at the bottom of the second interface. The first control area is the area where the prompt bar is located; The second control area is the left area and / or the right area of ​​the prompt bar.

14. A terminal device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 5, or any one of claims 6 to 9, or any one of claims 10 to 13 is implemented.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5, or any one of claims 6 to 9, or any one of claims 10 to 13 is implemented.

16. A chip, characterized in that: The chip includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 5, or any one of claims 6 to 9, or any one of claims 10 to 13 is implemented.

Images