Human-computer interaction method, terminal device, readable storage medium and chip

Abstract

The application provides a human-computer interaction method, a terminal device, a readable storage medium and a chip, and relates to the field of artificial intelligence (AI) technology. The method is applied to a terminal device provided with an AI assistant, and specifically comprises: displaying a first interface, the first interface comprising a target object; in response to a dragging operation on the target object, starting the AI assistant and displaying a first AI control at the bottom of the first interface; after the target object is dragged to a region where the first AI control is located and the dragging operation is released, controlling the AI assistant to store the target object. In the embodiment of the application, the AI assistant can provide a more abundant human-computer interaction mode, which comprises helping the user to store the target object for subsequent use by the user, and has better user experience.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and more particularly to a human-computer interaction method, a terminal device, a readable storage medium, and a chip. Background Technology

[0002] With the development of computer technology, AI assistants such as "Xiaoyi," "YOYO," and "Siri" have gradually appeared on mobile phones, tablets, and other terminal devices. Users can interact with these devices through AI assistants. For example, users can control their devices to make calls, play music, or plan navigation. However, current AI assistants mainly interact with users through dialogue, which is a relatively simple form of interaction and cannot meet the diverse interaction needs of users. Summary of the Invention

[0003] This application provides a human-computer interaction method, a terminal device, a readable storage medium, and a chip to solve the problem that the interaction methods between AI assistants and users in the prior art are relatively simple and cannot meet the diverse interaction needs of users.

[0004] In a first aspect, embodiments of this application provide a human-computer interaction method, which is applied to a terminal device equipped with an AI assistant. The method includes: displaying a first interface, the first interface including a target object; in response to a drag operation on the target object, activating the AI ​​assistant and displaying a first AI control at the bottom of the first interface; and controlling the AI ​​assistant to store the target object after the target object is dragged to the area where the first AI control is located and the drag operation is released.

[0005] For example, the AI ​​assistant could be "Xiaoyi", "YOYO" or "Siri", etc. The AI ​​assistant can determine the user's intent based on the interaction with the user, and perform corresponding actions based on the determined user intent.

[0006] The method provided in this application embodiment allows a terminal device to use an AI assistant to help users store target objects for later use. Compared to existing AI assistants, the AI ​​assistant provided in this embodiment offers richer human-computer interaction methods and a better user experience.

[0007] In some embodiments, after the AI ​​assistant is started, a second AI control is also displayed at the bottom of the first interface; the second AI control is used to control the AI ​​assistant to have a conversation with the user based on the target object after the target object is dragged to the area where the second AI control is located and the drag operation is released.

[0008] The method provided in this application allows terminal devices to interact with users via an AI assistant based on a target object. The interaction is more specific, detailed, and explicit, resulting in a better user experience. For example, users can engage in topic discussions with the AI ​​assistant based on the target object, or control the AI ​​assistant to preprocess the target object. This preprocessing can include image recognition, similar image searching, etc.

[0009] In addition, in this embodiment, the user can control the AI ​​assistant to perform two different AI interaction functions based on the target object, namely "talking to the AI" and "letting the AI ​​keep it safe", by dragging the target object to the target control (i.e., the first AI control or the second AI control). This not only makes the control operation simple, but also better enriches and covers a variety of scenarios of human-AI assistant interaction.

[0010] In some embodiments, during the process of dragging the target object, the method further includes: reducing the display size of the target object as it approaches the first AI control or the second AI control.

[0011] Since the target object displayed in the first interface is usually large, reducing the size of the target object allows it to be accurately dragged to the area where the first AI control or the second AI control is located, thus improving control precision.

[0012] In some embodiments, during the process of dragging the target object, the method further includes: determining the positional relationship between the dragged target object and the first AI control and the second AI control; increasing the display size of the first AI control and decreasing the display size of the second AI control when the dragged target object is near the first AI control; or increasing the display size of the second AI control and decreasing the display size of the first AI control when the dragged target object is near the second AI control.

[0013] This method allows users to accurately drag and drop target objects to the area where the target control (i.e., the first AI control or the second AI control) is located, improving control precision and user experience.

[0014] In some embodiments, the target object includes at least one of: images, text, files, audio / video, and web links.

[0015] Secondly, embodiments of this application provide a human-computer interaction method, which is applied to a terminal device equipped with an AI assistant. The method includes: displaying a second interface; displaying objects stored by the AI ​​assistant on the upper layer of the second interface in response to the operation of starting the AI ​​assistant; and adding the target object to the second interface in response to the operation of dragging the target object to the second interface; wherein the target object is any one or more objects stored by the AI ​​assistant.

[0016] Using the method provided in this embodiment, users can quickly drag and drop target objects from the AI ​​assistant as needed, based on the content they store in the AI ​​assistant. For example, a user can drag and drop a previously stored image A from the AI ​​assistant to a chat application's dialog box, thereby quickly sending image A to a friend.

[0017] In some embodiments, in response to the operation of activating the AI ​​assistant, displaying objects stored by the AI ​​assistant on the upper layer of the second interface includes: in response to the operation of activating the AI ​​assistant, displaying an AI comprehensive interface on the upper layer of the second interface, the AI ​​comprehensive interface including AI dialogue controls and objects stored by the AI ​​assistant.

[0018] In this embodiment, the AI ​​dialogue function and the AI ​​storage function are configured with the same launch entry point. In other words, the user can launch two different AI functions simultaneously with a single control operation.

[0019] In some embodiments, in response to activating the AI ​​assistant, displaying objects stored by the AI ​​assistant on the upper layer of the second interface includes: in response to activating the AI ​​assistant, displaying an AI dialogue interface on the upper layer of the second interface, the AI ​​dialogue interface including AI dialogue controls and AI assistance controls; and in response to an operation on the AI ​​assistance controls, displaying an AI storage interface on the upper layer of the second interface, the storage interface including objects stored by the AI ​​assistant.

[0020] In this embodiment, the entry point for launching the AI ​​storage function is embedded in the AI ​​dialogue function. The terminal device needs to launch the AI ​​dialogue function first to display the AI ​​dialogue interface, and then further control the display of objects stored by the AI ​​assistant within the AI ​​dialogue interface. The AI ​​dialogue function is usually used more frequently than the AI ​​storage function. Therefore, through the method provided in this embodiment, the terminal device can prioritize launching the AI ​​dialogue functions that users frequently use, ensuring that the display interface corresponding to frequently used functions is concise.

[0021] In some embodiments, activating the AI ​​assistant includes: activating the AI ​​assistant according to a wake word; or, controlling the AI ​​activation control; or, pressing a preset physical button.

[0022] Thirdly, embodiments of this application provide a human-computer interaction method, which is applied to a terminal device equipped with an AI assistant. The method includes: displaying a second interface; responding to a first control operation at the bottom of the second interface, displaying objects stored by the AI ​​assistant on the upper layer of the second interface; and responding to a drag-and-drop operation of a target object to the second interface, adding the target object to the second interface; wherein the target object is any one or more objects stored by the AI ​​assistant.

[0023] In some embodiments, the method further includes: displaying an AI dialogue interface on top of the second interface in response to a second control operation at the bottom of the second interface; wherein the first control operation and the second control operation are different.

[0024] In this embodiment, the terminal device sets different activation methods for the AI ​​storage function and the AI ​​dialogue function. This method allows users to quickly activate the target AI function as needed, without activating other AI functions. For example, when needing to converse with an AI assistant, only the AI ​​dialogue function can be activated, without activating the AI ​​storage function; or, when needing to retrieve a target object from the AI ​​assistant, only the AI ​​storage function can be activated, without activating the AI ​​dialogue function.

[0025] In some embodiments, the first control operation is a control operation applied to a first control area at the bottom of the second interface; the second control operation is a control operation applied to a second control area at the bottom of the second interface; wherein the first control area and the second control area do not overlap.

[0026] For example, a prompt bar is displayed at the bottom of the second interface. The first control area is the area where the prompt bar is located, and the second control area is the left and / or right area of ​​the prompt bar. Alternatively, the first control area is the left area of ​​the prompt bar, and the second control area is the right area of ​​the prompt bar. Or, the first control area is the right area of ​​the prompt bar, and the second control area is the left area of ​​the prompt bar.

[0027] Fourthly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the methods shown in the first, second, or third aspects above.

[0028] Fifthly, embodiments of this application provide a human-computer interaction device, which includes at least one module for performing the methods shown in the first, second, or third aspects above.

[0029] Sixthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods shown in the first, second, or third aspects above.

[0030] In a seventh aspect, embodiments of this application provide a chip including a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, implements the methods shown in the first, second, or third aspects above.

[0031] Eighthly, embodiments of this application provide a computer program product storing a computer program that, when executed by a processor, can implement the methods shown in the first, second, or third aspects above.

[0032] It is understood that the beneficial effects of aspects four through eight above can be found in the relevant descriptions in aspects one, two or three above, and will not be repeated here. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of a scenario for waking up an AI assistant provided in an embodiment of this application;

[0034] Figure 2 This is a schematic diagram illustrating a scenario where a user controls a terminal device through an AI assistant, as provided in an embodiment of this application.

[0035] Figure 3 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application;

[0036] Figure 4 This is a schematic flowchart of a human-computer interaction method provided in one embodiment of this application;

[0037] Figure 5 This is a schematic diagram of the first interface provided in an embodiment of this application;

[0038] Figure 6 This is a schematic diagram of the first AI control and the second AI control provided in the embodiments of this application;

[0039] Figure 7 This is a schematic diagram illustrating how the AI ​​assistant provided in this application stores target objects based on user operations;

[0040] Figure 8 This is a schematic diagram illustrating the dialogue between the AI ​​assistant and the user based on the target object, as provided in the embodiments of this application.

[0041] Figure 9 This is a schematic flowchart of a human-computer interaction method provided in another embodiment of this application;

[0042] Figure 10 This is a schematic diagram illustrating how an AI assistant, according to an embodiment of this application, outputs and uses a target object based on user operations;

[0043] Figure 11 This is a schematic flowchart of a human-computer interaction method provided in another embodiment of this application;

[0044] Figure 12 This is a schematic diagram illustrating the AI ​​assistant outputting and using the target object based on user operations, provided in another embodiment of this application;

[0045] Figure 13 This is a schematic diagram of the first control area and the second control area provided in the embodiments of this application;

[0046] Figure 14 This is a schematic flowchart of a human-computer interaction method provided in another embodiment of this application;

[0047] Figure 15 This is a schematic diagram of the AI ​​assistant provided in this application outputting and using image A based on user operations;

[0048] Figure 16 This is a schematic diagram illustrating the AI ​​assistant displaying the activated AI dialogue function based on user operation, provided in an embodiment of this application.

[0049] Figure 17 This is a schematic diagram illustrating how the AI ​​assistant provided in this application intelligently categorizes stored objects;

[0050] Figure 18 This is a schematic diagram of the AI ​​storage interface provided in an embodiment of this application;

[0051] Figures 19A-19C These are schematic diagrams illustrating the preprocessing of target objects by the AI ​​assistant provided in different embodiments of this application;

[0052] Figure 20 This is a schematic diagram of the human-computer interaction device provided in the embodiments of this application;

[0053] Figure 21 This is a schematic diagram of the chip structure provided in the embodiments of this application. Detailed Implementation

[0054] It should be understood that in the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone.

[0055] In this embodiment, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0056] The technical solutions provided in the embodiments of this application will be described below with reference to the accompanying drawings.

[0057] With the development of computer technology, AI assistants such as "Xiaoyi", "YOYO" and "Siri" have gradually appeared on mobile phones, tablets and other terminal devices to assist users in human-computer interaction.

[0058] In this embodiment, the AI ​​assistant can also be called a voice assistant, intelligent voice, digital assistant, virtual assistant, etc. It can be understood as an information processing system that can recognize natural language input in the form of speech and / or text to infer the user's intent and perform corresponding actions based on the inferred user intent. Furthermore, the system can respond to the user's input through audible (such as speech) and visual (such as text, images) output formats.

[0059] To conserve power on terminal devices, AI assistants are typically in a dormant state, but users can activate them through specific control operations. Activating the AI ​​assistant can also be referred to as waking it up. For example... Figure 1 As shown, users can activate the AI ​​assistant by saying wake words such as "Xiaoyi Xiaoyi" or "Hello YOYO"; or by clicking the AI ​​assistant icon; or by pressing and holding the power button. This embodiment does not restrict the method of activating the AI ​​assistant.

[0060] AI assistants can interact with users once they are activated. For example... Figure 2 As shown, the AI ​​assistant can control the terminal device to play song A based on the user's voice input "Play song A". Alternatively, it can plan the route home and provide navigation based on the user's voice input "Navigate home". Or, it can dial the contact Lucy's number based on the user's voice input "Call Lucy". It is evident that the AI ​​assistant significantly improves the convenience of user interaction with the terminal device, bringing users a completely new human-computer interaction experience.

[0061] Currently, AI assistants typically interact directly with users through voice and text conversations. This means the AI ​​assistant receives user input (voice or text) and immediately responds with corresponding voice, text, or control actions. However, as terminal devices offer increasingly diverse functionalities, users may have needs beyond simply conversing with AI assistants. For example, to synchronize context and background knowledge with chat friends and enhance communication, users might need to share content. Therefore, users may require AI assistants to temporarily store text, images, files, audio / video, and web links for later use. Additionally, users may need to engage in conversations with AI assistants based on specific content (such as images, text, or audio). It's evident that current AI assistants offer relatively limited human-computer interaction methods. For instance, they lack a place to handle objects that don't require immediate AI intervention, failing to meet diverse user needs for human-computer interaction.

[0062] Therefore, this application provides a human-computer interaction method. This method is applied to a terminal device. Through this method, the AI ​​assistant in the terminal device can provide diverse human-computer interaction methods to meet the diverse interaction needs of users, such as helping users to store content and having dialogues with users based on specific content, which has a better user experience.

[0063] In this application embodiment, the terminal device can be a mobile phone, tablet computer, computer with wireless transceiver function, smart TV, projector, wearable device (such as smartwatch), in-vehicle device, augmented reality (AR) / virtual reality (VR) device, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), and other electronic devices equipped with AI assistants, such as vehicle infotainment systems. This application embodiment does not limit the specific type of terminal device.

[0064] Figure 3 This is a schematic diagram of the structure of the terminal device 300 provided in an embodiment of this application. See also... Figure 3As shown, the terminal device 300 includes a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, antenna 1, antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, a headphone jack 370D, a sensor module 380, buttons 390, a motor 391, an indicator 392, a camera 393, a display screen 394, and a subscriber identification module (SIM) card interface 395, etc.

[0065] It is understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the terminal device 300. In other embodiments of this application, the terminal device 300 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0066] For example, when the terminal device 300 is a mobile phone or a tablet computer, it may include all the components shown in the figure, or it may include only some of the components shown in the figure.

[0067] For example, when the terminal device 300 is a large-screen device, it may include the processor 310 shown in the figure, external memory interface 320, internal memory 321, universal serial bus (USB) interface 330, charging management module 340, power management module 341, wireless communication module 360, audio module 370, speaker 370A, receiver 370B, microphone 370C, camera 393, and display screen 394.

[0068] Processor 310 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The controller may serve as the central nervous system and command center of the terminal device 300. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0069] The external storage interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 200. The external memory card communicates with the processor 310 through the external storage interface 320 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0070] Internal memory 321 can be used to store executable program code, including instructions. Processor 310 executes various functional applications and data processing of electronic device 200 by running the instructions stored in internal memory 321. Internal memory 321 may include a program storage area and a data storage area. The program storage area can store the operating system and at least one application program required for a given function (such as sound playback, image playback, etc.). The data storage area can store data created during the use of electronic device 200 (such as audio data, phonebook, etc.).

[0071] USB port 330 is a USB standard compliant interface, which can be a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 330 can be used to connect a charger to charge electronic device 200, and can also be used for data transfer between electronic device 200 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0072] The charging management module 340 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 340 can receive charging input from the wired charger via a USB interface 330. In some wireless charging embodiments, the charging management module 340 can receive wireless charging input via the wireless charging coil of the terminal device 300. While charging the battery 342, the charging management module 340 can also supply power to the electronic device via the power management module 341.

[0073] The power management module 341 connects the battery 342, the charging management module 340, and the processor 310. The power management module 341 receives input from the battery 342 and / or the charging management module 340, providing power to the processor 310, internal memory 321, external memory, display screen 394, camera 393, and wireless communication module 360. The power management module 341 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance).

[0074] The wireless communication function of the terminal device 300 can be implemented through antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, modem processor and baseband processor, etc.

[0075] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 300 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0076] The mobile communication module 350 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G on the terminal device 300. The mobile communication module 350 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 350 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 350 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1.

[0077] In some embodiments, at least some functional modules of the mobile communication module 350 may be disposed in the processor 310. In some embodiments, at least some functional modules of the mobile communication module 350 and at least some modules of the processor 310 may be disposed in the same device.

[0078] The wireless communication module 360 ​​can provide solutions for wireless communication applications on the terminal device 300, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 360 ​​can be one or more devices integrating at least one communication processing module. The wireless communication module 360 ​​receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 310. The wireless communication module 360 ​​can also receive signals to be transmitted from processor 310, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0079] The terminal device 300 implements display functions through a GPU, a display screen 394, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 310 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0080] Display screen 394 is used to display images, videos, etc. Display screen 394 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, terminal device 300 may include one or N displays 394, where N is a positive integer greater than 1.

[0081] The terminal device 300 can implement audio functions through an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, a headphone jack 370D, and an application processor.

[0082] The audio module 370 is used to convert digital audio signals into analog audio signals for output, and also to convert analog audio inputs into digital audio signals. The audio module 370 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 370 may be located in the processor 310, or some functional modules of the audio module 370 may be located in the processor 310.

[0083] The speaker 370A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The terminal device 300 can listen to music or hands-free calls through the speaker 370A. For example, the speaker can play the comparison analysis results provided in the embodiments of this application.

[0084] The receiver 370B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the terminal device 300 answers a phone call or voice message, the receiver 370B can be brought close to the listener's ear to hear the voice.

[0085] Microphone 370C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 370C, inputting the sound signal into microphone 370C. Terminal device 300 may be equipped with at least one microphone 370C. In some embodiments, terminal device 300 may be equipped with two microphones 370C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, terminal device 300 may be equipped with three, four, or more microphones 370C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0086] The 370D headphone jack is used to connect wired headphones. The 370D headphone jack can be a USB 330 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0087] The sensor module 380 may include a pressure sensor 280A, a gyroscope sensor 280B, a barometric pressure sensor 280C, a magnetic sensor 280D, an accelerometer sensor 280E, a distance sensor 280F, a proximity sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, a bone conduction sensor 280M, etc.

[0088] The pressure sensor 280A is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor 280A can be disposed on the display screen 394. There are many types of pressure sensors 280A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When a force is applied to the pressure sensor 280A, the capacitance between the electrodes changes. The terminal device 300 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to the display screen 394, the terminal device 300 detects the touch operation intensity based on the pressure sensor 280A. The terminal device 300 can also calculate the touch position based on the detection signal from the pressure sensor 280A.

[0089] In some embodiments, touch operations applied to the same touch location but with different touch intensity can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS message is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS message is executed.

[0090] Touch sensor 280K, also known as a "touch panel," can be located on display screen 394. The touch sensor 280K and display screen 394 together form a touchscreen, also known as a "touchscreen." Touch sensor 280K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 394. In other embodiments, touch sensor 280K may also be located on the surface of terminal device 300, in a different position than display screen 394.

[0091] Buttons 390 include a power button, volume buttons, etc. Buttons 390 can be mechanical buttons or touch-sensitive buttons. Terminal device 300 can receive button input and generate key signal inputs related to user settings and function control of terminal device 300.

[0092] Motor 391 can generate vibration alerts. Motor 391 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can be corresponding to touch operations applied to different applications (such as taking photos, playing audio, etc.). Motor 391 can also correspond to different vibration feedback effects for touch operations applied to different areas of the display screen 394. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0093] Indicator 392 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0094] The SIM card interface 395 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 395 to make contact with and detach from the terminal device 300. The terminal device 300 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 395 can support Nano SIM cards, Micro SIM cards, and other SIM cards.

[0095] Based on the aforementioned terminal device, the human-computer interaction method provided in this embodiment will be described in detail below.

[0096] In this embodiment, in addition to engaging in basic conversations with the user based on their natural language input, the AI ​​assistant can also perform the following operations: (i) The AI ​​assistant receives and processes the user's input. For example, it helps the user store the content, or it engages in conversation with the user based on the content. (ii) The AI ​​assistant outputs the content stored by the user. In other words, when the user needs to use the content stored by the AI ​​assistant, they can quickly retrieve and use the content from the AI ​​assistant. These will be explained in detail below.

[0097] (a) The AI ​​assistant receives and processes user input.

[0098] Figure 4 This is a schematic flowchart of a human-computer interaction method provided in an embodiment of this application, specifically including steps S401 to S404.

[0099] S401, The terminal device displays a first interface, which includes the target object.

[0100] In this embodiment, the first interface includes various web pages and application interfaces, such as browser interfaces, gallery interfaces, contact interfaces, SMS interfaces, email interfaces, Word document interfaces, memo interfaces, chat interfaces, desktops, etc. The target object can be text, images, files, audio and video, web page links, etc., in the first interface, and this embodiment does not impose any restrictions on this.

[0101] When the terminal device displays the first interface, the user's interaction needs with the AI ​​assistant may include: (1) having the AI ​​assistant temporarily store the target object; (2) having the AI ​​assistant discuss topics based on the target object.

[0102] The first interface is Figure 5 Taking the gallery interface shown as an example, this interface displays various photos and videos stored by the user. In some scenarios, the user needs to discuss a topic with the AI ​​assistant based on a specific photo in this interface, such as asking the AI ​​assistant where photo A was taken. In other scenarios, the user needs the AI ​​assistant to temporarily store some photos so that they can be added to other application interfaces later. For example, the user needs to entrust photo A in the gallery interface to the AI ​​assistant for temporary storage, and then quickly retrieve photo A from the AI ​​assistant and add it to the notes interface when switching to the notes interface.

[0103] The implementation process of the above interaction requirements (1) and (2) will be explained in detail below.

[0104] S402, in response to the user's drag operation on the target object, the terminal device starts the AI ​​assistant and displays the first AI control and / or the second AI control at the bottom of the first interface.

[0105] Since AI assistants are typically in a dormant state, the terminal device needs to respond to specific control operations to activate them. For example, in this embodiment, the terminal device activates the AI ​​assistant in response to a user's drag-and-drop operation on a target object in the first interface.

[0106] After the AI ​​assistant is launched, the terminal device displays a first AI control and / or a second AI control at the bottom of the first interface. The first AI control is used to control the AI ​​assistant to temporarily store the target object so that the user can quickly access it later. The second AI control is used to control the AI ​​assistant to engage in dialogue with the user based on the target object. The user can choose to drag and drop the target object onto either the first or second AI control according to their needs, thereby controlling the AI ​​assistant to perform the corresponding function.

[0107] See Figure 6As shown, taking the first interface as a gallery interface and image A as the target object, in response to the user's drag-and-drop operation on image A, the terminal device launches the AI ​​assistant and displays the first AI control "Let AI Store" and the second AI control "Talk to AI" at the bottom of the first interface. If the user's need is for the AI ​​assistant to temporarily store image A, image A is dragged to the area where the first AI control "Let AI Store" is located, and the drag operation is released. If the user's need is to have a conversation with the AI ​​assistant based on image A, image A is dragged to the area where the second AI control "Talk to AI" is located, and the drag operation is released.

[0108] In this embodiment, drag-and-drop operation can be interpreted as: the process by which a user selects a target object through mouse clicks, touch, or other control methods, and controls the movement of the target object on the display interface. Drag-and-drop operation makes user behavior more visible, improves user experience, and makes human-computer interaction more natural. Correspondingly, releasing a drag-and-drop operation can be interpreted as: the user stopping the drag-and-drop operation. After the drag-and-drop operation is released at a certain location on the display interface, the target object selected by the user is also released at that location.

[0109] S403, after the target object is dragged to the area where the first AI control is located and the drag operation is released, the terminal device controls the AI ​​assistant to store the target object.

[0110] In this embodiment, in addition to controlling the movement of the display position of the target object, the drag operation is also extended to include a copy function. That is, after the target object is dragged to the area where the first AI control (such as the "Let AI Keep" control) is located and the drag operation is released, the target object is copied and stored in the AI ​​assistant, while the original position of the target object is retained.

[0111] Optionally, the AI ​​assistant enters a sleep state after storing the target object. Correspondingly, the terminal device reverts to displaying the first interface, without showing the first AI control, second AI control, or other related content.

[0112] See Figure 7 As shown in (a) to (c), taking the first interface as the gallery interface and the target object as image A, in response to the user dragging image A to the area where the "Let AI Keep" control is located and releasing the drag operation, the AI ​​assistant stores image A. After image A is stored, the terminal device resumes displaying the first interface.

[0113] Through step S403, the terminal device can temporarily store the target object through the AI ​​assistant. Compared with the existing AI assistant, the AI ​​assistant provided in this embodiment can provide richer human-computer interaction methods and has a better user experience.

[0114] S404, after the target object is dragged to the area where the second AI control is located, and the drag operation is released, the terminal device controls the AI ​​assistant to have a dialogue with the user based on the target object.

[0115] In this embodiment, after the target object is dragged to the area where the second AI control (such as the "Talk to AI" control) is located, and the drag operation is released, the terminal device displays an AI dialogue interface, which includes the AI ​​dialogue control and a thumbnail of the target object. Based on this, the user can trigger the dialogue function of the AI ​​assistant through the AI ​​dialogue control and have a dialogue with the AI ​​assistant regarding the target object.

[0116] See Figure 8 As shown in (a) to (c), taking the first interface as a gallery interface and the target object as image A as an example, in response to the user dragging image A to the area where the "Talk to AI" control is located and releasing the drag operation, the terminal device displays the AI ​​dialogue interface. This AI dialogue interface includes an AI dialogue control (such as a "voice ball") and a thumbnail of image A. The user can trigger the AI ​​assistant's dialogue function through the "voice ball" control and have a dialogue with the AI ​​assistant based on image A.

[0117] For example, a user can instruct the AI ​​assistant to recognize the content in image A through voice interaction; or instruct the AI ​​assistant to perform specific processing on image A, such as recognizing text in image A, adjusting the contrast of image A, etc.; or ask for basic information about image A, such as the size of the memory occupied, the image format, the time of shooting, the location of shooting, etc.

[0118] It should be noted that in this embodiment, the dialogue between the AI ​​assistant and the user based on the target object should not be narrowly interpreted as voice communication. It may also include controlling the AI ​​assistant to process the target object through preset controls.

[0119] For example Figure 8 As shown in (c), the AI ​​dialogue interface also includes some preset controls, such as "Smart Image Recognition" and "Similar Images." "Smart Image Recognition" controls the AI ​​assistant to identify and process information in images, such as extracting text, translating text, extracting tables, recognizing QR codes, and identifying objects. "Similar Images" controls the AI ​​assistant to search for other images similar to the given image on local servers, cloud servers, or third-party servers.

[0120] Through step S404, the terminal device can interact with the user based on the target object through the AI ​​assistant, and the interaction content is more specific, detailed and clear, resulting in a better user experience.

[0121] Optionally, as the target object moves toward the first AI control or the second AI control according to the drag operation, the terminal device can also dynamically adjust the shape of the target object, the first AI control, and the second AI control to present a better visual and control effect to the user and improve the user experience.

[0122] In some embodiments, see Figure 7 (b) or Figure 8 As shown in (b), the size of the target object (e.g., image A) can gradually decrease to a preset size as it moves towards the first AI control or the second AI control according to the drag operation, so that the target object can be accurately dragged to the area where the first AI control or the second AI control is located. This embodiment does not limit the specific value of the preset size.

[0123] In other embodiments, see Figure 7 (b) or Figure 8 As shown in (b), the terminal device can also detect the relative positional relationship between the target object and the first AI control and the second AI control. When the target object is near the first AI control, the terminal device increases the display size of the first AI control and decreases the display size of the second AI control. Alternatively, when the target object is near the second AI control, the terminal device increases the display size of the second AI control and decreases the display size of the first AI control. Using this method, the user can accurately drag the target object to the area where the target control (i.e., the first AI control or the second AI control) is located.

[0124] In summary, through the human-computer interaction method provided in this application, users can control the AI ​​assistant to perform two different AI interaction functions based on the target object—"talking to the AI" and "letting the AI ​​keep it safe"—through a single control operation, namely, dragging the target object to the target control. This not only simplifies the control operation but also better enriches and covers various scenarios of human-AI assistant interaction.

[0125] (ii) The AI ​​assistant outputs the content stored by the user.

[0126] For content stored in the AI ​​assistant, users can quickly drag and drop target objects from the AI ​​assistant as needed. For example, a user can drag and drop a previously stored image A from the AI ​​assistant into a chat application's dialog box to quickly send image A to a friend.

[0127] Because AI assistants are configured with different functions such as AI dialogue and AI storage, terminal devices may set different launch points for these functions. The following example illustrates the process by which a user retrieves and uses a target object from the AI ​​assistant, using different launch point configurations as examples.

[0128] (1) Configure AI storage and AI dialogue functions with the same startup entry.

[0129] Figure 9 This is a schematic flowchart illustrating a human-computer interaction method provided in another embodiment of this application. See also... Figure 9 As shown, the method involves the process of a terminal device dragging and using a target object from an AI assistant based on user operation, specifically including steps S901 to S903.

[0130] S901, the terminal device displays a second interface. This second interface may be the same as or different from the first interface shown above.

[0131] S902, in response to the user's operation of activating the AI ​​assistant, the terminal device displays the AI ​​comprehensive interface on the upper layer of the second interface. The AI ​​comprehensive interface includes AI dialogue controls and objects stored by the AI ​​assistant.

[0132] In this embodiment, without involving dragging or dropping the target object, the user's control operations to activate the AI ​​assistant include: saying wake words such as "Xiaoyi Xiaoyi," "Hello YOYO," or "Hey Siri"; or clicking the AI ​​assistant icon; or pressing and holding the power button on the terminal device; or pressing and holding the prompt bar at the bottom of the second interface, etc. This prompt bar can be found in [reference needed]. Figure 5 As shown, this is used to prompt the terminal device for control functions, such as the function of activating the AI ​​assistant in this embodiment. After the AI ​​assistant is activated, the terminal device can display the objects stored by the AI ​​assistant.

[0133] In this embodiment, the AI ​​integrated interface is an AI control interface that combines "AI dialogue" and "AI storage." This AI integrated interface includes AI dialogue controls and objects stored by the AI ​​assistant.

[0134] It should be noted that this embodiment does not limit the way the AI ​​assistant displays stored content. Besides displaying it as thumbnails, it can also display it in folders, lists, or other formats. Furthermore, when some stored objects are displayed on the AI ​​interface, users can control the terminal device to display other stored objects by swiping left and right or up and down on the target object.

[0135] S903, in response to the user's operation of dragging and dropping the target object to the second interface, the terminal device adds the target object to the second interface.

[0136] In this embodiment, the target object that the user drags can be any one or any number of objects stored by the AI ​​assistant; this embodiment does not impose any restrictions.

[0137] For S901 to S903, see Figure 10As shown in (a) to (d), taking the memo interface as an example, in response to the user's long-press operation on the prompt bar at the bottom of the memo interface, the terminal device displays the AI ​​comprehensive interface. For example, this AI comprehensive interface is a control panel located at the bottom of the second interface. This control panel includes an AI dialogue control, a "voice ball," which is used to control the AI ​​assistant to converse with the user. Additionally, the AI ​​comprehensive interface also includes objects stored by the AI ​​assistant. These objects are presented in whole or in part in the control panel as thumbnails for user selection. In response to the user dragging image A onto the memo interface, the terminal device inserts image A into the memo interface.

[0138] (2) The entry point for the AI ​​storage function is embedded in the AI ​​dialogue function.

[0139] Figure 11 This is a schematic flowchart illustrating a human-computer interaction method provided in another embodiment of this application. See also... Figure 11 As shown, the method involves the process of a terminal device dragging and using a target object from an AI assistant based on user operation, specifically including steps S1101 to S1104.

[0140] S1101, the terminal device displays a second interface. This second interface may be the same as or different from the first interface shown above.

[0141] S1102, in response to the user's operation of activating the AI ​​assistant, the terminal device displays an AI dialogue interface on top of the second interface, which includes AI dialogue controls and AI assistance controls.

[0142] For details on activating the AI ​​assistant, please refer to S902; it will not be repeated here.

[0143] S1103, in response to the user's operation of the AI-assisted control, the terminal device displays the AI ​​storage interface on the upper layer of the second interface, which includes the objects stored by the AI ​​assistant.

[0144] S1104, in response to the user's operation of dragging and dropping the target object to the second interface, the terminal device adds the target object to the second interface.

[0145] For S1101 to S1104, see [link / reference]. Figure 12As shown in (a) to (d), taking the memo interface as an example, in response to the user's long press on the prompt bar at the bottom of the second interface, the terminal device displays an AI dialogue interface. This AI dialogue interface includes an AI dialogue control "Voice Ball" and an AI assistance control "Xiao Yi Helper". The "Voice Ball" is used to control the AI ​​assistant to start a dialogue with the user, and "Xiao Yi Helper" is used to control the terminal device to display the AI ​​storage interface. In response to the user's control operation on the "Xiao Yi Helper" control, the terminal device displays the AI ​​storage interface, which displays all or part of the content stored by the AI ​​assistant in the form of thumbnails. In response to the user dragging image A to the memo interface, the terminal device inserts image A into the memo interface.

[0146] (3) AI storage function and AI dialogue function are set with different launch entrances.

[0147] In this embodiment, the terminal device activates different functions of the AI ​​assistant according to different control methods. For example, the terminal device activates the AI ​​dialogue function and displays the AI ​​dialogue interface according to the first control operation; and activates the AI ​​storage function and displays the AI ​​storage interface according to the second control operation.

[0148] In some embodiments, the first control operation is a control operation applied to the first control area of ​​the second interface, and the second control operation is a control operation applied to the second control area of ​​the second interface. The control operation can be a click operation, a long press operation, etc. The first control area and the second control area can be two different hot zones on the terminal device screen, i.e., areas that the user frequently touches to control the terminal device. The first control area and the second control area do not need to overlap; this embodiment does not restrict the position of the first control area and the second control area.

[0149] For example, see Figure 13 As shown, the first control area is the middle area at the bottom of the second interface, such as the area where the prompt bar is located; the second control area is the left and / or right area at the bottom of the second interface, such as the left and / or right area of ​​the prompt bar. Alternatively, the first control area is the left area at the bottom of the second interface, such as the left area of ​​the prompt bar; the second control area is the right area at the bottom of the second interface, such as the right area of ​​the prompt bar.

[0150] In other embodiments, the control gestures for the first control operation and the second control operation are different. For example, the first control operation is a touch swipe operation from left to right at the bottom of the display interface, and the second control operation is a touch swipe operation from right to left at the bottom of the display interface. Of course, other control gestures are also possible, and this embodiment does not limit them.

[0151] The method provided in this embodiment will be described below with specific examples.

[0152] Figure 14 This is a schematic flowchart illustrating a human-computer interaction method provided in another embodiment of this application. The method involves a terminal device dragging and using a target object from an AI assistant based on user operation, specifically including steps S1401 to S1404.

[0153] S1401, the terminal device displays a second interface. The second interface may be the same as or different from the first interface shown above.

[0154] S1402, in response to the user's first control operation at the bottom of the second interface, the terminal device displays an AI storage interface on the upper layer of the second interface, which includes objects stored by the AI ​​assistant.

[0155] S1403, in response to the user's operation of dragging and dropping the target object to the second interface, the terminal device adds the target object to the second interface.

[0156] For S1402 to S1403, see [link / reference]. Figure 15 As shown in (a) to (c), taking the target object as image A and the second interface as the Notes interface as an example, in response to the user's long-press operation in the first control area at the bottom of the second interface, the terminal device displays the AI ​​Storage interface. This AI Storage interface displays all or part of the objects stored by the AI ​​assistant, including image A. In response to the user dragging image A from the AI ​​Storage interface to the Notes interface, the terminal device displays image A in the Notes interface.

[0157] S1404, in response to the user's second control operation at the bottom of the second interface, the terminal device displays the AI ​​dialogue interface.

[0158] For S1404, see Figure 16 As shown in (a) to (b), in response to a user's long-press operation in the second control area of ​​the second interface, the terminal device displays an AI dialogue interface. This AI dialogue interface includes AI dialogue controls (such as a "voice ball"), which are used to control the AI ​​assistant to begin a voice dialogue with the user.

[0159] It should be noted that the process of dragging the target object from the AI ​​assistant shown in S1402 to S1403, and the process of conversing with the assistant shown in S1404 are independent of each other, have no requirement for a specific order, and are not dependent on each other.

[0160] In summary, through the human-computer interaction method provided in this embodiment, the AI ​​assistant can not only interact with the user based on the target object, but also help the user temporarily store content for later use. It can be seen that the AI ​​assistant's human-computer interaction functions are richer, capable of meeting diverse user needs and providing a better user experience.

[0161] Optionally, in the above embodiments, in response to the user's operation of dragging the target object from the AI ​​assistant, the terminal device can copy the target object to the current display interface and continue to retain the target object stored in the AI ​​assistant; alternatively, the terminal device can also move the target object to the current display interface and delete the target object stored in the AI ​​assistant. This embodiment does not limit this.

[0162] Furthermore, the content stored in the AI ​​assistant can be used not only across applications but also across devices. For example, the AI ​​assistant used by the first and second terminal devices logged into the same device account is essentially the same. After the AI ​​assistant on the first terminal device obtains the target object stored by the user, it can synchronize the target object to the AI ​​assistant on the second terminal device via wireless communication methods such as a server, Bluetooth, or Wi-Fi. Based on this, the AI ​​assistant on the second terminal device can control the use of the target object according to the user's drag-and-drop operations, as detailed above, and will not be repeated here.

[0163] To further enhance the user experience, the AI ​​assistant can also analyze the objects stored by the user and intelligently categorize them based on the analysis results for easy retrieval by the user.

[0164] In some embodiments, the AI ​​assistant can intelligently categorize its stored objects according to different dimensions such as time, type, and topic to facilitate content retrieval for users. It's worth noting that after categorizing stored objects by topic, the AI ​​assistant can better facilitate human-computer interaction later on. For example, it can quickly retrieve the stored content needed by the user, or it can more deeply analyze the user's intent based on the stored objects.

[0165] See Figure 17As shown in (a) to (d), taking the "Xiaoyi Helper" AI storage interface as an example, this AI storage interface displays objects stored by the AI ​​assistant and switching controls. In response to the user's operation of the switching controls, the terminal device displays different category controls on the AI ​​storage interface, such as "Categorized by Time," "Categorized by Type," and "Categorized by Topic." These different category controls are used to control the AI ​​assistant to display its stored objects according to different categorization methods. For example, in response to the user's operation of the "Categorized by Topic" control, the AI ​​assistant categorizes the stored objects according to topics, storing objects with the same topic in the same folder and displaying the number of objects stored in that folder. For example, the topic categories could be "Cooking," "Fitness," "Travel," "Games," and "Music." In response to the user's selection of a target topic, the terminal device displays objects belonging to that topic on the AI ​​storage interface. For example, in response to the user's selection of the "Cooking" topic, the terminal device displays cooking-related objects stored by the AI ​​assistant, such as cooking-related text, images, audio, and video.

[0166] In addition, the AI ​​assistant can perform content preprocessing on the target object to improve the user experience. For example, this preprocessing includes extracting key information from the target object, such as keywords, phone numbers, ID card numbers, summaries, and event information from text; extracting text information, people information, location information, scene information, and keywords from images and videos; and generating video cover images and core video clips. This embodiment does not limit the preprocessing method.

[0167] Taking the AI ​​storage interface of the terminal device as an example, which is the "Xiaoyi Helper" interface, see [link / reference]. Figure 18 As shown in the image, the AI ​​storage interface displays objects stored by the AI ​​assistant and preprocessing controls. For example, the preprocessing control is... Figure 18 The "Smart Processing" control within the app. This preprocessing control enables the AI ​​assistant to preprocess the target object selected by the user.

[0168] For example Figure 19A As shown, after the AI ​​assistant detects that the user has selected the target text and clicked the "Smart Processing" control, the AI ​​assistant processes the text, extracts the phone number from the text, and displays the relevant controls to add the phone number to the address book.

[0169] For example Figure 19B As shown, after the AI ​​assistant detects that the user has selected the target text and clicked the "Smart Processing" control, the AI ​​assistant processes the text, extracts the ID card number from the text, and displays the relevant control to copy the ID card number.

[0170] For example Figure 19CAs shown, after the AI ​​assistant detects that the user has selected Attachment 1 and clicked the "Smart Processing" control, the AI ​​assistant can preprocess Attachment 1 and generate a summary of Attachment 1, so that the user can quickly understand the main information of Attachment 1.

[0171] In some embodiments, the AI ​​assistant can also store the preprocessing results of various objects to form an AI memory, enabling better human-computer interaction with the user later. For example, the AI ​​assistant can intelligently generate chat topics based on the AI ​​memory and engage in dialogue with the user.

[0172] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0173] Based on the same concept, as an implementation of the above method, this application provides a human-computer interaction device. This device embodiment corresponds to the aforementioned method embodiment. For ease of reading, this device embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspondingly implement all the contents of the aforementioned method embodiment.

[0174] Figure 20 This is a schematic diagram of the structure of a human-computer interaction device 2000 provided in one embodiment of this application. The human-computer interaction device 2000 includes: an input module 2001, a display module 2002, and a processing module 2003.

[0175] Input module 2001 is used to receive user input on the display interface of the terminal device, such as touch input, voice input, gesture input, etc. Input module 2001 is used to support the terminal device in performing the steps of receiving user operations in the above embodiments and / or other processes used in the technology described herein. The input module may be a touch screen or other hardware or a combination of hardware and software.

[0176] The display module 2002 is used to control the terminal device to perform the interface display operations involved in the above embodiments and / or other processes used in the technology described herein. The display module may be a touch screen or other hardware or a combination of hardware and software.

[0177] The processing module 2003 is used to control the terminal device to perform the control operations involved in the above embodiments. For example, it can start the AI ​​assistant based on user operation. Or, after the target object is dragged to the area where the first AI control is located and the drag operation is released, it can control the AI ​​assistant to store the target object. Alternatively, it can control the AI ​​assistant to engage in dialogue with the user based on the target object.

[0178] This application also provides a terminal device configured to execute the human-computer interaction methods shown in the above embodiments.

[0179] This application also provides a chip, see [link to relevant documentation] Figure 21 As shown, the chip includes a processor and a memory, in which a computer program is stored. When the computer program is executed by the processor, it implements the human-computer interaction methods described in the above embodiments.

[0180] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the human-computer interaction methods provided in the above embodiments.

[0181] This application also provides a computer program product, which includes a computer program that, when run by an electronic device, enables the electronic device to implement the human-computer interaction methods provided in the above embodiments.

[0182] It should be understood that the processor mentioned in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0183] It should also be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0184] In the embodiments provided in this application, the division of each framework or module is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple frameworks or modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0185] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0186] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0187] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0188] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A human-machine interaction method, characterized in that, Applied to a terminal device provided with an artificial intelligence (AI) assistant, the method comprises: displaying a first interface, the first interface comprising a target object; in response to a drag operation on the target object, starting the AI assistant and displaying first AI controls and second AI controls comprised by the AI assistant at the bottom of the first interface; the first AI controls being used for a storage function of the AI assistant, and the second AI controls being used for a dialogue function of the AI assistant; after the target object is dragged to a region where the first AI controls are located and the drag operation is released, controlling the AI assistant to store the target object; or after the target object is dragged to a region where the second AI controls are located and the drag operation is released, displaying an AI dialogue interface of the AI assistant, the AI dialogue interface comprising AI dialogue controls and a thumbnail of the target object, and the AI dialogue controls being used to realize a dialogue with the AI assistant based on the target object; displaying a second interface; in response to an operation of starting the AI assistant, displaying an AI comprehensive interface of the AI assistant, the AI comprehensive interface comprising AI dialogue controls and the target object stored by the AI assistant; or, in response to an operation of starting the AI assistant, displaying an AI dialogue interface, the AI dialogue interface comprising AI dialogue controls and AI assistance controls; in response to an operation on the AI assistance controls, displaying an AI storage interface, the storage interface comprising the target object; in response to an operation of dragging the target object to the second interface, adding the target object to the second interface.

2. The method of claim 1, wherein, In the process of dragging the target object, the method further comprises: in the process of the target object moving towards the first AI controls or the second AI controls, reducing the display size of the target object.

3. The method according to claim 1 or 2, characterized in that, In the process of dragging the target object, the method further comprises: determining the positional relationship between the dragged target object and the first AI controls and the second AI controls; when the dragged target object is located near the first AI controls, increasing the display size of the first AI controls and reducing the display size of the second AI controls; and / or, when the dragged target object is located near the second AI controls, increasing the display size of the second AI controls and reducing the display size of the first AI controls.

4. The method according to claim 1 or 2, characterized in that, The target object comprises at least one of a picture, a text, a file, an audio / video, and a webpage link.

5. The method according to claim 1 or 2, characterized in that, The operation of starting the AI assistant comprises: an operation of waking up the AI assistant according to a wake-up word; or, a control operation on AI start controls; or, a pressing operation on a preset physical button.

6. A human-machine interaction method characterized by, Applied to a terminal device provided with an artificial intelligence (AI) assistant, the method comprises: displaying a second interface; in response to a first control operation at the bottom of the second interface, displaying objects stored by the AI assistant on the upper layer of the second interface; In response to a target object operation of dragging to the second interface, the target object is added to the second interface; wherein the target object is any one or more of the objects stored by the AI assistant, and the target object is stored by the method of any one of claims 1-5.

7. The method of claim 6, wherein, The method further comprises: In response to a second control operation at the bottom of the second interface, an AI conversation interface is displayed on the second interface; wherein the first control operation and the second control operation are different.

8. The method of claim 7, wherein, The first control operation is a control operation acting on a first control area at the bottom of the second interface. The second control operation is a control operation acting on a second control area at the bottom of the second interface. The first control area and the second control area are not overlapped.

9. The method of claim 8, wherein, A prompt bar is displayed at the bottom of the second interface. The first control area is the area where the prompt bar is located. The second control area is the left area and / or the right area of the prompt bar.

10. A terminal device, comprising: A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method of any one of claims 1-5, or the method of any one of claims 6-9.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method of any one of claims 1-5, or the method of any one of claims 6-9.

12. A chip, characterized by The chip comprises a processor and a memory, and the memory stores a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1-5, or the method of any one of claims 6-9.

Images