Human-machine interaction method, terminal device, readable storage medium, and chip

Abstract

The present application relates to the technical field of artificial intelligence (AI), and provides a human-machine interaction method, a terminal device, a readable storage medium, and a chip. The method is applied to a terminal device provided with an AI assistant, and specifically comprises: displaying a first interface, wherein the first interface comprises a target object; in response to a dragging operation on the target object, starting the AI assistant, and displaying a first AI control at the bottom of the first interface; and when the target object is dragged to an area where the first AI control is located and the dragging operation is released, controlling the AI assistant to store the target object. In the embodiments of the present application, the AI assistant can provide more diverse human-machine interaction modes, including assisting users in storing target objects for subsequent use, thereby providing better user experience.

Description

Human-computer interaction method, terminal device, readable storage medium and chip

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on January 17, 2024, with application number 202410071228.3 and application name “Human-computer interaction method, terminal device, readable storage medium and chip”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a human-computer interaction method, a terminal device, a readable storage medium, and a chip. Background Art

[0003] With the advancement of computer technology, AI assistants such as "Xiaoyi," "YOYO," and "Siri" have gradually appeared on mobile phones, tablets, and other terminal devices. Users can use AI assistants to interact with these devices. For example, users can use AI assistants to control their devices to make calls, play music, or plan navigation. However, current AI assistants primarily interact with users through conversations, which is a relatively simple form of interaction and cannot meet users' diverse interaction needs. Summary of the Invention

[0004] The present application provides a human-computer interaction method, terminal device, readable storage medium and chip, which are used to solve the problem in the prior art that the interaction method between AI assistants and users is relatively single and cannot meet the diverse interaction needs of users.

[0005] In a first aspect, an embodiment of the present application provides a human-computer interaction method, which is applied to a terminal device provided with an AI assistant, and the method includes: displaying a first interface, the first interface including a target object; in response to a drag operation on the target object, starting the AI ​​assistant and displaying a first AI control at the bottom of the first interface; after the target object is dragged to the area where the first AI control is located and the drag operation is released, controlling the AI ​​assistant to store the target object.

[0006] Exemplarily, the AI ​​assistant may be "Xiaoyi", "YOYO" or "Siri", etc. The AI ​​assistant can determine the user intent based on the interaction with the user and perform corresponding actions based on the determined user intent.

[0007] Through the method provided in the embodiment of this application, the terminal device can help the user store the target object through the AI ​​assistant for subsequent use by the user. Compared with existing AI assistants, the AI ​​assistant provided in this embodiment can provide richer human-computer interaction methods and have a better user experience.

[0008] In some embodiments, after the AI ​​assistant is started, a second AI control is also displayed at the bottom of the first interface; the second AI control is used to control the AI ​​assistant to communicate with the user based on the target object after the target object is dragged to the area where the second AI control is located and the drag operation is released.

[0009] Through the methods provided in the embodiments of this application, a terminal device can conduct human-computer interaction with a user based on a target object through an AI assistant. The interactive content is more specific, detailed, and clear, providing a better user experience. For example, a user can discuss a topic with the AI ​​assistant based on the target object, or control the AI ​​assistant to pre-process the target object. This pre-processing can be image recognition, similar image search, etc.

[0010] In addition, in this embodiment, the user can control the AI ​​assistant to implement two different AI interaction functions, "talk to AI" and "let AI keep" based on the target object through a control operation, that is, dragging the target object to the target control (i.e., the first AI control or the second AI control). This not only makes the control operation simple, but also can better enrich and cover various scenarios of interaction between people and AI assistants.

[0011] In some embodiments, while the target object is being dragged, the method further includes: while the target object is approaching the first AI control or the second AI control, reducing the display size of the target object.

[0012] Since the target object displayed in the first interface is usually large in size, reducing the size of the target object allows it to be accurately dragged to the area where the first AI control or the second AI control is located, thereby improving control accuracy.

[0013] In some embodiments, while the target object is being dragged, the method further includes: determining the positional relationship between the dragged target object and the first AI control and the second AI control; when the dragged target object is near the first AI control, increasing the display size of the first AI control and reducing the display size of the second AI control; or, when the dragged target object is near the second AI control, increasing the display size of the second AI control and reducing the display size of the first AI control.

[0014] Through this method, the user can accurately drag the target object to the area where the target control (i.e., the first AI control or the second AI control) is located, thereby improving control accuracy and user experience.

[0015] In some embodiments, the target object includes at least one of: a picture, text, a file, audio or video, and a web link.

[0016] In a second aspect, an embodiment of the present application provides a human-computer interaction method, which is applied to a terminal device provided with an AI assistant, and the method includes: displaying a second interface; in response to the operation of starting the AI ​​assistant, displaying an object stored by the AI ​​assistant on the upper layer of the second interface; in response to the operation of dragging a target object to the second interface, adding the target object to the second interface; wherein the target object is any one or more objects stored by the AI ​​assistant.

[0017] Through the method provided in this embodiment, users can quickly drag and drop content stored in the AI ​​assistant and use the target object as needed. For example, a user can drag and drop a previously stored picture A from the AI ​​assistant to the chat application dialog box to quickly send picture A to a friend.

[0018] In some embodiments, in response to the operation of starting the AI ​​assistant, objects stored by the AI ​​assistant are displayed on the upper layer of the second interface, including: in response to the operation of starting the AI ​​assistant, an AI comprehensive interface is displayed on the upper layer of the second interface, and the AI ​​comprehensive interface includes AI dialogue controls and objects stored by the AI ​​assistant.

[0019] In this embodiment, the AI ​​conversation function and the AI ​​storage function are configured with the same startup entry. In other words, the user can activate two different AI functions at the same time through a single control operation.

[0020] In some embodiments, in response to the operation of starting the AI ​​assistant, objects stored by the AI ​​assistant are displayed on the upper layer of the second interface, including: in response to the operation of starting the AI ​​assistant, an AI dialogue interface is displayed on the upper layer of the second interface, and the AI ​​dialogue interface includes AI dialogue controls and AI auxiliary controls; in response to the operation of the AI ​​auxiliary control, an AI storage interface is displayed on the upper layer of the second interface, and the storage interface includes objects stored by the AI ​​assistant.

[0021] In this embodiment, the AI ​​storage function's launch entry is embedded within the AI ​​conversation function. The terminal device must first launch the AI ​​conversation function to display the AI ​​conversation interface, and further control the display of objects stored by the AI ​​assistant within the AI ​​conversation interface. The AI ​​conversation function is typically used more frequently than the AI ​​storage function. Therefore, through the method provided in this embodiment, the terminal device can prioritize launching the AI ​​conversation function that the user frequently uses, ensuring a concise display interface for these frequently used functions.

[0022] In some embodiments, the operation of starting the AI ​​assistant includes: waking up the AI ​​assistant according to a wake-up word; or, controlling the AI ​​startup control; or, pressing a preset physical button.

[0023] In a third aspect, an embodiment of the present application provides a human-computer interaction method, which is applied to a terminal device provided with an AI assistant, and the method includes: displaying a second interface; in response to a first control operation at the bottom of the second interface, displaying an object stored by the AI ​​assistant on the upper layer of the second interface; in response to an operation of dragging a target object to the second interface, adding the target object to the second interface; wherein the target object is any one or more objects stored by the AI ​​assistant.

[0024] In some embodiments, the method further includes: displaying an AI dialogue interface on an upper layer of the second interface in response to a second control operation at the bottom of the second interface; wherein the first control operation and the second control operation are different.

[0025] In this embodiment, the terminal device sets different activation methods for the AI ​​storage function and the AI ​​conversation function. This method allows users to quickly activate the target AI function as needed without activating other AI functions. For example, when a conversation with an AI assistant is needed, only the AI ​​conversation function is activated, not the AI ​​storage function; or when a target object needs to be retrieved from the AI ​​assistant, only the AI ​​storage function is activated, not the AI ​​conversation function.

[0026] In some embodiments, the first control operation is: a control operation acting on the first control area at the bottom of the second interface; the second control operation is: a control operation acting on the second control area at the bottom of the second interface; wherein the first control area and the second control area do not overlap.

[0027] For example, a prompt bar is displayed at the bottom of the second interface, the first control area is the area where the prompt bar is located, and the second control area is the left area and / or right area of ​​the prompt bar. Alternatively, the first control area is the left area of ​​the prompt bar, and the second control area is the right area of ​​the prompt bar. Alternatively, the first control area is the right area of ​​the prompt bar, and the second control area is the left area of ​​the prompt bar.

[0028] In a fourth aspect, an embodiment of the present application provides a terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method as shown in the first, second, or third aspect above is implemented.

[0029] In a fifth aspect, an embodiment of the present application provides a human-computer interaction device, which includes at least one module, and the at least one module is used to execute the method shown in the first aspect, second aspect or third aspect above.

[0030] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method shown in the first aspect, the second aspect or the third aspect above.

[0031] In the seventh aspect, an embodiment of the present application provides a chip, which includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method shown in the first aspect, the second aspect or the third aspect above is implemented.

[0032] In an eighth aspect, an embodiment of the present application provides a computer program product, which stores a computer program. When the computer program is executed by a processor, it can implement the method shown in the first aspect, the second aspect or the third aspect above.

[0033] It can be understood that the beneficial effects of the fourth to eighth aspects can be found in the relevant descriptions of the first, second or third aspects, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1 is a schematic diagram of a scenario for waking up an AI assistant provided in an embodiment of the present application;

[0035] FIG2 is a schematic diagram of a scenario in which a user controls a terminal device through an AI assistant according to an embodiment of the present application;

[0036] FIG3 is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0037] FIG4 is a schematic flow chart of a human-computer interaction method provided by an embodiment of the present application;

[0038] FIG5 is a schematic diagram of a first interface provided in an embodiment of the present application;

[0039] FIG6 is a schematic diagram of a first AI control and a second AI control provided in an embodiment of the present application;

[0040] FIG7 is a schematic diagram of an AI assistant storing a target object according to a user operation according to an embodiment of the present application;

[0041] FIG8 is a schematic diagram of an AI assistant communicating with a user based on a target object according to an embodiment of the present application;

[0042] FIG9 is a schematic flow chart of a human-computer interaction method provided in another embodiment of the present application;

[0043] FIG10 is a schematic diagram of an AI assistant outputting and using a target object according to a user operation according to an embodiment of the present application;

[0044] FIG11 is a schematic flow chart of a human-computer interaction method provided in another embodiment of the present application;

[0045] FIG12 is a schematic diagram of an AI assistant outputting and using a target object according to a user operation provided by another embodiment of the present application;

[0046] FIG13 is a schematic diagram of a first control area and a second control area provided in an embodiment of the present application;

[0047] FIG14 is a schematic flow chart of a human-computer interaction method provided in yet another embodiment of the present application;

[0048] FIG15 is a schematic diagram of an AI assistant according to an embodiment of the present application outputting and using picture A according to a user operation;

[0049] FIG16 is a schematic diagram of an AI assistant according to an embodiment of the present application providing a function of starting an AI dialogue based on a user operation display;

[0050] FIG17 is a schematic diagram of an AI assistant according to an embodiment of the present application intelligently classifying storage objects;

[0051] FIG18 is a schematic diagram of an AI storage interface provided in an embodiment of the present application;

[0052] 19A to 19C are schematic diagrams of the AI ​​assistant preprocessing a target object according to different embodiments of the present application;

[0053] FIG20 is a schematic diagram of a human-computer interaction device provided in an embodiment of the present application;

[0054] FIG21 is a schematic diagram of the structure of the chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] It should be understood that in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a way to describe the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0056] In this embodiment, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0057] The technical solutions provided in the embodiments of the present application are described below with reference to the accompanying drawings.

[0058] With the development of computer technology, AI assistants such as "Xiaoyi", "YOYO" and "Siri" have gradually appeared on mobile phones, tablet computers and other terminal devices to assist users in human-computer interaction.

[0059] In this embodiment, the AI ​​assistant may also be referred to as a voice assistant, intelligent voice, digital assistant, virtual assistant, etc. It can be understood as an information processing system that can recognize natural language input in the form of voice and / or text to infer user intent and perform corresponding actions based on the inferred user intent. In addition, the system can respond to user input through output forms such as audible (such as voice) and visual (such as text, pictures).

[0060] In order to save power consumption of the terminal device, the AI ​​assistant is usually in a dormant state, and the user can start the AI ​​assistant through specific control operations. Among them, starting the AI ​​assistant can also be called waking up the AI ​​assistant. For example, as shown in Figure 1, the user can start the AI ​​assistant by saying wake-up words such as "Xiaoyi Xiaoyi" and "Hello YOYO"; or, by clicking the AI ​​assistant icon to start the AI ​​assistant; or, by long pressing the power button to start the AI ​​assistant. This embodiment does not limit the method of starting the AI ​​assistant.

[0061] Once activated, the AI ​​assistant can interact with the user. For example, as shown in Figure 2, the AI ​​assistant can control the terminal device to play song A based on the user's voice input of "Play song A." Alternatively, it can plan a route home and provide navigation based on the user's voice input of "Navigate home." Alternatively, it can call the contact Lucy based on the user's voice input of "Call Lucy." As can be seen, the AI ​​assistant significantly improves the convenience of user interaction with terminal devices, providing users with a brand new human-computer interaction experience.

[0062] Currently, AI assistants typically engage in direct conversations with users, either through voice or text. This means the AI ​​assistant receives voice or text input from the user and immediately responds with a corresponding voice, text, or control response. However, as terminal devices become increasingly service-rich, users may have other needs beyond simply communicating with AI assistants. For example, to synchronize context and knowledge with chat friends and enhance communication, users may need to share content with their chat friends. Therefore, users may also need the AI ​​assistant to temporarily store text, images, files, audio and video, and web links while waiting to use the content. Furthermore, users may also need to communicate with the AI ​​assistant based on specific content (such as images, text, or audio). As can be seen, the current forms of human-computer interaction between AI assistants and users are relatively simple. For example, some objects that do not require immediate processing by the AI ​​assistant have no storage space, which cannot meet users' diverse human-computer interaction needs.

[0063] To this end, an embodiment of the present application provides a human-computer interaction method, which is applied to a terminal device. Through this method, the AI ​​assistant in the terminal device can provide a variety of human-computer interaction methods to meet the user's diverse interaction needs, such as helping users to store content, and having conversations with users based on specific content, etc., with a better user experience.

[0064] In the embodiments of the present application, the terminal device may be a mobile phone, a tablet computer, a computer with wireless transceiver function, a smart TV, a projector, a wearable device (such as a smart watch), an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), an in-vehicle computer, or other electronic device equipped with an AI assistant. The embodiments of the present application do not limit the specific type of the terminal device.

[0065] FIG3 is a schematic diagram of the structure of a terminal device 300 provided in an embodiment of the present application. As shown in FIG3 , the terminal device 300 includes a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, an earphone interface 370D, a sensor module 380, a button 390, a motor 391, an indicator 392, a camera 393, a display 394, and a subscriber identification module (SIM) card interface 395.

[0066] It should be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the terminal device 300. In other embodiments of the present application, the terminal device 300 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0067] For example, when the terminal device 300 is a mobile phone or a tablet computer, it may include all the components shown in the figure, or may include only some of the components shown in the figure.

[0068] As an example, when the terminal device 300 is a large-screen device, it can include the processor 310 shown in the figure, the external memory interface 320, the internal memory 321, the universal serial bus (USB) interface 330, the charging management module 340, the power management module 341, the wireless communication module 360, the audio module 370, the speaker 370A, the receiver 370B, the microphone 370C, the camera 393, and the display screen 394.

[0069] The processor 310 may include one or more processing units. For example, the processor 310 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The controller may be the nerve center and command center of the terminal device 300. The controller may generate an operation control signal based on the instruction opcode and timing signal to complete the control of instruction fetching and execution.

[0070] The external memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 200. The external memory card communicates with the processor 310 via the external memory interface 320 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0071] The internal memory 321 can be used to store computer executable program code, which includes instructions. The processor 310 executes various functional applications and data processing of the electronic device 200 by running the instructions stored in the internal memory 321. The internal memory 321 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.). The data storage area may store data created during the use of the electronic device 200 (such as audio data, a phone book, etc.).

[0072] USB interface 330 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. USB interface 330 can be used to connect a charger to charge electronic device 200 and to transfer data between electronic device 200 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as AR devices.

[0073] The charging management module 340 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 340 can receive charging input from the wired charger via the USB interface 330. In some wireless charging embodiments, the charging management module 340 can receive wireless charging input via the wireless charging coil of the terminal device 300. While charging the battery 342, the charging management module 340 can also power the electronic device through the power management module 341.

[0074] The power management module 341 is configured to connect the battery 342, the charging management module 340, and the processor 310. The power management module 341 receives input from the battery 342 and / or the charging management module 340 to provide power to the processor 310, the internal memory 321, the external memory, the display 394, the camera 393, and the wireless communication module 360. The power management module 341 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance), among others.

[0075] The wireless communication function of the terminal device 300 can be implemented through antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, modem processor and baseband processor.

[0076] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 300 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0077] The mobile communication module 350 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for the terminal device 300. The mobile communication module 350 may include at least one filter, a switch, a power amplifier, a low-noise amplifier (LNA), and the like. The mobile communication module 350 can receive electromagnetic waves from antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 350 can also amplify the signals modulated by the modem processor and convert them into electromagnetic waves for radiation via antenna 1.

[0078] In some embodiments, at least some functional modules of the mobile communication module 350 may be provided in the processor 310. In some embodiments, at least some functional modules of the mobile communication module 350 may be provided in the same device as at least some functional modules of the processor 310.

[0079] The wireless communication module 360 ​​can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the terminal device 300. The wireless communication module 360 ​​can be one or more devices that integrate at least one communication processing module. The wireless communication module 360 ​​receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 310. The wireless communication module 360 ​​can also receive the signal to be sent from the processor 310, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0080] The terminal device 300 implements display functions through a GPU, display screen 394, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 310 may include one or more GPUs that execute program instructions to generate or modify display information.

[0081] Display screen 394 is used to display images, videos, etc. Display screen 394 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, terminal device 300 may include one or N display screens 394, where N is a positive integer greater than one.

[0082] The terminal device 300 can implement audio functions through the audio module 370, the speaker 370A, the receiver 370B, the microphone 370C, the headphone jack 370D, and the application processor.

[0083] The audio module 370 is used to convert digital audio signals into analog audio signals for output, and is also used to convert analog audio input into digital audio signals. The audio module 370 can also be used to encode and decode audio signals. In some embodiments, the audio module 370 can be provided in the processor 310, or some functional modules of the audio module 370 can be provided in the processor 310.

[0084] The speaker 370A, also called a "speaker," is used to convert audio electrical signals into sound signals. The terminal device 300 can listen to music or listen to hands-free calls through the speaker 370A. For example, the speaker can play the comparison and analysis results provided in the embodiments of the present application.

[0085] The receiver 370B, also called a "handset", is used to convert audio electrical signals into sound signals. When the terminal device 300 receives a call or voice message, the user can hear the voice by placing the receiver 370B close to the ear.

[0086] Microphone 370C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 370C to input the sound signal into the microphone 370C. The terminal device 300 can be provided with at least one microphone 370C. In other embodiments, the terminal device 300 can be provided with two microphones 370C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the terminal device 300 can also be provided with three, four or more microphones 370C to realize sound signal collection, noise reduction, and identification of sound sources, and realize directional recording function, etc.

[0087] The headphone jack 370D is used to connect a wired headphone. The headphone jack 370D can be a USB interface 330 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0088] The sensor module 380 may include a pressure sensor 280A, a gyroscope sensor 280B, an air pressure sensor 280C, a magnetic sensor 280D, an acceleration sensor 280E, a distance sensor 280F, a proximity light sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, a bone conduction sensor 280M, etc.

[0089] Among them, the pressure sensor 280A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 280A can be set on the display screen 394. There are many types of pressure sensors 280A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a device including at least two parallel plates with conductive material. When a force acts on the pressure sensor 280A, the capacitance between the electrodes changes. The terminal device 300 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 394, the terminal device 300 detects the intensity of the touch operation based on the pressure sensor 280A. The terminal device 300 can also calculate the position of the touch based on the detection signal of the pressure sensor 280A.

[0090] In some embodiments, touch operations applied to the same touch location but with different touch operation intensities may correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than a first pressure threshold is applied to a short message application icon, an instruction to view short messages is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold is applied to a short message application icon, an instruction to create a new short message is executed.

[0091] The touch sensor 280K is also called a "touch panel." The touch sensor 280K can be set on the display screen 394. The touch sensor 280K and the display screen 394 form a touch screen, also called a "touch screen." The touch sensor 280K is used to detect touch operations acting on or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 394. In other embodiments, the touch sensor 280K can also be set on the surface of the terminal device 300, at a different location from the display screen 394.

[0092] The buttons 390 include a power button, a volume button, and the like. The buttons 390 may be mechanical buttons or touch buttons. The terminal device 300 may receive key inputs and generate key signal inputs related to user settings and function control of the terminal device 300.

[0093] Motor 391 can generate vibration prompts. Motor 391 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 394, motor 391 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0094] Indicator 392 can be an indicator light, which can be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0095] SIM card interface 395 is used to connect a SIM card. A SIM card can be connected to and disconnected from terminal device 300 by inserting or removing it from SIM card interface 395. Terminal device 300 may support one or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 395 can support Nano SIM cards, Micro SIM cards, and SIM cards.

[0096] Based on the above terminal device, the human-computer interaction method provided by this embodiment is described in detail below.

[0097] In this embodiment, in addition to being able to conduct basic conversations with users based on the user's natural language input, the AI ​​assistant can also perform the following operations: (1) The AI ​​assistant receives and processes the content input by the user. For example, it helps the user store the content, or has a conversation with the user based on the content. (2) The AI ​​assistant outputs the content stored by the user. In other words, when the user needs to use the content stored by the AI ​​assistant, he can quickly retrieve and use the content from the AI ​​assistant. The following are detailed descriptions of each of them.

[0098] (1) The AI ​​assistant receives and processes user input

[0099] FIG4 is a schematic flow chart of a human-computer interaction method provided in an embodiment of the present application, which specifically includes steps S401 to S404 .

[0100] S401: The terminal device displays a first interface, where the first interface includes a target object.

[0101] In this embodiment, the first interface includes various web pages and application interfaces, such as a browser interface, a gallery interface, a contact interface, a text message interface, an email interface, a Word document interface, a memo interface, a chat interface, a desktop, etc. The target object can be text, images, files, audio and video, web links, and other content in the first interface, and this embodiment does not limit this.

[0102] When the terminal device displays the first interface, the user's interaction needs with the AI ​​assistant may include: (1) asking the AI ​​assistant to temporarily keep the target object; (2) discussing topics with the AI ​​assistant based on the target object.

[0103] Taking the first interface as the gallery interface shown in Figure 5 as an example, the gallery interface displays various pictures and videos stored by the user. In some scenarios, the user needs to discuss a topic with the AI ​​assistant based on a certain picture in the interface, such as asking the AI ​​assistant where picture A was taken. In other scenarios, the user needs the AI ​​assistant to temporarily keep some pictures so that they can be added to other application interfaces later. For example, the user needs to temporarily keep picture A in the gallery interface to the AI ​​assistant, and when switching to the memo interface, quickly obtain picture A from the AI ​​assistant and add it to the memo interface.

[0104] The implementation process of the above interaction requirements (1) and (2) is described in detail below.

[0105] S402, in response to the user's dragging operation on the target object, the terminal device starts the AI ​​assistant and displays the first AI control and / or the second AI control at the bottom of the first interface.

[0106] Since the AI ​​assistant is usually in a dormant state, the terminal device needs to respond to a specific control operation to start the AI ​​assistant. For example, in this embodiment, the terminal device starts the AI ​​assistant in response to the user's drag operation on the target object in the first interface.

[0107] After the AI ​​assistant is started, the terminal device displays the first AI control and / or the second AI control at the bottom of the first interface. Among them, the first AI control is used to control the AI ​​assistant to temporarily store the target object so that the user can quickly use the target object later. The second AI control is used to control the AI ​​assistant to communicate with the user based on the target object. The user can choose to drag the target object to the first AI control or the second AI control according to actual needs, thereby controlling the AI ​​assistant to implement the corresponding function.

[0108] As shown in Figure 6, taking the first interface as the gallery interface and the target object as picture A as an example, in response to the user's drag operation on picture A, the terminal device starts the AI ​​assistant and displays the first AI control "Let AI Keep" and the second AI control "Talk to AI" at the bottom of the first interface. If the user's requirement is to let the AI ​​assistant temporarily store picture A, drag picture A to the area where the first AI control "Let AI Keep" is located and release the drag operation. If the user's requirement is to have a conversation with the AI ​​assistant based on picture A, drag picture A to the area where the second AI control "Talk to AI" is located and release the drag operation.

[0109] In this embodiment, a drag operation can be interpreted as the process in which a user selects a target object through a mouse click, touch, or other control method and controls the movement of the target object on the display interface. The drag operation can visualize user behavior, enhance the user experience, and make the human-computer interaction process more natural. Accordingly, releasing a drag operation can be interpreted as the user ceasing the drag operation. When the drag operation is released at a certain location on the display interface, the target object selected by the user is also released at that location.

[0110] S403: After the target object is dragged to the area where the first AI control is located and the drag operation is released, the terminal device controls the AI ​​assistant to store the target object.

[0111] In this embodiment, in addition to being able to control the movement of the display position of the target object, the drag operation is also extended to define a copy function, that is, after the target object is dragged to the area where the first AI control (such as the "Let AI Keep" control) is located and the drag operation is released, the target object is copied and stored in the AI ​​assistant, and the target object is retained in the original position.

[0112] Optionally, the AI ​​assistant enters a dormant state after storing the target object. Accordingly, the terminal device returns to displaying the first interface, without displaying the first AI control, the second AI control, and other contents.

[0113] As shown in Figures 7 (a) to (c), the first interface is the gallery interface, and the target object is picture A. In response to the user dragging picture A to the area where the "Let AI Store" control is located and releasing the drag operation, the AI ​​assistant stores picture A. After picture A is stored, the terminal device resumes displaying the first interface.

[0114] Through step S403, the terminal device can temporarily store the target object through the AI ​​assistant. Compared with the existing AI assistant, the AI ​​assistant provided in this embodiment can provide richer human-computer interaction methods and have a better user experience.

[0115] S404: After the target object is dragged to the area where the second AI control is located and the drag operation is released, the terminal device controls the AI ​​assistant to communicate with the user based on the target object.

[0116] In this embodiment, after a target object is dragged to the area where a second AI control (such as a "Talk to AI" control) is located and the drag operation is released, the terminal device displays an AI dialogue interface that includes the AI ​​dialogue control and a thumbnail of the target object. Based on this, the user can trigger the AI ​​assistant's dialogue function through the AI ​​dialogue control and have a conversation with the AI ​​assistant regarding the target object.

[0117] As shown in (a) to (c) of Figure 8, taking the first interface as the gallery interface and the target object as picture A as an example, in response to the user dragging picture A to the area where the "Talk to AI" control is located and releasing the drag operation, the terminal device displays the AI ​​dialogue interface, which includes the AI ​​dialogue control (such as the "voice ball") and a thumbnail of picture A. The user can trigger the conversation function of the AI ​​assistant through the "voice ball" control and have a conversation with the AI ​​assistant based on picture A.

[0118] For example, the user can instruct the AI ​​assistant to identify the content in picture A through voice interaction; or, instruct the AI ​​assistant to perform specific processing on picture A, such as identifying text in picture A, adjusting the contrast of picture A, etc.; or, inquire about basic information of picture A, such as the memory size occupied, picture format, shooting time, shooting location, etc.

[0119] It should be noted that, in this embodiment, the dialogue between the AI ​​assistant and the user based on the target object should not be narrowly understood as voice communication, but may also include controlling the AI ​​assistant to process the target object through preset controls.

[0120] For example, as shown in (c) of Figure 8, the AI ​​dialogue interface also includes some preset controls, such as "Smart Image Recognition" and "Similar Images." "Smart Image Recognition" controls the AI ​​assistant's ability to identify and process information within images, such as text extraction, text translation, table extraction, QR code recognition, and object identification. "Similar Images" controls the AI ​​assistant's ability to search for other images similar to the current one locally, on a cloud server, or on a third-party server.

[0121] Through step S404, the terminal device can conduct human-computer interaction with the user based on the target object through the AI ​​assistant, and the interaction content is more specific, detailed and clear, with a better user experience.

[0122] Optionally, while the target object is moving toward the first AI control or the second AI control according to the drag operation, the terminal device may also dynamically adjust the shape of the target object, the first AI control, and the second AI control to present better visual and control effects to the user and improve the user experience.

[0123] In some embodiments, as shown in FIG7(b) or FIG8(b), as the target object (e.g., Image A) moves toward the first or second AI widget in accordance with the drag operation, its size can gradually decrease to a preset size to facilitate accurate dragging of the target object to the area where the first or second AI widget is located. This embodiment does not limit the specific value of the preset size.

[0124] In other embodiments, as shown in (b) of Figure 7 or (b) of Figure 8, the terminal device can also detect the relative position relationship between the target object and the first AI control and the second AI control. When the target object is near the first AI control, the terminal device increases the display size of the first AI control and reduces the display size of the second AI control. Alternatively, when the target object is near the second AI control, the terminal device increases the display size of the second AI control and reduces the display size of the first AI control. Through this method, the user can accurately drag the target object to the area where the target control (i.e., the first AI control or the second AI control) is located.

[0125] To sum up, through the human-computer interaction method provided by this application, the user can control the AI ​​assistant to implement two different AI interaction functions of "talking to AI" and "letting AI keep" based on the target object through a control operation, that is, dragging the target object to the target control. This not only makes the control operation simple, but also can better enrich and cover various scenarios of interaction between people and AI assistants.

[0126] (2) AI Assistant Outputs User-Stored Content

[0127] For content stored in the AI ​​Assistant, users can quickly drag and drop it from the AI ​​Assistant and use the target object as needed. For example, a user can drag and drop a previously stored picture A from the AI ​​Assistant to the chat application dialog box to quickly send picture A to a friend.

[0128] Because AI assistants are configured with different functions, such as AI conversation and AI storage, terminal devices may set up different launch portals for these functions. The following example illustrates the process of users accessing and using target objects from the AI ​​assistant, combining different launch portal configurations.

[0129] (1) AI storage and AI dialogue functions are configured with the same startup entry

[0130] Figure 9 is a schematic flow chart of a human-computer interaction method provided by another embodiment of the present application. Referring to Figure 9 , the method involves a process in which a terminal device drags out and uses a target object from an AI assistant according to a user operation, specifically including steps S901 to S903.

[0131] S901: The terminal device displays a second interface, which may be the same as or different from the first interface shown above.

[0132] S902, in response to the user's operation of starting the AI ​​assistant, the terminal device displays an AI comprehensive interface on the upper layer of the second interface, and the AI ​​comprehensive interface includes AI dialogue controls and objects stored by the AI ​​assistant.

[0133] In this embodiment, when no target object is dragged, the user's control operations for activating the AI ​​assistant include: saying a wake-up word such as "Xiaoyi Xiaoyi," "Hello YOYO," or "Hey Siri"; or clicking the AI ​​assistant icon; or long pressing the power button of the terminal device; or long pressing the prompt bar at the bottom of the second interface. This prompt bar, as shown in Figure 5, is used to prompt the terminal device's control functions, such as the function of activating the AI ​​assistant in this embodiment. After the AI ​​assistant is activated, the terminal device can display the objects stored by the AI ​​assistant.

[0134] In this embodiment, the AI ​​integrated interface is an AI control interface that integrates "AI dialogue" and "AI storage". The AI ​​integrated interface includes AI dialogue controls and objects stored by the AI ​​assistant.

[0135] It should be noted that this embodiment does not limit the way in which the AI ​​assistant displays stored content. In addition to displaying it in the form of thumbnails, it can also be displayed in folders, lists, etc. In addition, when the AI ​​comprehensive interface displays some storage objects, the user can control the terminal device to display other storage objects by sliding the target object left or right or up and down.

[0136] S903: In response to the user's operation of dragging the target object to the second interface, the terminal device adds the target object to the second interface.

[0137] In this embodiment, the target object dragged by the user can be any one of the objects stored in the AI ​​assistant, or any multiple ones, which is not limited in this embodiment.

[0138] For S901 to S903, refer to (a) to (d) in Figure 10. Taking the second interface as a memo interface as an example, in response to the user's long press operation on the prompt bar at the bottom of the memo interface, the terminal device displays the AI ​​comprehensive interface. Exemplarily, the AI ​​comprehensive interface is a control panel located at the bottom of the second interface, and the control panel includes: an AI dialogue control "voice ball", which is used to control the AI ​​assistant to communicate with the user. In addition, the AI ​​comprehensive interface also includes objects stored by the AI ​​assistant. Among them, the objects stored by the AI ​​assistant are presented in full or in part in the control panel in the form of thumbnails for user selection. In response to the user dragging picture A to the memo interface, the terminal device inserts picture A into the memo interface.

[0139] (2) The startup entry of the AI ​​storage function is embedded in the AI ​​dialogue function

[0140] Figure 11 is a schematic flow chart of a human-computer interaction method provided by another embodiment of the present application. Referring to Figure 11 , the method involves a process in which a terminal device drags out and uses a target object from an AI assistant according to a user operation, specifically including steps S1101 to S1104.

[0141] S1101: The terminal device displays a second interface, which may be the same as or different from the first interface shown above.

[0142] S1102, in response to the user's operation of starting the AI ​​assistant, the terminal device displays an AI dialogue interface on the upper layer of the second interface, and the AI ​​dialogue interface includes AI dialogue controls and AI auxiliary controls.

[0143] Among them, the operation of starting the AI ​​assistant is specifically referred to in S902 and will not be repeated here.

[0144] S1103, in response to the user's operation on the AI ​​auxiliary control, the terminal device displays the AI ​​storage interface on the upper layer of the second interface, and the AI ​​storage interface includes objects stored by the AI ​​assistant.

[0145] S1104: In response to the user's operation of dragging the target object to the second interface, the terminal device adds the target object to the second interface.

[0146] For S1101 to S1104, refer to (a) to (d) in Figure 12. Taking the second interface as a memo interface as an example, in response to the user's long press operation on the prompt bar at the bottom of the second interface, the terminal device displays an AI dialogue interface, which includes the AI ​​dialogue control "Voice Ball" and the AI ​​auxiliary control "Xiao Yi Bang Ji". Among them, the "Voice Ball" is used to control the AI ​​assistant to start a conversation with the user, and the "Xiao Yi Bang Ji" is used to control the terminal device to display the AI ​​storage interface. In response to the user's control operation on the "Xiao Yi Bang Ji" control, the terminal device displays the AI ​​storage interface, which displays all or part of the content stored by the AI ​​assistant in the form of thumbnails. In response to the user's operation of dragging picture A to the memo interface, the terminal device inserts picture A into the memo interface.

[0147] (3) Different startup entries are set for the AI ​​storage function and the AI ​​dialogue function

[0148] In this embodiment, the terminal device activates different functions of the AI ​​assistant according to different control methods. For example, the terminal device activates the AI ​​conversation function and displays the AI ​​conversation interface according to the first control operation; and activates the AI ​​storage function and displays the AI ​​storage interface according to the second control operation.

[0149] In some embodiments, the first control operation is a control operation applied to the first control area of ​​the second interface, and the second control operation is a control operation applied to the second control area of ​​the second interface. The control operation may be a click operation, a long press operation, or the like. The first control area and the second control area may be two different hotspots on the screen of the terminal device, i.e., areas that a user frequently touches to control the terminal device. The first control area and the second control area may not overlap, and this embodiment does not restrict the locations of the first control area and the second control area.

[0150] For example, as shown in FIG13 , the first control area is the middle area at the bottom of the second interface, such as the area where the prompt bar is located; the second control area is the left and / or right area at the bottom of the second interface, such as the left and / or right area of ​​the prompt bar. Alternatively, the first control area is the left area at the bottom of the second interface, such as the left area of ​​the prompt bar; the second control area is the right area at the bottom of the second interface, such as the right area of ​​the prompt bar.

[0151] In other embodiments, the first control operation and the second control operation have different control gestures. For example, the first control operation is a touch-swipe operation from left to right at the bottom of the display interface, and the second control operation is a touch-swipe operation from right to left at the bottom of the display interface. Of course, other control gestures are also possible, and this embodiment is not limited thereto.

[0152] The method provided in this embodiment is described below with reference to specific examples.

[0153] Figure 14 is a schematic flow chart of a human-computer interaction method provided by another embodiment of the present application. The method involves a process in which a terminal device drags out and uses a target object from an AI assistant according to a user operation, specifically including steps S1401 to S1404.

[0154] S1401: The terminal device displays a second interface, which is the same as or different from the first interface shown above.

[0155] S1402: In response to the user's first control operation at the bottom of the second interface, the terminal device displays an AI storage interface on the upper layer of the second interface, where the AI ​​storage interface includes objects stored by the AI ​​assistant.

[0156] S1403: In response to the user's operation of dragging the target object to the second interface, the terminal device adds the target object to the second interface.

[0157] Regarding S1402 to S1403, see (a) to (c) in FIG15 . For example, taking the target object as picture A and the second interface as the memo interface, in response to the user's long press operation on the first control area at the bottom of the second interface, the terminal device displays the AI ​​storage interface. The AI ​​storage interface displays all or part of the objects stored by the AI ​​assistant, including picture A. In response to the user dragging picture A from the AI ​​storage interface to the memo interface, the terminal device displays picture A on the memo interface.

[0158] S1404: In response to the user's second control operation at the bottom of the second interface, the terminal device displays the AI ​​dialogue interface.

[0159] Regarding S1404, as shown in (a) and (b) of Figure 16, in response to the user's long press operation on the second control area in the second interface, the terminal device displays the AI ​​dialogue interface. The AI ​​dialogue interface includes an AI dialogue control (such as a "voice ball"), which is used to control the AI ​​assistant to start a voice dialogue with the user.

[0160] It should be noted that the process of dragging the target object from the AI ​​assistant shown in S1402 to S1403 and the process of communicating with the assistant shown in S1404 are independent of each other, have no order requirements, and have no dependency relationship between them.

[0161] In summary, through the human-computer interaction method provided in this embodiment, the AI ​​assistant can not only interact with the user based on the target object, but also help the user temporarily store content for subsequent use. It can be seen that the human-computer interaction function of the AI ​​assistant is more abundant, which can meet the diverse usage needs of users and provide a better user experience.

[0162] Optionally, in each of the above embodiments, in response to the user's operation of dragging the target object from the AI ​​assistant, the terminal device can copy the target object to the current display interface and continue to retain the target object stored in the AI ​​assistant; in addition, the terminal device can also move the target object to the current display interface and delete the target object stored in the AI ​​assistant. This embodiment does not limit this.

[0163] In addition, the content stored in the AI ​​assistant can be used not only across applications, but also across devices. For example, for the first terminal device and the second terminal device that are logged in to the same device account, the AI ​​assistants they use are essentially the same. After the AI ​​assistant of the first terminal device obtains the target object stored by the user, it can synchronize the target object to the AI ​​assistant of the second terminal device through wireless communication methods such as a server, Bluetooth, or wireless fidelity (Wi-Fi). Based on this, the AI ​​assistant of the second terminal device can control the use of the target object according to the user's drag operation. Please refer to the previous text for details and I will not repeat them here.

[0164] To further improve the user experience, the AI ​​assistant can also analyze the objects stored by the user and intelligently classify them based on the analysis results to facilitate user search.

[0165] In some embodiments, the AI ​​assistant can intelligently categorize its stored objects by time, type, and topic, making it easier for users to find content. It's worth noting that categorizing the stored objects by topic allows for better human-computer interaction in later stages, such as quickly retrieving the stored content a user needs or more deeply analyzing the user's intent based on the stored objects.

[0166] As shown in Figures (a) to (d) of Figure 17, the AI ​​storage interface, "Xiao Yi Bang Ji," displays objects stored by the AI ​​assistant and a toggle control. In response to a user's operation on the toggle control, the terminal device displays different classification controls on the AI ​​storage interface, such as "Classify by Time," "Classify by Type," and "Classify by Topic." These different classification controls control how the AI ​​assistant displays its stored objects. For example, in response to a user's operation on the "Classify by Topic" control, the AI ​​assistant categorizes stored objects by topic, storing objects of the same topic in the same folder and displaying the number of objects stored in that folder. Exemplary topic categories include "cooking," "fitness," "travel," "games," and "music." In response to a user's selection of a target topic, the terminal device displays objects related to that topic on the AI ​​storage interface. For example, in response to a user's selection of the "cooking" topic, the terminal device displays cooking-related objects stored by the AI ​​assistant, such as text, images, audio, and video.

[0167] In addition, the AI ​​assistant can also pre-process the content of the target object to improve the user experience. Exemplarily, this pre-processing includes extracting key information of the target object, such as keywords, phone numbers, ID numbers, summaries, event information, etc. from the text; extracting text information, character information, location information, scene information, and keywords from pictures and videos; and generating video cover images and core video clips. This embodiment does not limit the pre-processing method.

[0168] Taking the AI ​​storage interface of the terminal device as the "Xiao Yi Bang Ji" interface as an example, as shown in Figure 18, the AI ​​storage interface displays the objects stored by the AI ​​assistant and the pre-processing control. Exemplarily, the pre-processing control is the "Intelligent Processing" control in Figure 18. The pre-processing control can control the AI ​​assistant to pre-process the target object selected by the user.

[0169] For example, as shown in Figure 19A, after the AI ​​assistant detects that the user has selected the target text and clicks the "Intelligent Processing" control, the AI ​​assistant processes the text, extracts the phone number in the text, and displays the relevant controls for adding the phone number to the address book.

[0170] For example, as shown in Figure 19B, after the AI ​​assistant detects that the user has selected the target text and clicks the "Intelligent Processing" control, the AI ​​assistant processes the text, extracts the ID number from the text, and displays the relevant controls to copy the ID number.

[0171] For example, as shown in Figure 19C, after the AI ​​assistant detects that the user has selected Attachment 1 and clicks the "Intelligent Processing" control, the AI ​​assistant can pre-process Attachment 1 and generate a summary of Attachment 1, etc., so that the user can quickly understand the main information of Attachment 1.

[0172] In some embodiments, the AI ​​assistant can also store the pre-processing results of each object to form an AI memory to facilitate subsequent human-computer interaction with the user. For example, the AI ​​assistant can intelligently generate chat topics based on the AI ​​memory and conduct a conversation with the user.

[0173] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0174] Based on the same concept, as an implementation of the above method, an embodiment of the present application provides a human-computer interaction device, and the device embodiment corresponds to the aforementioned method embodiment. For ease of reading, the present device embodiment will no longer repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspond to and implement all the contents of the aforementioned method embodiment.

[0175] 20 is a schematic diagram of a human-computer interaction device 2000 according to an embodiment of the present application. The human-computer interaction device 2000 includes an input module 2001 , a display module 2002 , and a processing module 2003 .

[0176] Input module 2001 is used to receive user input on the display interface of the terminal device, such as touch input, voice input, gesture input, etc. Input module 2001 is used to support the terminal device in executing the steps of receiving user operations in the above embodiments and / or other processes used in the technology described herein. The input module can be a touch screen or other hardware or a combination of hardware and software.

[0177] The display module 2002 is used to control the terminal device to perform the interface display operations involved in the above embodiments and / or other processes used in the technology described herein. The display module can be a touch screen or other hardware or a combination of hardware and software.

[0178] Processing module 2003 is used to control the terminal device to perform the control operations involved in the above embodiments. For example, it can start the AI ​​assistant based on the user operation. Alternatively, after the target object is dragged to the area where the first AI control is located and the drag operation is released, it can control the AI ​​assistant to store the target object. Alternatively, it can control the AI ​​assistant to communicate with the user based on the target object.

[0179] An embodiment of the present application also provides a terminal device, which is configured to execute the human-computer interaction method shown in the above embodiments.

[0180] An embodiment of the present application also provides a chip, as shown in Figure 21, which includes a processor and a memory. The memory stores a computer program, and when the computer program is executed by the processor, the human-computer interaction method in the above embodiments is implemented.

[0181] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the human-computer interaction method provided in the above embodiments is implemented.

[0182] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by an electronic device, the electronic device implements the human-computer interaction method provided in the above embodiments.

[0183] It should be understood that the processor mentioned in the embodiments of the present application may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0184] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0185] In the embodiments provided in this application, the division of each framework or module is only a logical functional division. In actual implementation, there may be other division methods, for example, multiple frameworks or modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the functional modules in the various embodiments of this application can be integrated into a processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0186] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0187] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0188] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A human-computer interaction method, characterized in that, Applied to a terminal device equipped with an artificial intelligence (AI) assistant, the method includes: Display a first interface, where the first interface includes a target object; In response to a drag operation on the target object, start the AI assistant and display a first AI control at the bottom of the first interface; After the target object is dragged to the area where the first AI control is located and the drag operation is released, control the AI assistant to store the target object.

2. The method according to claim 1, wherein After the AI assistant is started, a second AI control is also displayed at the bottom of the first interface; the second AI control is used to control the AI assistant to have a conversation with the user based on the target object after the target object is dragged to the area where the second AI control is located and the drag operation is released.

3. The method according to claim 2, wherein During the process of dragging the target object, the method further includes: Shrink the display size of the target object while the target object is moving towards the first AI control or the second AI control.

4. The method according to claim 2 or 3, characterized in that, During the process of dragging the target object, the method further includes: Determine the positional relationship between the dragged target object and the first AI control and the second AI control; When the dragged target object is near the first AI control, increase the display size of the first AI control and shrink the display size of the second AI control; and / or, When the dragged target object is near the second AI control, increase the display size of the second AI control and shrink the display size of the first AI control.

5. The method according to any one of claims 1 to 4, characterized in that The target object includes at least one of a picture, text, file, audio / video, and web link.

6. A human-computer interaction method, characterized in that, Applied to a terminal device equipped with an artificial intelligence (AI) assistant, the method includes: Display a second interface; In response to an operation to start the AI assistant, display the objects stored by the AI assistant on top of the second interface; In response to an operation of dragging a target object to the second interface, add the target object to the second interface; where the target object is any one or more of the objects stored by the AI assistant.

7. The method according to claim 6, characterized in that, In response to an operation to start the AI assistant, displaying the objects stored by the AI assistant on top of the second interface includes: In response to an operation to start the AI assistant, display an AI comprehensive interface on top of the second interface, where the AI comprehensive interface includes an AI conversation control and the objects stored by the AI assistant.

8. The method according to claim 6, characterized in that, In response to an operation to start the AI assistant, displaying the objects stored by the AI assistant on top of the second interface includes: In response to an operation to start the AI assistant, display an AI conversation interface on top of the second interface, where the AI conversation interface includes an AI conversation control and an AI auxiliary control; In response to an operation on the AI auxiliary control, display an AI storage interface on top of the second interface, where the storage interface includes the objects stored by the AI assistant.

9. The method according to any one of claims 6 to 8, characterized in that The operation to start the AI assistant includes: An operation to wake up the AI assistant according to a wake-up word; or, A control operation on an AI start control; or, A pressing operation on a preset physical button.

10. A human-computer interaction method, characterized in that, Applied to a terminal device equipped with an artificial intelligence (AI) assistant, the method includes: Display a second interface; In response to a first control operation at the bottom of the second interface, display the objects stored by the AI assistant above the second interface; In response to a drag-and-drop operation of a target object onto the second interface, add the target object to the second interface; wherein the target object is any one or more of the objects stored by the AI assistant.

11. The method according to claim 10, characterized in that, The method further includes: In response to a second control operation at the bottom of the second interface, display an AI dialogue interface above the second interface; wherein the first control operation and the second control operation are different.

12. The method according to claim 11, wherein: The first control operation is a control operation acting on a first control area at the bottom of the second interface; The second control operation is a control operation acting on a second control area at the bottom of the second interface; wherein the first control area and the second control area do not overlap.

13. The method according to claim 12, wherein A prompt bar is displayed at the bottom of the second interface, The first control area is the area where the prompt bar is located; The second control area is the left area and / or the right area of the prompt bar.

14. A terminal device, characterized in that, Comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program, it implements the method according to any one of claims 1 to 5, or the method according to any one of claims 6 to 9, or the method according to any one of claims 10 to 13.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, when the computer program is executed by a processor, it implements the method according to any one of claims 1 to 5, or the method according to any one of claims 6 to 9, or the method according to any one of claims 10 to 13.

16. A chip, characterized in that, The chip includes a processor and a memory, the memory stores a computer program, when the computer program is executed by the processor, it implements the method according to any one of claims 1 to 5, or the method according to any one of claims 6 to 9, or the method according to any one of claims 10 to 13.

Images