Human-computer interaction device and human-computer interaction method

Abstract

The embodiment of the invention provides a man-machine interaction device and a man-machine interaction method, the man-machine interaction device has a plurality of interaction modes, and the man-machine interaction device comprises a detection unit used for detecting a first event; the processing unit is used for determining feedback to the first event according to a current first interaction mode of the man-machine interaction device, the first interaction mode is one of a plurality of interaction modes, and the plurality of interaction modes further comprise a second interaction mode, the feedback priority of the first event in the second interaction mode is different from the feedback priority of the first event in the first interaction mode. The man-machine interaction device is diversified and flexible in interaction form, and user experience can be improved.

Description

Human-computer interaction device and human-computer interaction method Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and more specifically, to a human-computer interaction device and a human-computer interaction method. Background Art

[0002] With the continuous development of artificial intelligence technology, there are more and more types of robots. Among them, home robots are a relatively important type of robots. They can increase users' happiness and reduce their stress through interaction with users.

[0003] However, the interactions between existing home robots and users are all pre-written and no longer change. That is, existing home robots can only perform simple human-computer interactions in a mechanical manner for the same event, and the interaction form is single and not flexible enough, resulting in a poor user experience.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a human-computer interaction method and a human-computer interaction device. The interaction forms of the human-computer interaction device are diverse and flexible, which helps to improve user experience.

[0006] In a first aspect, a human-computer interaction device is provided, which has multiple interaction modes, and the human-computer interaction device includes: a detection unit for detecting a first event; a processing unit for determining feedback on the first event based on the current first interaction mode of the human-computer interaction device, the first interaction mode is one of the multiple interaction modes, and the multiple interaction modes also include a second interaction mode, wherein the feedback priority of the first event in the second interaction mode is different from the feedback priority of the first event in the first interaction mode.

[0007] In the embodiment of the present application, since the human-computer interaction device has multiple interaction modes, and the feedback priority of the same event (such as the first event) in different interaction modes is different, the human-computer interaction device has various feedback methods for the same event. Then, when the human-computer interaction device detects the first event, the feedback method for the first event can be flexibly selected according to the current interaction mode. Therefore, compared to the existing solution that uses the same feedback for the same event in different interaction modes, the embodiment of the present application can use different feedback for the same event in different interaction modes, which helps to improve the user experience.

[0008] In combination with the first aspect, in some implementations of the first aspect, the detection unit is further used to detect a second event; the processing unit is further used to determine feedback for the second event based on the first interaction mode, wherein the feedback priority of the second event in the first interaction mode is higher than the feedback priority of the first event in the first interaction mode, and the feedback for the second event is a normal response to the second event, and the feedback for the first event is no response or a weak response to the first event.

[0009] The human-computer interaction device can, based on its current interaction mode, determine whether to respond to a first event with a lower feedback priority (relative to a second event) as a non-response or a weak response, and to respond to a second event with a higher feedback priority (relative to the first event) as a normal response. In other words, the human-computer interaction device can determine whether to respond to an event normally, or to respond to an event with no or a weak response, based on the event's feedback priority in the current interaction mode, thereby further improving the user experience.

[0010] In combination with the first aspect, in certain implementations of the first aspect, the first interaction mode is a humanoid mode, and the second event is a touch event, a face event, or a gesture event.

[0011] In humanoid mode, the device responds normally to touch events, face events, or gesture events, but does not respond or responds weakly to other events. This allows the human-computer interaction device to provide feedback that better matches the user's behavior, improving the interaction between the human-computer interaction device and the user, and further enhancing the user experience.

[0012] In combination with the first aspect, in certain implementations of the first aspect, the first interaction mode is a face mode, and the second event is a touch event or a gesture event.

[0013] In face mode, it responds normally to touch events or gesture events, and does not respond or responds weakly to other events. This allows the human-computer interaction device to provide feedback that is more closely aligned with the user's behavior, improving the interaction between the human-computer interaction device and the user, and further enhancing the user experience.

[0014] In combination with the first aspect, in some implementations of the first aspect, the first interaction mode is an application control mode, and the second event is an application manipulation event.

[0015] In application control mode, the device responds normally to application manipulation events and does not respond or responds weakly to other events. This allows the human-computer interaction device to provide feedback that better matches user behavior, improving the interaction between the human-computer interaction device and the user, and further enhancing the user experience.

[0016] In combination with the first aspect, in some implementations of the first aspect, the first interaction mode is a voice control mode, and the second event is a voice input event.

[0017] In voice control mode, the device responds normally to voice input and does not respond or responds weakly to other events. This allows the human-computer interaction device to provide feedback that is more closely aligned with the user's behavior, improving the interaction between the human-computer interaction device and the user, and further enhancing the user experience.

[0018] In combination with the first aspect, in some implementations of the first aspect, the first interaction mode is a desktop mode, and the second event is a desktop edge event.

[0019] In desktop mode, the device responds normally to desktop edge events and does not respond or responds weakly to other events. This allows the human-computer interaction device to provide feedback that is more consistent with the external environment, improving the interaction effect between the human-computer interaction device and the external environment, and helping to further enhance the user experience.

[0020] In combination with the first aspect, in some implementations of the first aspect, the first interaction mode is a standby mode, and in the first interaction mode, feedback priorities of all events are the same.

[0021] In standby mode, all events have the same feedback priority. This allows the device to use the same feedback method for each event when the device is unclear about the user's behavioral intent or the external environment. This improves the interaction between the device and the user or the external environment, further enhancing the user experience.

[0022] In combination with the first aspect, in certain implementations of the first aspect, each of the interaction modes includes at least one interaction state, and the processing unit is further specifically used to determine feedback on the first event based on the first interaction mode and the first interaction state under the first interaction mode.

[0023] The event feedback method is set to be related to the current interaction mode of the human-computer interaction device and the interaction state in the current interaction mode. This can improve the accuracy of the human-computer interaction device's analysis of the user's behavioral intention or external environment, improve the interaction effect between the user and the human-computer interaction device, and help further enhance the user experience.

[0024] In a second aspect, a human-computer interaction method is provided, which is applied to a human-computer interaction device, and the human-computer interaction device has multiple interaction modes. The human-computer interaction method includes: detecting a first event; determining feedback for the first event based on the current first interaction mode of the human-computer interaction device, the first interaction mode being one of the multiple interaction modes, and the multiple interaction modes also including a second interaction mode, wherein the feedback priority of the first event in the second interaction mode is different from the feedback priority of the first event in the first interaction mode.

[0025] In combination with the second aspect, in certain implementations of the second aspect, the method further includes: detecting a second event; determining feedback for the second event based on the first interaction mode, wherein the feedback priority of the second event in the first interaction mode is higher than the feedback priority of the first event in the first interaction mode, and the feedback for the second event is a normal response to the second event, and the feedback for the first event is no response or a weak response to the first event.

[0026] In combination with the second aspect, in certain implementations of the second aspect, the first interaction mode is a humanoid mode, and the second event is a touch event, a face event, or a gesture event.

[0027] In combination with the second aspect, in certain implementations of the second aspect, the first interaction mode is a face mode, and the second event is a touch event or a gesture event.

[0028] In combination with the second aspect, in some implementations of the second aspect, the first interaction mode is an application control mode, and the second event is an application manipulation event.

[0029] In combination with the second aspect, in some implementations of the second aspect, the first interaction mode is a voice control mode, and the second event is a voice input event.

[0030] In combination with the second aspect, in some implementations of the second aspect, the first interaction mode is a desktop mode, and the second event is a desktop edge event.

[0031] In combination with the second aspect, in some implementations of the second aspect, the first interaction mode is a standby mode, and in the first interaction mode, feedback priorities of all events are the same.

[0032] In combination with the second aspect, in certain implementations of the second aspect, each of the interaction modes includes at least one interaction state, and determining the feedback for the first event based on the current first interaction mode of the human-computer interaction device includes: determining the feedback for the first event based on the first interaction mode and the first interaction state under the first interaction mode.

[0033] The technical effects of any possible implementation method of the second aspect can refer to the technical effects of the corresponding implementation method of the first aspect, and will not be repeated here.

[0034] In a third aspect, a human-computer interaction device is provided, comprising: one or more processors; one or more memories; and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions, which, when executed by the one or more processors, enable the human-computer interaction device to execute the human-computer interaction method described in the second aspect and any possible implementation of the second aspect.

[0035] In a fourth aspect, a computer-readable storage medium is provided, comprising computer instructions. When the computer instructions are executed on a human-computer interaction device, the human-computer interaction device executes the human-computer interaction method described in the above-mentioned second aspect and any possible implementation of the second aspect.

[0036] In the fifth aspect, a chip is provided, comprising at least one processor and an interface circuit, wherein the interface circuit is used to provide program instructions or data to the at least one processor, and the at least one processor is used to execute the program instructions to implement the human-computer interaction method described in the above-mentioned second aspect and any possible implementation method of the second aspect.

[0037] In the sixth aspect, a computer program product is provided, which includes computer instructions; when part or all of the computer instructions are run on a computer, the human-computer interaction method described in the above-mentioned second aspect and any possible implementation method of the second aspect is executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] FIG1 is a schematic diagram of the hardware structure of a human-computer interaction device provided in an embodiment of the present application.

[0039] FIG2 is a schematic flow chart of a human-computer interaction method provided in an embodiment of the present application.

[0040] FIG3 is a schematic structural diagram of an example of a human-computer interaction device provided in an embodiment of the present application.

[0041] FIG4 is a schematic structural diagram of another human-computer interaction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0043] In the description of the embodiments of this application, unless otherwise specified, " / " represents or. For example, A / B can represent A or B. "And / or" in this article is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application, "plurality" or "a plurality" means two or more than two.

[0044] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0045] For example, FIG1 shows a schematic diagram of the structure of a human-computer interaction device 100 provided in an embodiment of the present application.

[0046] For example, as shown in Figure 1, the human-computer interaction device 100 may include a processor 110, an actuator 111, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna, a wireless communication module 150, a sensor module 160, an audio module 170, a speaker 170A, a microphone 170B, a camera 180, a display screen 190, etc.

[0047] It is understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the human-computer interaction device 100. In other embodiments of the present application, the human-computer interaction device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0048] The processor 110 may include one or more processing units. For example, the processor 110 may include a graphics processing unit (GPU), a controller, a memory, etc. Different processing units may be independent devices or integrated into one or more processors.

[0049] The controller may be the nerve center and command center of the human-computer interaction device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0050] Memory is used to store instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids repeated accesses, reduces processor 110 latency, and thus improves system efficiency.

[0051] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, and / or a USB interface. The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. The PCM interface can also be used for audio communication to sample, quantize, and encode analog signals. In some embodiments, the audio module 170 and the wireless communication module 150 can be coupled via a PCM bus interface. The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 150. The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 190 and the camera 180. The GPIO interface can be configured through software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 180, the display 190, the wireless communication module 150, the sensor module 160, the audio module 170, etc.

[0052] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely illustrative and does not constitute a structural limitation on the human-computer interaction device 100. In other embodiments of the present application, the human-computer interaction device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0053] The actuator 111 is used to control the human-computer interaction device 100 to move, rotate, jump, etc. Optionally, in some embodiments, if the human-computer interaction device 100 includes ears, a torso, and legs, the actuator 111 is also used to control the torso to rotate relative to the legs, the legs to rotate relative to the torso, the torso to shake, or the ears to rotate along the torso. Optionally, in some embodiments, the actuator 111 may include at least one motor.

[0054] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the human-computer interaction device 100 .

[0055] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the human-computer interaction device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data (such as audio data, etc.) created during the use of the human-computer interaction device 100. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0056] The USB interface 130 is an interface that complies with USB standards, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the human-computer interaction device 100, and can also be used to transmit data between the human-computer interaction device 100 and peripheral devices.

[0057] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through the wireless charging coil of the human-computer interaction device 100. While the charging management module 140 charges the battery 142, it can also power the electronic device through the power management module 141. The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110.

[0058] The wireless communication module 150 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), etc. applied to the human-computer interaction device 100.

[0059] In some embodiments, the antenna of the human-computer interaction device 100 is coupled to the wireless communication module 150 , so that the human-computer interaction device 100 can communicate with a network and other devices through wireless communication technology.

[0060] The sensor module 160 may include at least one sensor. For example, the sensor module 160 may include a touch sensor, a distance sensor, a posture sensor, and the like. In some embodiments, the touch sensor is a capacitive sensor that can be positioned on the top, neck, back, abdomen, or other locations of the human-computer interaction device to sense user interactions such as stroking and tapping. The distance sensor is used to measure the distance between the human-computer interaction device and external objects or the user. The posture sensor is a gyroscope that is used to sense changes in the posture of the human-computer interaction device.

[0061] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110. The speaker 170A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The microphone 170B, also known as the "microphone" or "microphone", is used to convert sound signals into electrical signals.

[0062] The human-computer interaction device 100 can implement audio functions such as voice playback and recording through the audio module 170 , the speaker 170A, the microphone 170B, and the processor 110 .

[0063] The camera 180 is used to capture static images or videos so that the processor 110 can detect events based on the images or videos acquired by the camera 180 , thereby providing feedback on the events.

[0064] The display screen 190 is used to display images, videos, etc. In some embodiments, the display screen 190 can display facial expression animations to express the current emotional state of the human-computer interaction device.

[0065] FIG2 is a schematic flowchart of a human-computer interaction method 200 provided in an embodiment of the present application.

[0066] The human-computer interaction method 200 shown in FIG. 2 can be applied to the human-computer interaction device 100 shown in FIG. 1 .

[0067] For example, as shown in FIG2 , method 200 includes S210 and S220 , where S220 is performed after S210 . S210 and S220 are described in detail below.

[0068] S210: Detect a first event.

[0069] The events involved in the embodiments of the present application can be understood as external environment change events of the human-computer interaction device.

[0070] Exemplarily, events may include but are not limited to obstacle events, face events, touch events, gesture events, application manipulation events, voice input events, desktop edge events, human shape events, pick-up events, fall events, etc.

[0071] Exemplarily, the human-computer interaction device may include a detection unit (such as a distance sensor, a camera, etc.), and the human-computer interaction device determines whether an obstacle event occurs through the detection unit.

[0072] For example, the human-computer interaction device will periodically send a detection signal through a distance sensor. Based on the signal reflected back from the detection signal received by the human-computer interaction device, and / or combined with the static image or video captured by the camera, the human-computer interaction device can detect whether there is an obstacle in the external environment. At this time, the human-computer interaction device has detected an obstacle event.

[0073] Exemplarily, the human-computer interaction device may include a camera, which collects images or videos through the camera, and determines whether there is a face event based on the collected images or videos.

[0074] For example, the human-computer interaction device may select target rectangular regions in the image from left to right and from top to bottom, using these as observation windows. Features are then extracted from the image region corresponding to each observation window, and the extracted features are determined to be features corresponding to a human face. If the extracted features match features corresponding to a human face, it is assumed that a human face exists in the external environment of the human-computer interaction device, and the human-computer interaction device has detected a face event. The size of the target rectangular region may be determined based on the statistically determined sizes of multiple human faces.

[0075] Exemplarily, the human-computer interaction device may include a touch sensor, and the human-computer interaction device determines whether a touch event occurs through a signal detected by the touch sensor.

[0076] For example, if the touch sensor is a capacitive sensor, and a capacitance signal change is detected during a certain event, and the capacitance signal is greater than a threshold value 1, then it can be considered that the user has touched the human-computer interaction device, and the human-computer interaction device has detected the touch event. The threshold value 1 can be determined based on multiple capacitance signals corresponding to multiple touch events.

[0077] Exemplarily, the human-computer interaction device may capture images or videos through a camera, and determine whether there is a gesture event based on the captured images or videos.

[0078] For example, the human-computer interaction device first extracts features from the image, and then matches the features in the image with preset gesture features (which may be stored in the human-computer interaction device). If the features in the image match the preset gesture features, then it can be considered that the user is making a gesture to the human-computer interaction device, and the human-computer interaction device has detected the gesture event.

[0079] Exemplarily, the human-computer interaction apparatus may determine whether there is an application manipulation event based on controls associated with the application.

[0080] For example, when the human-computer interaction device receives an operation instruction from the user to manipulate a control related to an application, the human-computer interaction device detects an application manipulation event.

[0081] For example, the human-computer interaction device may include a microphone, and the human-computer interaction device may determine whether there is a voice input event based on whether the microphone captures sound in the external environment.

[0082] For example, when the microphone of the human-computer interaction device acquires the sound in the external environment, the human-computer interaction device detects a voice input event.

[0083] Exemplarily, a plurality of light sensors may be provided at the bottom of the human-computer interaction device, so that whether a desktop edge event occurs may be determined based on the light intensity collected by each light sensor.

[0084] For example, if one of the multiple light sensors in a human-computer interaction device collects light intensities less than threshold 3 and another collects light intensities greater than threshold 4, the human-computer interaction device can be considered to be at the edge of the desktop, and the human-computer interaction device has detected a desktop edge event. Threshold 3 can be determined based on the light intensities collected by the light sensors when the human-computer interaction device is stationary on the desktop with its bottom parallel to the desktop, and when the human-computer interaction device is walking on the desktop. Threshold 4 can be determined based on the light intensities collected by the light sensors when the human-computer interaction device is walking on the desktop.

[0085] Exemplarily, the human-computer interaction device may capture images or videos through a camera, and determine whether there is a human-shaped event based on the captured images or videos.

[0086] For example, the human-computer interaction device first extracts features from the image and then matches the features in the image with human joint features and human torso features (which may be stored in the human-computer interaction device). If the features in the image match the preset hand joint and torso features, then it can be assumed that a user is present in the external environment of the human-computer interaction device and that the user is not very close to the human-computer interaction device. In this case, the human-computer interaction device has detected the human event. The human joint features and human torso features can be determined based on multiple human joint features and human torso features.

[0087] S220: Determine feedback for the first event according to the current first interaction mode of the human-computer interaction device.

[0088] Optionally, in some embodiments, in order to improve the interaction efficiency between the user and the human-computer interaction device, the interaction mode of the human-computer interaction device may be set to multiple interaction modes.

[0089] Exemplarily, the interaction modes of the human-computer interaction device may include but are not limited to standby mode, human shape mode, face mode, application control mode, voice control mode, desktop mode, etc.

[0090] The first interaction mode is one of multiple interaction modes, and which interaction mode among the multiple interaction modes is determined based on the current external environment of the human-computer interaction device. For example, the human shape mode may be the interaction mode of the human-computer interaction device when it detects the presence of a human shape in its external environment. The face mode may be the interaction mode of the human-computer interaction device when it detects the presence of a close-up human face in its external environment. The application control mode may be the interaction mode of the human-computer interaction device when it detects that an application is being manipulated. The voice control mode may be the interaction mode of the human-computer interaction device when it detects a voice command. The desktop mode may be the interaction mode of the human-computer interaction device when it detects that it is in a smaller active plane (such as a desktop). The standby mode may be the default or conventional interaction mode of the human-computer interaction device.

[0091] Exemplarily, the feedback on an event may include: a normal response to the event, no response to the event, and a weak response to the event.

[0092] Exemplarily, the specific response content of the normal response / weak response corresponding to each event may be preset.

[0093] For example, a normal response to an obstacle event might include the human-computer interaction device pushing the obstacle forward or circling around it, indicating the device is playing with the obstacle. A weak response to an obstacle event might include circling around the obstacle, with the ear moving slowly and slightly, or with the indicator light on the ear flashing, indicating the device has detected the obstacle but is focused on something else, such as following the user.

[0094] For example, a normal response to a face event might include the human-machine interaction device shaking its torso, rotating its ears along the torso, rotating the entire device, jumping, or lifting its legs. In this case, the human-machine interaction device might appear to be very happy. A weak response to a face event might include a small, slow movement of the ears, or a flashing indicator light on the ears. In this case, the human-machine interaction device might be considered to have seen the face but is focused on something else, such as waiting for a user's voice command.

[0095] For example, a normal response to a touch event might include the head of the human-computer interaction device resting on the user's palm or the ears rotating along the torso. In this case, the human-computer interaction device might be considered to be enjoying itself. A weak response to a touch event might include the ears of the human-computer interaction device rotating slowly and with a small amplitude, or the indicator lights on the ears flashing. In this case, the human-computer interaction device might be considered to be aware of being touched but focused on something else, such as performing a performance or playing a game.

[0096] For example, a normal response to a start game event (an example of controlling an application) may include the human-computer interaction device shaking its torso, rotating its ears along the torso, and rotating, jumping, or lifting its legs, indicating that the human-computer interaction device is looking forward to playing a game with the user. A weak response to an open game application event may include outputting an abnormal status prompt message and / or flashing ear indicators, indicating that the human-computer interaction device has received the start game command but is temporarily unable to enter the game, such as when the human-computer interaction device is picked up by the user or falls to the ground.

[0097] Below, a solution for determining event feedback is described by taking Method 1 and Method 2 as examples.

[0098] Method 1: The feedback of the event is set to be related to the feedback priority of the event in the current interaction mode of the human-computer interaction device. In this way, after the human-computer interaction device detects the event, it can directly determine the feedback of the event based on the feedback priority level of the event in the current interaction mode.

[0099] For example, each event in each interaction mode corresponds to a feedback priority level, and each feedback priority level corresponds to a feedback method. Thus, S220 may specifically include: determining feedback for the first event according to the priority level of the first event in the first interaction mode.

[0100] Feedback priority can be understood as the order of priority for feedback on multiple events that occur simultaneously. Thus, the higher the feedback priority of an event, the stronger the feedback capability for that event.

[0101] The embodiment of the present application does not limit the number of feedback priority levels.

[0102] For example, if the feedback priority levels can be divided into high feedback priority levels and low feedback priority levels, the feedback capability of the event corresponding to the high feedback priority level is stronger than the feedback capability of the event corresponding to the low feedback priority level. For example, the feedback mode of the event corresponding to the high feedback priority level is a normal response, and the feedback mode of the event corresponding to the low feedback priority level is no response or a weak response.

[0103] For example, feedback priority levels can be divided into high, medium, and low feedback priority levels, with the feedback capabilities of events corresponding to high, medium, and low feedback priority levels decreasing in order. For example, the feedback method corresponding to high feedback priority level is normal response, the feedback method corresponding to medium feedback priority level is weak response, and the feedback method corresponding to low feedback priority level is no response.

[0104] In order to improve the accuracy of the human-computer interaction device's analysis of the user's behavioral intention or external environment, the feedback priority of the same event can be different in different interaction modes. This may result in different feedback for the same event in different interaction modes.

[0105] For example, the multiple interaction modes include a first interaction mode and a second interaction mode, and the feedback priority of the first event in the first interaction mode is different from the feedback priority of the first event in the second interaction mode. In addition, the feedback of the first event in the first interaction mode may be different from the feedback of the first event in the second interaction mode.

[0106] In humanoid mode, trigger events such as touch events, facial events, or gesture events receive higher feedback priority. In other words, the feedback priority for trigger events such as touch events, facial events, or gesture events in humanoid mode is higher than the feedback priority for trigger events such as touch events, facial events, or gesture events in application control mode or voice control mode. Furthermore, the feedback for trigger events such as touch events, facial events, or gesture events in humanoid mode may be a normal response, while the feedback for trigger events such as touch events, facial events, or gesture events in application control mode or voice control mode may be no response or a weak response.

[0107] In face mode, feedback for interactive events such as touches or gestures is prioritized. In other words, feedback for interactive events such as touches or gestures in face mode is prioritized over feedback for events in application control mode or voice control mode. Furthermore, feedback for interactive events such as touches or gestures in face mode may be normal, while feedback for interactive events such as touches or gestures in application control mode or voice control mode may be non-responsive or weak.

[0108] In application control mode, feedback for application manipulation events takes precedence. In other words, feedback for application manipulation events in application control mode takes precedence over feedback for events in humanoid mode, face mode, voice control mode, or desktop mode. Furthermore, feedback for application manipulation events in application control mode may be a normal response, while feedback for events in humanoid mode, face mode, voice control mode, or desktop mode may be no response or a weak response.

[0109] In voice control mode, voice input event feedback has a higher priority. In other words, voice input event feedback in voice control mode has a higher priority than in human shape mode, face mode, application control mode, or desktop mode. Furthermore, a voice input event in voice control mode might respond normally, while a voice input event in human shape mode, face mode, application control mode, or desktop mode might respond with no response or a weak response.

[0110] In desktop mode, feedback for desktop edge events takes precedence. In other words, feedback for desktop edge events in desktop mode takes precedence over feedback in human shape mode, face mode, application control mode, or voice control mode. Furthermore, feedback for desktop edge events in desktop mode may be a normal response, while feedback for desktop edge events in human shape mode, face mode, application control mode, or voice control mode may be no response or a weak response.

[0111] In standby mode, all events have the same feedback priority. In other words, face events, touch events, gesture events, application control events, voice input events, desktop edge events, or human figure events have equal feedback priority in human figure mode, face mode, application control mode, voice control mode, or desktop mode. Furthermore, face events, touch events, gesture events, application control events, voice input events, desktop edge events, or human figure events in human figure mode, face mode, application control mode, voice control mode, or desktop mode may all be responded to normally.

[0112] Method 2: The feedback for an event is set to be related to the current interaction mode of the human-computer interaction device and the interaction status in the current interaction mode. In this way, after the human-computer interaction device detects an event, it can directly determine the feedback for the event based on the current interaction mode and the interaction status in the current interaction mode.

[0113] For example, each interaction mode may include at least one interaction state, and each event corresponds to a feedback method in each interaction state of each interaction mode. In this way, S220 may specifically include: determining feedback for the first event according to the first interaction mode and the first interaction state in the first interaction mode.

[0114] In order to improve the accuracy of the human-computer interaction device's analysis of the user's behavioral intention or the external environment, the feedback on the same event may be different in different interaction states under the same interaction mode.

[0115] For example, when the first interaction mode is the standby mode, if the first interaction state is the performance state, then the human-computer interaction device's feedback to the obstacle event is no response, and the feedback to the face event is a normal response; if the first interaction state is the obstacle state, then the human-computer interaction device's feedback to the obstacle event is a normal response, and the feedback to the face event is no response or a weak response.

[0116] For example, in the case where the first interaction mode is the humanoid mode, if the first interaction state is the gesture state, then the feedback of the human-computer interaction device to the humanoid event is no response or a weak response, and the feedback to the gesture event is a normal response; if the first interaction state is the follow state, then the feedback of the human-computer interaction device to the humanoid event is a normal response, and the feedback to the gesture event is no response or a weak response.

[0117] For example, in the case where the first interaction mode is the face mode, if the first interaction state is the stroking state, then the feedback of the human-computer interaction device to the stroking event is a normal response, and the feedback to the gesture event is no response or a weak response; if the first interaction state is the gesture state, then the feedback of the human-computer interaction device to the stroking event is no response or a weak response, and the feedback to the gesture event is a normal response.

[0118] For example, when the first interaction mode is the voice control mode, if the first interaction state is the specific instruction state, then the feedback of the human-computer interaction device to the voice input event is a normal response; if the first interaction state is the falling state, then the feedback of the human-computer interaction device to the voice input event is a weak response or no response.

[0119] Optionally, in some embodiments, after S220 , method 200 may further include: providing feedback on the first event.

[0120] Optionally, in some embodiments, method 200 further includes S230 and S240, where S240 is performed after S230. S230 and S240 are described in detail below.

[0121] S230: Detect a second event.

[0122] S240: Determine feedback for the second event according to the first interaction mode.

[0123] The human-computer interaction device may also determine feedback of the second event based on the above-mentioned method 1 and method 2.

[0124] Exemplarily, in the manner 1, S240 may specifically include: determining feedback on the second event according to the priority level of the second event in the first interaction mode.

[0125] Since the feedback priority of the same event is different in different interaction modes, the feedback priority of the second event in the second interaction mode is different from the feedback priority of the second event in the first interaction mode.

[0126] In addition, in the same interaction mode, the feedback priorities of different events may be different or the same.

[0127] For example, in one example, the feedback priority of the second event in the first interaction mode is higher than the feedback priority of the first event in the first interaction mode. Then, the feedback to the second event is a normal response, and the feedback to the first event is no response or a weak response.

[0128] For example, as in the example above, in humanoid mode, the feedback priority of triggering events such as touch events, face events, or gesture events is high. Then, when the first interaction mode is humanoid mode, the second event can be a touch event, face event, or gesture. In face mode, the feedback priority of interactive events such as touch events or gesture events is high. Then, when the first interaction mode is face mode, the second event can be an interactive event such as a touch event or gesture event. In application control mode, the feedback priority of manipulating application events is high. Then, when the first interaction mode is application control mode, the second event can be a manipulating application event. In voice control mode, the feedback priority of voice input events is high. Then, when the first interaction mode is voice control mode, the second event can be a voice input event. In desktop mode, the feedback priority of desktop edge events is high. Then, when the first interaction mode is desktop mode, the second event can be a desktop edge event.

[0129] For example, in another example, the feedback priority of the second event in the first interaction mode is equal to the feedback priority of the first event in the first interaction mode. Then, when the feedback to the first event is a normal response, the feedback to the second event is also a normal response; when the feedback to the first event is no response, the feedback to the second event is also no response; when the feedback to the first event is a weak response, the feedback to the second event is also a weak response.

[0130] Exemplarily, in mode 2, S240 may specifically include: determining feedback on the second event according to the first interaction mode and the second interaction state in the first interaction mode.

[0131] The second interaction state and the first interaction state may be the same or different, and this embodiment of the present application does not limit this.

[0132] The solution for determining feedback for the second event according to the first interaction mode is similar to the solution for determining feedback for the first event according to the first interaction mode in S220 above. Please refer to the corresponding description above and will not be repeated here.

[0133] Optionally, in some embodiments, after S240 , method 200 may further include: providing feedback on the second event.

[0134] It should be noted that the embodiment of the present application does not limit the execution order of S230, S210, and S220, respectively, and does not limit the execution order of S240, S210, and S220, respectively.

[0135] In addition, some examples in method 200, such as examples of events, examples of interaction modes, and examples of interaction states, are merely examples provided to facilitate understanding of the solution and should not constitute limitations to this application.

[0136] In the above method 200, since the human-computer interaction device has multiple interaction modes, and the feedback priority of the same event (such as the first event, the second event) in different interaction modes is different, the human-computer interaction device has various feedback methods for the same event. Then, when the human-computer interaction device detects an event, it can flexibly select the feedback method for the event according to the current interaction mode. Therefore, compared to the existing solution that uses the same feedback for the same event in different interaction modes, the embodiment of the present application can use different feedback for the same event in different interaction modes, which helps to improve the user experience.

[0137] Below, the device provided in the embodiment of the present application is described in detail with reference to FIG3 and FIG4 .

[0138] FIG3 is a schematic structural diagram of a human-computer interaction device 300 provided in an embodiment of the present application.

[0139] For example, as shown in FIG3 , the human-computer interaction device 300 has multiple interaction modes and includes a detection unit 310 and a processing unit 320. The detection unit 310 is configured to detect a first event; the processing unit 320 is configured to determine feedback for the first event based on the current first interaction mode of the human-computer interaction device. The first interaction mode is one of multiple interaction modes, and the multiple interaction modes also include a second interaction mode. The feedback priority of the first event in the second interaction mode is different from the feedback priority of the first event in the first interaction mode.

[0140] Optionally, in some embodiments, the detection unit 310 is further used to detect a second event; the processing unit 320 is further used to determine feedback for the second event based on the first interaction mode, wherein the feedback priority of the second event in the first interaction mode is higher than the feedback priority of the first event in the first interaction mode, and the feedback for the second event is a normal response to the second event, and the feedback for the first event is no response or a weak response to the first event.

[0141] Optionally, in some embodiments, each interaction mode includes at least one interaction state, and the processing unit 320 is further specifically configured to determine feedback for the first event according to the first interaction mode and the first interaction state in the first interaction mode.

[0142] For other parts not described in FIG3 , please refer to the relevant description in method 200 , which will not be repeated here.

[0143] FIG4 shows a schematic structural diagram of a human-computer interaction device 400 provided in an embodiment of the present application.

[0144] For example, as shown in FIG4 , the human-computer interaction device 400 includes: one or more processors 410 and one or more memories 420. The one or more memories 420 store one or more computer programs, each of which includes instructions. When the instructions are executed by the one or more processors 410, the human-computer interaction device 400 performs the above-described method 200.

[0145] The embodiment of the present application provides a computer program product, which, when executed on a human-computer interaction device, causes the human-computer interaction device to execute the above method 200. The implementation principle and technical effect thereof are similar to those of the above method 200 and will not be described in detail here.

[0146] The embodiment of the present application provides a computer-readable storage medium, which contains instructions. When the instructions are executed on a device, the device executes the above method 200. The implementation principles and technical effects are similar and will not be repeated here.

[0147] The embodiment of the present application provides a chip, which is used to execute instructions. When the chip is running, the above method 200 is executed. The implementation principle and technical effect thereof are similar and will not be described in detail here.

[0148] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0149] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0150] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0151] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0152] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0153] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0154] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A human-computer interaction device, characterized in that: The human-computer interaction device has multiple interaction modes, and the human-computer interaction device includes: a detection unit, configured to detect a first event; A processing unit is used to determine feedback for the first event based on a current first interaction mode of the human-computer interaction device, where the first interaction mode is one of the multiple interaction modes, and the multiple interaction modes also include a second interaction mode, wherein the feedback priority of the first event in the second interaction mode is different from the feedback priority of the first event in the first interaction mode.

2. The human-computer interaction device according to claim 1, wherein: The detection unit is further configured to detect a second event; The processing unit is further used to determine feedback for the second event based on the first interaction mode, wherein the feedback priority of the second event in the first interaction mode is higher than the feedback priority of the first event in the first interaction mode, and the feedback for the second event is a normal response to the second event, and the feedback for the first event is no response or a weak response to the first event.

3. The human-computer interaction device according to claim 2, wherein: The first interaction mode is a humanoid mode, and the second event is a touch event, a face event, or a gesture event.

4. The human-computer interaction device according to claim 2, wherein: The first interaction mode is a face mode, and the second event is a touch event or a gesture event.

5. The human-computer interaction device according to claim 2, wherein: The first interaction mode is an application control mode, and the second event is an application manipulation event.

6. The human-computer interaction device according to claim 2, wherein: The first interaction mode is a voice control mode, and the second event is a voice input event.

7. The human-computer interaction device according to claim 2, wherein: The first interaction mode is a desktop mode, and the second event is a desktop edge event.

8. The human-computer interaction device according to claim 1, wherein: The first interaction mode is a standby mode. In the first interaction mode, feedback priorities of all events are the same.

9. The human-computer interaction device according to any one of claims 1 to 8, characterized in that: Each of the interaction modes includes at least one interaction state, The processing unit is further specifically configured to determine feedback for the first event according to the first interaction mode and the first interaction state under the first interaction mode.

10. A human-computer interaction method, characterized in that: The human-computer interaction method is applied to a human-computer interaction device, the human-computer interaction device has multiple interaction modes, and the human-computer interaction method includes: detecting a first event; Based on the current first interaction mode of the human-computer interaction device, feedback for the first event is determined, where the first interaction mode is one of the multiple interaction modes, and the multiple interaction modes also include a second interaction mode, wherein the feedback priority of the first event in the second interaction mode is different from the feedback priority of the first event in the first interaction mode.

11. The human-computer interaction method according to claim 10, wherein: The method further comprises: detecting a second event; Based on the first interaction mode, feedback for the second event is determined, wherein the feedback priority of the second event in the first interaction mode is higher than the feedback priority of the first event in the first interaction mode, the feedback for the second event is a normal response to the second event, and the feedback for the first event is no response or a weak response to the first event.

12. The human-computer interaction method according to claim 11, wherein: The first interaction mode is a humanoid mode, and the second event is a touch event, a face event, or a gesture event.

13. The human-computer interaction method according to claim 11, wherein: The first interaction mode is a face mode, and the second event is a touch event or a gesture event.

14. The human-computer interaction method according to claim 11, wherein: The first interaction mode is an application control mode, and the second event is an application manipulation event.

15. The human-computer interaction method according to claim 11, wherein: The first interaction mode is a voice control mode, and the second event is a voice input event.

16. The human-computer interaction method according to claim 11, wherein: The first interaction mode is a desktop mode, and the second event is a desktop edge event.

17. The human-computer interaction method according to claim 11, wherein: The first interaction mode is a standby mode. In the first interaction mode, feedback priorities of all events are the same.

18. The human-computer interaction method according to any one of claims 10 to 17, characterized in that: Each of the interaction modes includes at least one interaction state, and determining feedback for the first event according to the current first interaction mode of the human-computer interaction device includes: Feedback for the first event is determined according to the first interaction mode and a first interaction state in the first interaction mode.

19. A human-computer interaction device, characterized in that: include: one or more processors; one or more memories; and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions that, when executed by the one or more processors, enable the human-computer interaction device to perform the human-computer interaction method as described in any one of claims 10 to 18.

20. A computer-readable storage medium, characterized in that The method comprises computer instructions, which, when executed on a human-computer interaction device, enable the human-computer interaction device to execute the human-computer interaction method according to any one of claims 10 to 18.

21. A chip, characterized in that: The system comprises at least one processor and an interface circuit, wherein the interface circuit is used to provide program instructions or data to the at least one processor, and the at least one processor is used to execute the program instructions to implement the human-computer interaction method according to any one of claims 10 to 18.

22. A computer program product, characterized in that The computer program product includes computer instructions; when part or all of the computer instructions are run on a computer, the human-computer interaction method according to any one of claims 10 to 18 is executed.

Images