Method of human-machine interaction and electronic device

Abstract

The application provides a method and an electronic device for human-computer interaction. The method is applied to a first electronic device, and the method comprises: acquiring a first image of a first limb of a user by using a first camera when the first electronic device is in an unlocked state; displaying first prompt information according to the first limb in the first image, wherein the first prompt information is used to prompt the user to start a limb action; acquiring first action information corresponding to the limb action of the user and a first media file of the limb action of the user, wherein the first action information comprises depth information of the limb of the user; adding a first dynamic effect corresponding to a target action in the first media file to obtain a second media file when the first action information can be matched to the target action; and playing the second media file. The technical solution can improve the diversity of a human-computer interaction scene.

Description

Technical Field

[0001] This application relates to the field of electronic technology, and more specifically, to a method and electronic device for human-computer interaction. Background Technology

[0002] During the use of electronic devices, there are some interaction scenarios that can enhance the user experience. For example, air gestures and air teleportation are examples of such interactions. However, these scenarios are relatively simple, offer little playability, and can easily lead to user fatigue. Summary of the Invention

[0003] This application provides a method and electronic device for human-computer interaction, aiming to improve the diversity of human-computer interaction scenarios.

[0004] In a first aspect, a human-computer interaction method is provided, applied to a first electronic device, comprising: acquiring a first image of a user's first limb using a first camera; displaying first prompt information based on the first limb in the first image, the first prompt information being used to prompt the user to begin performing a limb movement; acquiring first action information corresponding to the user's limb movement and a first media file of the user's limb movement, wherein the first action information includes three-dimensional information of the user's limb; if the first action information can match a preset target action, adding a first motion effect corresponding to the target action to the first media file to obtain a second media file; and playing the second media file.

[0005] A user's body movements can include single body movements, such as gestures, or combined movements of multiple body movements, such as drawing a preset shape with arms and hands, or a certain dance move, etc.

[0006] It should be understood that the three-dimensional information of the user's limbs can be determined by depth sensing information or by the relative motion trajectory determined by the star flash method.

[0007] The target action and the first animation can be preset in the system of the first electronic device, added manually by the user, or downloaded by the first electronic device from the theme market.

[0008] In some cases, the first electronic device can use its first camera to acquire a first image of the user's first limb while the device is unlocked. Alternatively, the first electronic device can also use its first camera to acquire a first image of the user's first limb after receiving an instruction from another electronic device while the device is locked.

[0009] Based on the embodiments of this application, a first electronic device acquires a first image of a user's first limb through a first camera, and displays a first prompt message based on the first limb to prompt the user to start performing a limb movement; when the user performs a limb movement, the device acquires the user's first movement information and a first media file of the user performing the limb movement, and when the first movement information can match a target movement, the device adds a corresponding first motion effect to the first media file to obtain a second media file, and plays the second media file.

[0010] In this way, the first electronic device can acquire the user's body movement information and the first media file, and add the first animation corresponding to the target action matched with the action information to the first media file. Different actions can correspond to different animations, thereby improving the playability of human-computer interaction and the diversity of application scenarios.

[0011] In some implementations, the target action corresponds to the first operation. The method also includes: executing the first operation after the second media file has finished playing.

[0012] For example, the first operation can be user-defined or a system default. For instance, the first operation could be making a phone call, opening an application, adding special effects, switching or adding background music, etc.

[0013] Based on the embodiments of this application, the target action can correspond to the first operation, and after the second media file is played, the first electronic device can perform the first operation, thereby improving the playability of human-computer interaction.

[0014] In some implementations, acquiring the first action information corresponding to the user's body movement and the first media file of the user's body movement includes: acquiring the first action information using a second camera and acquiring the first media file using a third camera, wherein the second camera is a depth-sensing camera and the third camera is a non-depth-sensing camera.

[0015] Based on the embodiments of this application, the first electronic device uses a depth-sensing camera to acquire the user's first action information and uses a non-depth-sensing camera to acquire the user's first media file of limb movements. This makes the action information acquired by the first electronic device more accurate. Using different types of cameras to acquire different information makes the acquisition efficiency of the first electronic device more efficient.

[0016] In some implementations, the first camera is part or all of the second or third camera.

[0017] For example, if the first camera and the second camera are the same camera, and the third camera is a separate camera, the electronic device may include two cameras to implement the technical solution. Alternatively, if the first camera and the third camera are the same camera, and the second camera is a separate camera, the electronic device may include two cameras to implement the technical solution.

[0018] For example, if a portion of the second camera can capture the user's first image, then the first camera can be a portion of the second camera. Or, if a portion of the third camera can capture the user's first image, then the first camera can be a portion of the third camera.

[0019] In other examples, the first camera may be a camera other than the second or third camera. In this case, the electronic device may include three cameras to implement the technical solution.

[0020] In some implementations, the first camera is in low-power mode, or the first camera is a low-power camera.

[0021] It should be understood that the first camera being a low-power camera means that the power consumption of the first camera is lower. For example, compared to a regular camera, the power consumption of the first camera is lower, or compared to the previous generation product, the power consumption of the first camera is lower.

[0022] The first camera is in a low-power mode, or the first camera is a low-power camera that can reduce the power consumption of the first electronic device when the first camera needs to constantly detect whether the user's first limb is in the field of view.

[0023] In some implementations, a second camera is used to acquire first motion information, and a third camera is used to acquire a first media file. This includes: starting at a first moment, using the second camera to acquire the first motion information, and starting at a second moment, using the third camera to acquire the first media file. The first moment is the same as the second moment, or the second moment is before or after the first moment, or the second moment is the moment when the third camera receives instruction information indicating that it is starting to work.

[0024] For example, the instruction information may be sent by a second camera.

[0025] Based on the embodiments of this application, the second camera is used to acquire first action information starting at a first moment, and the third camera is used to acquire the first media file starting at a second moment, with the second moment occurring after the first moment. This ensures that the third camera starts working only after the second camera has begun operating. This avoids the power consumption waste that would occur if the user did not perform any actions for a period of time before the third camera started operating.

[0026] In some implementations, before using the second camera to acquire first motion information at a first moment and using the third camera to acquire the first media file at a second moment, the method further includes: determining that the first limb in the first image acquired by the first camera includes the limb required in a preset motion, wherein the first camera is a low-power camera.

[0027] Based on the embodiments of this application, when the first limb in the first image acquired by the first electronic device through the low-power first camera includes the limb required in the preset action, the second camera is used to acquire the first action information at the first moment, and the third camera is used to acquire the first media file at the second moment. This can avoid the second and third cameras being constantly activated, thereby reducing the power consumption of the first electronic device.

[0028] In some implementations, obtaining the first action information corresponding to the user's limb movement and the first media file of the user's limb movement includes: determining the first action information based on the motion trajectory of the second electronic device relative to the first electronic device, and using a third camera to acquire the first media file, wherein the second electronic device is communicatively connected to the first electronic device.

[0029] For example, the first electronic device and the second electronic device can be connected via a star connection or a Bluetooth connection.

[0030] Based on the embodiments of this application, the first electronic device can also determine the user's first action information by acquiring the motion trajectory of the second electronic device relative to the first electronic device. For example, the second electronic device can be a stylus or a wearable device, which can improve the application scenarios of human-computer interaction and enhance the user's interactive experience.

[0031] In some implementations, before using the first camera to acquire a first image of the user's first limb, the method further includes: receiving first instruction information sent by a second electronic device, the first instruction information being used to instruct the first camera to be activated.

[0032] Based on the embodiments of this application, the first electronic device only starts the first camera to acquire the first image after receiving the first instruction information sent by the second electronic device. This allows the first camera to be started on demand, avoiding continuous operation of the first camera and thus reducing the power consumption of the electronic device.

[0033] In some implementations, before determining the first motion information based on the motion trajectory of the second electronic device relative to the first electronic device and acquiring the first media file using the third camera, the method further includes: determining that the first limb in the first image acquired by the first camera includes the limb required in a preset motion, wherein the first camera is a low-power camera.

[0034] In this way, the first electronic device only starts using the second camera to acquire the first action information when the first limb in the first image acquired by the first electronic device through the low-power first camera includes the limb required in the preset action, and starts using the third camera to acquire the first media file at the second moment. This avoids the second and third cameras from being constantly activated, thereby reducing the power consumption of the first electronic device.

[0035] In some implementations, a first motion effect corresponding to the target action is added to the first media file to obtain a second media file, including: adding a first motion effect to the limb movements included in the first media file to obtain a second media file.

[0036] In this way, since the user's body movements will not be exactly the same each time, the resulting second media file will be different. This can improve the user's interactive experience.

[0037] In some implementations, the first electronic device, with its first display interface in an unlocked state, plays a second media file, including: playing the second media file on top of the first display interface.

[0038] For example, the display area of ​​the second media file can cover part of the first display interface, such as being located in the middle or upper part of the first display interface.

[0039] Based on the embodiments of this application, a second media file is played on a first display interface, so that the user can both browse the second media file and know the current specific display interface.

[0040] In some implementations, the first electronic device is in an unlocked state with a second display interface, which is the display interface of a third-party application. After the second media file is played, the first operation is performed, including: after the second media file is played using the third-party application, the first operation is performed using the third-party application.

[0041] For example, the third-party application can be a live streaming application. For instance, the first operation could be adding background music, adding special effects, etc. This allows users to conveniently control the first electronic device to perform operations through body language interaction, even when the user is far away from the device.

[0042] In this way, third-party applications can also use the above-mentioned interactive functions, thereby increasing the application diversity of this human-computer interaction.

[0043] In some implementations, before acquiring a first image of the user's limb using a first camera, the method further includes: associating the target action with the first action in response to a user's setting operation.

[0044] Based on the embodiments of this application, users can set actions and associated first operations in the first electronic device, thereby greatly enhancing users' independent creativity and improving the subsequent interaction experience with users.

[0045] In some implementations, the target action is either preset by the system or added based on the user's operation; the first operation is either preset by the system or added based on the user's operation.

[0046] In other examples, the target action and the first operation could also be something the user downloaded from an app store.

[0047] In some implementations, the method further includes: playing a demonstration animation of the target action; using a depth camera to acquire second action information of the user's limb movements; when it is determined that the action indicated by the second action information matches the target action, displaying a second prompt message to prompt the user that the target action learning is complete; or, when it is determined that the action indicated by the second action information does not match the target action, displaying a third prompt message to prompt the user that the limb movements need to be corrected.

[0048] Based on the embodiments of this application, the first electronic device can also play a demonstration of the target action to allow the user to learn, and can verify the user's learning results and provide guidance for the target action.

[0049] In some implementations, displaying a first prompt message based on a first limb in a first image includes: displaying the first prompt message when the first limb matches a limb included in any preset action.

[0050] It should be understood that the first limb here matches the limbs included in any preset action, which can be understood as the first limb including the limbs required in any preset action.

[0051] For example, if the first limb is the head and hand, and the preset action 1 requires the head as the limb, and action 2 requires the hand as the limb, then the first limb includes the limbs required for both action 1 and action 2. This first prompt information can be the head and / or hand, or a uniform icon, etc.

[0052] Based on the embodiments of this application, when it is determined that the first limb matches a limb included in any preset action, it can be determined that the user may have a need for human-computer interaction. Therefore, a first prompt message can be displayed to prompt the user to start performing a limb movement. This avoids the user performing an invalid action because the first electronic device has not started recognizing the limb.

[0053] In some implementations, the first prompt message includes an icon or animation of the first limb.

[0054] In other examples, the first prompt message includes an icon or animation of a limb included in any of the aforementioned preset actions.

[0055] In this way, the first electronic device provides users with more accurate action guidance.

[0056] In a second aspect, a method for human-computer interaction is provided, including modules for implementing the method described in the first aspect and any possible implementation thereof.

[0057] Thirdly, an electronic device is provided, comprising: one or more processors; one or more memories; said one or more memories storing one or more programs that, when executed by said one or more processors, cause the method described in the first aspect and any possible implementation thereof to be performed.

[0058] Fourthly, a chip is provided, the chip including a processor and a communication interface, the communication interface being used to receive a signal and transmit the signal to the processor, the processor processing the signal such that the method described in the first aspect and any possible implementation thereof is performed.

[0059] Fifthly, a readable storage medium is provided, wherein instructions are stored therein, which, when executed on an electronic device, cause the method described in the first aspect and any possible implementation thereof to be performed.

[0060] In a sixth aspect, a program product is provided, the program product comprising program code that, when run on an electronic device, causes the method described in the first aspect and any possible implementation thereof to be executed. Attached Figure Description

[0061] Figure 1 A schematic diagram of the structure of the electronic device 100 is shown.

[0062] Figure 2 This is a schematic diagram of a set of graphical user interfaces (GUIs) provided in the embodiments of this application.

[0063] Figure 3This is a schematic diagram of another set of GUIs provided in this application.

[0064] Figure 4 This is a schematic diagram of a set of GUIs provided in the embodiments of this application.

[0065] Figure 5 This is a schematic diagram of another set of GUIs provided in this application.

[0066] Figure 6 This is a schematic flowchart of a human-computer interaction method provided in an embodiment of this application.

[0067] Figure 7 This is a schematic flowchart of a human-computer interaction method provided in an embodiment of this application.

[0068] Figure 8 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0069] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0070] The methods described in this application embodiment can be applied to smartphones, wearable devices, smart TVs, tablets, laptops, personal computers (PCs), ultra-mobile personal computers (UMPCs), netbooks, in-vehicle devices, foldable devices, Internet of Things (IoT) devices, augmented reality devices, virtual reality devices, etc.

[0071] Figure 1 A schematic diagram of the structure of electronic device 100 is shown. Electronic device 100 may include processor 110, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, display screen 194, camera 193, internal memory 121, and touch sensor 180K, etc.

[0072] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0073] Processor 110 may include one or more processing units, such as a central processing unit (CPU), a modem, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0074] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0075] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), Bluetooth low energy (BLE), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), ultra-wideband (UWB) technology, and near link.

[0076] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with networks and other devices through wireless communication technology.

[0077] Electronic device 100 implements display functions through a GPU, a display screen 194, and a central processing unit. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0078] The display screen 194 is used to display images, videos, etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1. In this embodiment, the display screen 194 can be used to display various display interfaces and application switching entries, etc.

[0079] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0080] The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture media files, which may include images, moving images, or videos.

[0081] In some examples, electronic device 100 may include one or N cameras. For example, electronic device 100 may include depth-sensing cameras, ordinary cameras, low-power cameras, etc. In the embodiments of this application, ordinary cameras can also be understood as non-depth-sensing cameras.

[0082] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function, etc. The data storage area may store data created during the use of electronic device 100, etc.

[0083] Touch sensor 180K can be disposed on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen". Touch sensor 180K is used to detect touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be disposed on the surface of electronic device 100, in a different location than display screen 194.

[0084] Before introducing the technical solutions of the embodiments of this application, some technical terms that may be involved in this application will be introduced as follows.

[0085] Depth information refers to the three-dimensional coordinate information of an object.

[0086] Motion effects are a form that combines time and space, using moving elements to convey information, enhance visual effects, and improve user experience.

[0087] Artificial intelligence generated content (AIGC) is a technique that uses artificial intelligence methods such as generative adversarial networks and large-scale pre-trained models to generate relevant content with appropriate generalization ability through learning and recognition of existing data. For example, it can generate text, images, videos, and audio.

[0088] During the use of electronic devices, there are some interaction scenarios that can enhance the user experience. For example, air gestures and air teleportation are examples of such interactions. However, these scenarios are relatively simple, lack playability, and can easily lead to user fatigue.

[0089] In view of this, embodiments of this application provide a method and electronic device for human-computer interaction, aiming to improve the diversity of human-computer interaction scenarios.

[0090] The following will combine Figure 2-7 The technical solutions in the embodiments of this application are introduced.

[0091] For example, Figure 2 This is a schematic diagram of a set of graphical user interfaces (GUIs) provided in an embodiment of this application. Among them, from Figure 2 (a) through (e) show the process by which a user sets up user actions and associated operations in electronic device 200.

[0092] See Figure 2 In (a), the GUI can be a settings display interface 210 for electronic device 200. The display interface 210 may include system options, application and service options, battery options, storage options, magic effects options 211, user and account options, system update options, and about phone options, etc.

[0093] It should be understood that the name of this magical animation is merely illustrative; in other examples, it may also be referred to as "animation," "magical effect," etc.

[0094] For example, when the electronic device 200 detects that the user has clicked the magic animation option 211, it can display something like this: Figure 2 The GUI shown in (b) is shown in the image.

[0095] See Figure 2 In (b) of this document, the GUI can be a display interface 220 for magical animation effects. This display interface 220 may include a motion capture list 221, which may include 3D camera options and star flash options, etc.

[0096] In other examples, the motion capture list may also include other options, such as UWB, Bluetooth, etc., which are not limited in this embodiment.

[0097] When the electronic device 200 detects that the user has clicked on the 3D camera option, it can display something like this: Figure 2 The GUI shown in (c) is shown in the image.

[0098] See Figure 2 In (c), the GUI is a display interface 230 that captures the user's movements via a 3D camera. The display interface 230 may include a list of movements 231.

[0099] For example, the action list 231 may include multiple actions preset by the system, such as action 1, action 2 and action 3.

[0100] It is understandable that the actions in the action list 231 can be the user's physical actions, such as gestures, hand and arm movements, head and body movements, etc.

[0101] The action list 231 may also include a function control 232 for adding custom actions. When the user clicks the function control 232, the electronic device 200 can acquire the user's demonstration action through the depth camera and add the user's demonstration action to the action list 231.

[0102] This allows users to freely add actions, thereby increasing the diversity of user actions and improving the user experience.

[0103] In other examples, electronic devices can also acquire pre-designed actions from a theme marketplace, thereby increasing the variety of user actions and simplifying user operation.

[0104] For example, when the electronic device 200 detects that the user has clicked the action 1, it can display something like this. Figure 2 The GUI shown in (d) is shown in the image.

[0105] See Figure 2 In (d), the GUI can be a display interface 240 related to action 1. The display interface 240 may include a demonstration animation 241 of action 1 and an operation list 243 associated with action 1.

[0106] The display interface 240 may also include a function control 242 for learning action 1. When the user clicks on the function control 242, the electronic device 200 can acquire the user's body movements to determine whether the user's body movements meet the requirements of action 1. When it is determined that the user's body movements meet the requirements of action 1, prompt message A can be displayed to indicate that the user has completed the learning. When it is determined that the user's body movements do not meet the requirements of action 1, prompt message B can be displayed to prompt the user to correct their body movements.

[0107] For example, electronic device 200 can use a depth camera to acquire the user's body movements, or electronic device 200 can also use a binocular camera to acquire the user's body movements.

[0108] In this way, users can freely add operations associated with Action 1, thereby increasing the diversity of operations associated with user actions and improving the user experience.

[0109] The operation list 243 may include multiple system-preset operations, such as operation 1, operation 2, and operation 3. The operation list 243 may also include a function control 245 for adding custom operations. Users can add operations that they wish to associate with operation 1 by clicking on the function control 245.

[0110] For example, when the electronic device 200 detects that the user has clicked the settings control 244 of operation 1, it can display as follows: Figure 2 The GUI shown in (e) is shown in the image.

[0111] See Figure 2 In (e), the setting control 244 changes to the setting state.

[0112] Based on the embodiments of this application, users can set up body movements and related operations in the settings. When the user performs the body movement, the electronic device can recognize the user's body movement and perform related operations, thereby improving the user's interactive experience.

[0113] The user performs in electronic device 200 Figure 2 After the setup process, the electronic device can recognize the user's body movements in the unlocked state. When the user's body movements match the preset movements, it can display the corresponding animation and perform the corresponding operation. The following text combines... Figure 3 This technical solution will be introduced.

[0114] For example, Figure 3 This is a schematic diagram of another set of GUIs provided in this application. Among them, from Figure 3Figures (a) to (d) illustrate the process by which electronic device 200 detects a user’s limb movements and performs operations related to those limb movements.

[0115] See Figure 3 In (a), the GUI can be the display interface 250 of the electronic device 200. The display interface 250 can be the desktop of the electronic device 200, which can include multiple applications installed on the electronic device 200.

[0116] It should be understood that the embodiments of this application are described using the example of the electronic device 200 being on the desktop. In other examples, the electronic device 200 can be in an unlocked state, for example, the electronic device 200 is in the display interface of an application.

[0117] In this way, the electronic device 200 is in an unlocked state, thereby reducing the possibility of accidental operation.

[0118] When the electronic device 200 detects that the user's limb A matches the limb required for a preset target action, it can display, for example... Figure 3 The GUI shown in (b) is shown in the image.

[0119] For example, an electronic device can use a depth-sensing camera to capture an image of a user and identify limbs within it, such as hands, arms, and heads. The limbs identified by the electronic device can include those required for preset actions within the system. For instance, after the electronic device identifies the user's arms and head, and determines that a preset action requires either an arm or head movement, it can then display... Figure 3 The GUI shown in (b) is shown in the image.

[0120] In this application, the electronic device's use of a camera to collect image information from the user can be done with the user's authorization, which complies with relevant regulations.

[0121] See Figure 3 In (b) of the display interface 250, a prompt card 251 can be displayed. The prompt card 251 may include a prompt animation 1.

[0122] The specific form of the prompt animation 1 is not limited in the embodiments of this application.

[0123] In some implementations, the prompt animation 1 can be associated with the limbs required in a preset action among the limbs recognized by the electronic device.

[0124] For example, if an electronic device recognizes a user's hands and arms, and the limb required in the preset action A is the hand, then the prompt animation 1 can be a gesture.

[0125] Alternatively, if the electronic device recognizes the user's hands and arms, and the preset action A requires the hands, while the preset action B requires both hands and arms, then the prompt animation 1 can be the movement of the arms and hands to indicate to the user what action needs to be performed next.

[0126] For example, if an electronic device recognizes a user's hand and arm, and preset action A requires the hand, while preset action B requires both the hand and arm, then the prompt animation 1 can include a gesture icon 1 and arm and hand icons 2. That is, the prompt animation 1 can include multiple prompt icons to better guide the user through the action.

[0127] In other examples, electronic devices may display the same motion effects for different body movements, which is not limited in this application embodiment.

[0128] In other examples, the prompt card 251 may include cartoon icons of body movements to indicate to the user what action needs to be taken next.

[0129] Alternatively, when a user wears a wearable device, the electronic device can also display the aforementioned prompt animation 1 through the wearable device. For example, the electronic device can send the information of the prompt animation 1 to the wearable device and instruct the wearable device to display the prompt animation 1.

[0130] In some cases, electronic devices can also output voice prompts to users to begin an action. Alternatively, when a user is wearing a wearable device, the electronic device can output the voice prompt through the wearable device.

[0131] For example, electronic device 200 uses a depth-sensing camera to acquire motion information of a user's body movements, and uses a regular camera to acquire media file A containing the user's body movements. When the motion information matches a target motion, the corresponding animation is overlaid onto media file A to obtain media file B. When media file B is played, it can display, as shown in the image below. Figure 3 The GUI shown in (c) is shown in the image.

[0132] The media file A can be a video, a moving image, or multiple images.

[0133] See Figure 3 In (c), the electronic device 200 can display the playback card 252.

[0134] In some examples, the user's body movement of waving their arm in a circle is used as an illustration. The playback card 252 can be used to play a video or animated image of the user 254 waving their arm in a circle. At the same time, the video or animated image can also display the animation effects of the circling process. For example, it can display the trajectory 253 of the circle drawn by the user, and the trajectory 253 has visual effects such as glowing and outlining.

[0135] It is understood that the trajectory 253 can be displayed along with the user's waving motion, or it can be displayed after the user has completed the entire waving motion, or it can begin to be displayed within a preset time after the user begins the waving motion; this embodiment of the application does not limit the scope of the display. In this way, the generated trajectory 253 is basically consistent with the user's waving motion trajectory, thereby improving the user's human-computer interaction experience.

[0136] It should also be understood that when the electronic device 200 overlays the motion effect corresponding to the target action onto the media file A, it adds the motion effect to the user's body movements to generate the aforementioned motion-effected trajectory 253. For example, the motion effect is... Figure 3 The portal effect shown in (c) refers to the same target action. Although all actions correspond to the portal effect, the trajectory 253 generated by the electronic device is different each time the user performs a movement because there are subtle differences in the user's movements. This greatly enhances the user's playability.

[0137] In some embodiments, electronic devices may use AIGC to add the aforementioned motion effects to body movements. Alternatively, the motion effects may be generated by AIGC.

[0138] It should be understood that the playback card 252 can be superimposed on the display interface 250, and the playback card can have a certain degree of transparency, opacity, partial transparency, or gradient transparency, etc. In other examples, the playback card 252 can also completely cover the display interface 250.

[0139] After the electronic device 200 finishes playing the aforementioned media file B, it can execute the operation A corresponding to the target action and display the following: Figure 3 The GUI shown in (d) is shown in the image.

[0140] For example, operation A could be making a phone call to contact A.

[0141] In other examples, operation A could also be opening an application, such as a camera. Alternatively, operation A could be entering a mode, such as power saving mode or silent mode. In a live streaming scenario, operation A could also be changing background music, adding special effects, or turning on beauty filters, etc., which are not limited in the embodiments of this application.

[0142] See Figure 3 In (d), a dialing card 255 can be displayed within the area formed by trajectory 253. The card 255 can cover part or all of the area formed by trajectory 253, or the card 255 can have a certain degree of transparency, opacity, partial transparency, or gradient transparency, etc.

[0143] For example, the card 255 may include the text content "Calling contact A", or the card 255 may also include contact A's phone number, or the card 255 may display contact A's phone number separately, etc.

[0144] Based on the embodiments of this application, the electronic device can recognize the user's body and display corresponding prompt animations to prompt the user to start performing an action. It uses a depth-sensing camera to recognize the user's body movements, and a regular camera to acquire media files showing the user's body movements. When it is determined that the user's body movement matches one of the preset body movements, the corresponding animation is added to the aforementioned media file, and the media file with the animation is played, thereby enhancing the user's interactive experience. Furthermore, after playing the media file with the animation, the electronic device can also execute the operation corresponding to the body movement, thereby enhancing the playability of the electronic device.

[0145] In some cases, electronic devices can also determine a user's action information by recognizing the trajectory of another electronic device relative to the user. For example, the other electronic device could be a stylus or a wearable device, such as a smartwatch, smart bracelet, wireless headphones, smart glasses, virtual reality or augmented reality glasses, etc. The following will combine... Figures 4-5 The GUI shown illustrates this technical solution.

[0146] In this embodiment of the application, the example of an electronic device determining the trajectory of another electronic device relative to the electronic device by means of star flash is used for illustration.

[0147] For example, Figure 4 This is a schematic diagram of a set of GUIs provided in an embodiment of this application. Among them, from Figure 4 (a) through (e) show the process by which a user sets up user actions and associated operations in electronic device 200.

[0148] It should be understood that Figure 4 (a) to (b) can be found in [reference]. Figure 2 For the sake of brevity, the relevant descriptions in (a) to (b) are not repeated here.

[0149] For example, when the electronic device 200 detects that the user clicks on the star flash in the motion capture list 221, it can display something like this. Figure 4The GUI shown in (c) is shown in the image.

[0150] See Figure 4 In (c), the GUI can be the display interface 260 of the electronic device 200. The difference between the display interface 260 and the display interface 230 is that the display interface 260 can include the text content "Star Flash", while the display interface 230 includes the text content "3D Camera".

[0151] Users can also add new actions by clicking the "Add Custom Action Control" button.

[0152] For example, when the electronic device 200 detects that the user has clicked on action 2 in the action list 261, it can display something like this. Figure 4 The GUI shown in (d) is shown in the image.

[0153] See Figure 4 In (d), the GUI can be the display interface 270 for action 2. The display interface 270 may include a demonstration animation 271 for action 2, a function control 272 for learning action 2, an operation list associated with action 2 273, and a function to add custom operation controls 275.

[0154] It should be understood that the display interface 270 can be referred to in the previous description of the display interface 240.

[0155] For example, when the electronic device 200 detects that the user has clicked the settings control 274 of operation 2, it can display something like this: Figure 4 The GUI is shown in (e).

[0156] See Figure 4 In (e), the setting control 274 changes to the setting state.

[0157] Based on the embodiments of this application, users can set up body movements and related operations in the settings. When a user performs a body movement, the electronic device can identify the relative trajectory of another electronic device through star flashing to determine the user's body movement and perform related operations, thereby improving the user's interactive experience.

[0158] After the actions and corresponding operations are set, the electronic device can recognize the user's body movements in the unlocked state. It is understood that when the user's body movements match the preset actions, the technical solution in this application embodiment will only be executed after receiving the instruction information from another electronic device in the scenario where the electronic device is not unlocked. The camera in the electronic device does not need to work continuously, thereby saving the power consumption of the electronic device.

[0159] See Figure 5In (b) of this, the electronic device 200 can display a prompt card 256 on the display interface 250. The prompt card 256 can be used to display a prompt animation 2.

[0160] It should be understood that the description of Prompt Animation 2 can be found in the description of Prompt Animation 1, and for the sake of brevity, it will not be repeated here.

[0161] In some cases, electronic devices can also output voice prompts to users to begin an action. Alternatively, when a user is wearing a wearable device, the electronic device can output the voice prompt through the wearable device.

[0162] In some cases, the prompt card 256 may also include a star icon 2561 to indicate to the user that the electronic device is currently in a star connection state with other electronic devices, and that the electronic device needs to use the star method to determine the trajectory of other electronic devices in order to determine the user's body movements.

[0163] For example, a user can hold a stylus and make gestures, with the electronic device 200 connected to the stylus via a star-shaped sensor. The electronic device 200 can use the high-precision positioning capability of the star-shaped sensor to locate the stylus, determine the relative trajectory between the stylus and the electronic device, and thus determine the user's actions.

[0164] It is understandable that the relative trajectory of the stylus and the electronic device can be interpreted as the user's movement trajectory.

[0165] In other examples, the stylus can also be replaced by a wearable device, such as a smartwatch, bracelet, wireless earphone, virtual reality glasses, augmented reality glasses, or smart glasses that support StarFlash connectivity. This application does not limit this to any particular type of device.

[0166] While the electronic device 200 uses the StarFlash stylus for positioning, a regular camera captures media file C showing the user's body movements. When the user's movements match a preset target movement in the system, the corresponding animation is overlaid on media file C to obtain media file D, which can be displayed as shown below. Figure 5 The GUI shown in (c) is shown in the image.

[0167] See Figure 5 In (c), the electronic device 200 can display a playback card 257. This playback card 257 can be used to play media file D.

[0168] For example, in the media file D, user 254 can use stylus 258 to perform corresponding body movements, such as holding the stylus to draw preset shapes (such as circles, squares, etc.).

[0169] It is understood that this application embodiment uses a circle as an example to illustrate the preset shape. The animation display method of the circle's trajectory 259 can be found in the preceding text. Figure 3 The relevant description in (c) is not repeated here.

[0170] After the aforementioned media file D has finished playing, the electronic device 200 can execute the operation corresponding to the target action to display, as shown below. Figure 5 The GUI shown in (d) is shown in the image.

[0171] For example, taking the operation corresponding to the target operation as turning on silent mode as an example, see [link to relevant documentation]. Figure 5 In step (d), the electronic device 200 may display a prompt card 280. This prompt card 280 may include the text "To enable silent mode," thereby notifying the user that the electronic device 200 has performed the corresponding operation.

[0172] It should be understood that Figure 5 The technical solution shown in (d) is an optional solution. In other examples, the electronic device 200 may not perform the corresponding operation. For example, if the user does not set the operation corresponding to the target action, the electronic device 200 may not continue to perform other operations after playing the media file D.

[0173] Based on the embodiments of this application, the electronic device can recognize the user's limbs and display corresponding prompt animations to prompt the user to start performing an action. It uses the positioning function of a satellite to identify the relative trajectory of other electronic devices relative to the electronic device to determine the user's limb movements. A regular camera is used to acquire media files showing the user's limb movements. When it is determined that the user's limb movement matches one of the preset limb movements, the corresponding animation is added to the aforementioned media file, and the media file with the animation is played to enhance the user's interactive experience. Furthermore, after playing the media file with the animation, the electronic device can also execute the operation corresponding to the limb movement, thereby enhancing the playability of the electronic device.

[0174] The following will combine Figure 6 The flowchart shown details the technical solutions for human-computer interaction in the embodiments of this application.

[0175] For example, Figure 6 This is a schematic flowchart illustrating a human-computer interaction method provided in an embodiment of this application. Figure 6 As shown, the method 500 can be applied to electronic devices, and the method 500 may include steps 510 to 570.

[0176] 510. When the electronic device is in an unlocked state, the electronic device uses a low-power camera to acquire image A of the user.

[0177] In some examples, the electronic device can be placed on a desktop, and a low-power camera can capture an image A of the user.

[0178] In some examples, the electronic device can also be integrated into the display interface of other applications, and the low-power camera can also capture the user's image A.

[0179] It should be understood that when the electronic device is unlocked, the low-power camera can be in low-power standby mode to detect the presence of a user in real time. When a user appears in the low-power camera's field of view, the low-power camera can operate to acquire an image A of the user.

[0180] Electronic devices may include various cameras, such as depth-sensing cameras and ordinary cameras (such as front-facing cameras). The low-power camera may be a depth-sensing camera or an ordinary camera, or it may be the one with lower power consumption between the depth-sensing camera and the ordinary camera, or it may be a part of a depth-sensing camera or an ordinary camera. Alternatively, a low-power camera can also be understood as a depth-sensing camera or an ordinary camera operating in a low-power mode. Alternatively, the low-power camera may be an independent camera other than a depth-sensing camera or an ordinary camera; this is not limited in the embodiments of this application.

[0181] In this way, the human-computer interaction process is performed while the electronic device is in an unlocked state, thereby reducing the possibility of user misoperation.

[0182] 520. When image A meets preset condition A, the electronic device displays animation A. Animation A is associated with the content included in image A and is used to instruct the user to begin an action.

[0183] Image A satisfies the preset condition A, which can be understood as the user's body information in image A including the body information required for a certain action or several actions preset by the system.

[0184] For example, after an electronic device uses a low-power camera to recognize a user's arm and head, and determines that the limb information required for action 1 in the system is the arm, it can display motion effect A. Motion effect A can include motion effects of the person's arm.

[0185] Alternatively, if the electronic device recognizes the user's arm and head, and the preset action A requires the head as the limb, while the preset action B requires both the head and arm, then the motion effect A can be the movement of the head and arm to prompt the user what action needs to be performed next.

[0186] It is understandable that the description of animation A can be found in the previous description of animation 1.

[0187] In this way, different animations can be displayed based on the different image content identified, thus providing users with certain action guidance. By seeing the different animations displayed, users can determine which actions to take.

[0188] In other examples, the electronic device may display the same motion effect for different body movements, as long as the motion effect can prompt the user to start the body movement. This application embodiment does not limit this.

[0189] 530. The electronic device acquires the user's action information A and uses a regular camera to acquire media files A including the user's data.

[0190] In some implementations, the electronic device can use a depth-sensing camera to acquire the user's motion information A. Because a depth-sensing camera can acquire 3D data, using a depth-sensing camera to acquire the user's motion information A can make the acquired motion information A more accurate.

[0191] In some implementations, an electronic device can acquire user action information A by acquiring the trajectory of another electronic device.

[0192] For example, if an electronic device supports a star-shaped flash and the user's stylus also supports a star-shaped flash, the user can hold the stylus and perform actions. With the electronic device and stylus maintaining a star-shaped flash connection, the electronic device can use the star-shaped flash positioning method to obtain the relative trajectory of the stylus and use this trajectory as the user's action trajectory.

[0193] It is understood that the scheme of obtaining the trajectory of another electronic device by using star flashing in the embodiments of this application is the same as the scheme of positioning by star flashing.

[0194] For example, electronic devices can also use other positioning methods to obtain the trajectory of another electronic device, such as through Bluetooth, Wi-Fi, UWB, etc., in order to determine the user's action information A.

[0195] In some examples, an electronic device can simultaneously use a depth-sensing camera to acquire the user's motion information A, and a regular camera to acquire media files A including the user.

[0196] In some examples, the electronic device may first activate the depth-sensing camera, and then activate the regular camera after a preset duration of 1. For example, the preset duration of 1 can be 1 second or 2 seconds, and the specific value of the preset duration is not limited in this embodiment.

[0197] In this way, since the user may not perform the action immediately after the electronic device displays animation A, but only after a certain period of time, this method can reduce the power consumption of the electronic device to a certain extent.

[0198] In some examples, electronic devices can also first activate the depth-sensing camera, and when the depth-sensing camera recognizes the user's body movements, it can send instructions to the regular camera to instruct the regular camera to start working.

[0199] In this way, the regular camera only starts working after the depth-sensing camera recognizes the user's body movements, avoiding the acquisition of useless data before the user performs any actions. This method can reduce the power consumption of electronic devices to some extent.

[0200] 540. The electronic device determines whether action information A can match the preset target action.

[0201] In some implementations, if the electronic device determines that the action information A does not match a preset target action, then step 545 can be executed.

[0202] In one implementation, if the electronic device determines that the action information A can match a preset target action, then step 550 can be executed.

[0203] It should be understood that the preset target action can be a pre-set action in the system or an action added by the electronic device based on the user's operation. This application embodiment does not limit this.

[0204] 545, The electronic device displays prompt message A.

[0205] It should be understood that when the electronic device determines that the action information A does not match the preset target action, the electronic device may display prompt information A. Prompt information A may prompt the user to adjust the body movement, or the prompt information may be used to indicate that the user's body movement is not standard and cannot be recognized.

[0206] In other examples, the prompt A can also be used to guide the user on how to adjust their body movements to meet the standard body movement. For instance, if the target movement is a circle, but the user's movement information A indicates an ellipse, and the difference between the ellipse and the circle is too large for the electronic device to recognize, then the electronic device can prompt the user to make the circle they are drawing more round.

[0207] The electronic device can display the prompt message A through a display interface, or it can play the prompt message A via voice. Alternatively, if the user is wearing a wearable device, the prompt message A can also be displayed or played through the wearable device.

[0208] After step 545, the electronic device may continue to step 530 to reacquire new user action information and media files.

[0209] 550. The electronic device adds the animation effect corresponding to the target action to media file A, resulting in media file B.

[0210] When the electronic device determines that the action information A can match the preset target action, it adds the animation effect corresponding to the target action to the media file A to obtain the media file B.

[0211] For example, when adding animation effects corresponding to a target action, an electronic device can add such animation effects to the user's body movements.

[0212] For example, if the target action is a user waving their arm to draw a circle, and the corresponding animation effect is a portal, when adding the portal animation effect to the media file, the electronic device adds the portal animation effect specifically to the circle drawn by the user. Therefore, the animation effect in the final generated media file B is related to the circle drawn by the user. Since the circle drawn by the user each time is not exactly the same, the animation effect of the circle in the generated media file B will also be different for the same circle-drawing action. This greatly enhances the playability of the technical solution and brings a good interactive experience to the user.

[0213] It should be understood that each target action can have a corresponding animation effect. The mode of this animation effect (such as a portal, magic wand, etc.) can be the system default, added by the user, or downloaded by the user from the theme market. This application embodiment does not limit this. 560, The electronic device plays media file B.

[0214] For example, see (c) in 3, the electronic device can play media file B and present corresponding animations to the user.

[0215] 570, The electronic device executes operation A corresponding to the target action.

[0216] For example, operation A can be an operation associated with a target action, such as calling contact A, opening an application, or changing the mode of an electronic device (such as silent mode, vibration mode, etc.); or, in a live streaming scenario, operation A can also be adding special effects, switching background music, switching backgrounds, or turning on beauty filters, etc.

[0217] It should be understood that step 570 may be an optional step, and in some examples, step 570 may not be performed.

[0218] Based on the embodiments of this application, when the electronic device is in an unlocked state, it can use a low-power camera to acquire an image A including the user, and when image A meets preset conditions, it displays the corresponding motion effect A. Subsequently, the electronic device can acquire the user's action information and a media file containing the user's limb movements. When the action information matches a target action, it adds corresponding motion effects to the limb movements in the media file to obtain another media file B, which can then be played to present the corresponding motion effects to the user.

[0219] In some cases, the above-described methods for recognizing user gestures and adding corresponding animation effects in this embodiment can also be applied to third-party applications. For example, third-party applications can call the system's open interfaces to implement the above functions.

[0220] For example, consider a third-party application, live streaming application A. Application A can call the system interfaces of electronic devices, and users can perform actions such as... Figure 2 Similar settings to (b) to (e) in the above, such as setting the motion capture method, setting the motion list, adding custom actions, and setting actions associated with operations.

[0221] In current live streaming scenarios, users may use gestures to perform simple operations. In this application, users can set their body movements and associated operations within application A, enhancing the fun and convenience of the operation.

[0222] For example, during a live stream, an electronic device can capture a user's body movement 1. When body movement 1 is associated with a target operation, application A can execute that target operation, such as switching background music, adding or turning off or changing special effects, turning beauty filters on or off, or turning off the live stream audio or video.

[0223] In this way, users can make the application perform different operations through different body movements, thereby enhancing the fun and convenience of human-computer interaction.

[0224] Figure 7 This is a schematic flowchart illustrating a human-computer interaction method provided in an embodiment of this application. Figure 7 As shown, the method 600 can be applied to a first electronic device, and the method 600 may include steps 610 to 650.

[0225] 610, The first electronic device uses a first camera to acquire a first image of the user's first limb.

[0226] In some cases, the first electronic device can use its first camera to acquire a first image of the user's first limb while the device is unlocked. Alternatively, the first electronic device can also use its first camera to acquire a first image of the user's first limb after receiving an instruction from another electronic device while the device is locked.

[0227] A user's body movements can include single body movements, such as gestures, or combined movements of multiple body movements, such as drawing a preset shape with arms and hands, or a certain dance move, etc.

[0228] 620, the first electronic device displays a first prompt message based on the first limb in the first image, the first prompt message being used to prompt the user to begin performing a limb movement.

[0229] For example, when the first limb in the first image includes a limb required for one or more preset actions in the system, the first electronic device may display a first prompt message to prompt the user to start performing the limb action.

[0230] The embodiments of this application do not limit the specific form of the first prompt information. For example, the first prompt information may include icons, animations, text, etc.

[0231] The first electronic device can display the same or different prompts for different body movements.

[0232] 630, the first electronic device acquires first action information corresponding to the user's limb movement and a first media file of the user's limb movement, wherein the first action information includes three-dimensional information of the user's limbs.

[0233] It should be understood that the three-dimensional information of the user's limbs can be determined by depth sensing information or by the relative motion trajectory determined by the star flash method.

[0234] 640, if the first action information can match the preset target action, the first electronic device adds the first motion effect corresponding to the target action to the first media file to obtain the second media file.

[0235] In some examples, if the similarity between the first action information and the target action is greater than a preset value, it can be considered that the first action information can match the preset target action.

[0236] The specific value of the preset value is not limited in the embodiments of this application. For example, the preset value can be 80% or 70%, etc.

[0237] The target action and the first animation can be preset in the system of the first electronic device, added manually by the user, or downloaded by the first electronic device from the theme market.

[0238] 650, The first electronic device plays the second media file.

[0239] It should be understood that the embodiments of this application do not limit the form in which the first electronic device plays the second media file. For example, the first electronic device can play the second media file in full screen, or it can play the second media file in half screen.

[0240] Based on the embodiments of this application, a first electronic device acquires a first image of a user's first limb through a first camera, and displays a first prompt message based on the first limb to prompt the user to start performing a limb movement; when the user performs a limb movement, the device acquires the user's first movement information and a first media file of the user performing the limb movement, and when the first movement information can match a target movement, the device adds a corresponding first motion effect to the first media file to obtain a second media file, and plays the second media file.

[0241] In this way, the first electronic device can acquire the user's body movement information and the first media file, and add the first animation corresponding to the target action that matches the action information to the first media file. Different actions can correspond to different animations, thereby increasing the playability of human-computer interaction and the diversity of application scenarios.

[0242] In some implementations, the target action corresponds to the first operation; this method also includes:

[0243] After the second media file has finished playing, the first electronic device performs the first operation.

[0244] For example, the first operation can be user-defined or a system default. For instance, the first operation could be making a phone call, opening an application, adding special effects, switching or adding background music, etc.

[0245] For example, see (d) in 3, the first operation is to call contact A.

[0246] Based on the embodiments of this application, the target action can correspond to the first operation, and after the second media file is played, the first electronic device can perform the first operation, thereby improving the playability of human-computer interaction.

[0247] In some implementations, the first action information corresponding to the user's physical action and the first media file of the user's physical action are obtained, including:

[0248] The first electronic device uses a second camera to acquire first motion information and a third camera to acquire a first media file, wherein the second camera is a depth-sensing camera and the third camera is a non-depth-sensing camera.

[0249] Based on the embodiments of this application, the first electronic device uses a depth-sensing camera to acquire the user's first action information and uses a non-depth-sensing camera to acquire the user's first media file of limb movements. This makes the action information acquired by the first electronic device more accurate. Using different types of cameras to acquire different information makes the acquisition efficiency of the first electronic device more efficient.

[0250] In some implementations, the first camera is part or all of the second or third camera.

[0251] For example, the first camera and the second camera may be the same camera, while the third camera is a separate camera. In this case, the electronic device may include two cameras to implement the technical solution. Alternatively, the first camera and the third camera may be the same camera, while the second camera is a separate camera. In this case, the electronic device may include two cameras to implement the technical solution.

[0252] For example, if a portion of the second camera can capture the user's first image, then the first camera can be a portion of the second camera. Or, if a portion of the third camera can capture the user's first image, then the first camera can be a portion of the third camera.

[0253] In other examples, the first camera may be a camera other than the second or third camera. In this case, the electronic device may include three cameras to implement the technical solution.

[0254] In some implementations, the first camera is in low-power mode, or the first camera is a low-power camera.

[0255] The first camera is in a low-power mode, or the first camera is a low-power camera that can reduce the power consumption of the first electronic device when the first camera needs to constantly detect whether the user's first limb is in the field of view.

[0256] In some implementations, a second camera is used to acquire the first motion information, and a third camera is used to acquire the first media file, including:

[0257] The first electronic device starts using the second camera to acquire first motion information at a first moment, and starts using the third camera to acquire a first media file at a second moment, wherein the first moment is the same as the second moment, or the second moment is before or after the first moment, or the second moment is the moment when the third camera receives instruction information indicating that it has started working.

[0258] For example, the instruction information may be sent by a second camera.

[0259] Based on the embodiments of this application, the second camera is used to acquire first action information starting at a first moment, and the third camera is used to acquire the first media file starting at a second moment, with the second moment occurring after the first moment. This ensures that the third camera starts working only after the second camera has begun operating. This avoids the power consumption waste that would occur if the user did not perform any actions for a period of time before the third camera started operating.

[0260] In some implementations, before using the second camera to acquire first motion information at the first moment and using the third camera to acquire the first media file at the second moment, method 600 further includes:

[0261] The first electronic device determines that the first limb in the first image acquired by the first camera includes the limb required in a preset action, wherein the first camera is a low-power camera.

[0262] Based on the embodiments of this application, when the first limb in the first image acquired by the first electronic device through the low-power first camera includes the limb required in the preset action, the second camera is used to acquire the first action information at the first moment, and the third camera is used to acquire the first media file at the second moment. This can avoid the second and third cameras being constantly activated, thereby reducing the power consumption of the first electronic device.

[0263] In some implementations, the first action information corresponding to the user's physical action and the first media file of the user's physical action are obtained, including:

[0264] The first electronic device determines first action information based on the motion trajectory of the second electronic device relative to the first electronic device, and uses a third camera to acquire a first media file, wherein the second electronic device is communicatively connected to the first electronic device.

[0265] For example, the first electronic device and the second electronic device can be connected via a star connection or a Bluetooth connection.

[0266] Based on the embodiments of this application, the first electronic device can also determine the user's first action information by acquiring the motion trajectory of the second electronic device relative to the first electronic device. For example, the second electronic device can be a stylus or a wearable device, which can improve the application scenarios of human-computer interaction and enhance the user's interactive experience.

[0267] In some implementations, before using the first camera to acquire a first image of the user's first limb, the method 600 further includes:

[0268] The first electronic device receives a first instruction message sent by the second electronic device, the first instruction message being used to instruct the first camera to be activated.

[0269] For example, the second electronic device is a stylus. When the stylus detects that the user is pressing the pen body or a button, it can send the first instruction information to the first electronic device to instruct it to activate the first camera.

[0270] Alternatively, the second electronic device is a smartwatch. When the smartwatch detects the user pressing the knob, it can send the first instruction information to the first electronic device to instruct it to activate the first camera.

[0271] Based on the embodiments of this application, the first electronic device only starts the first camera to acquire the first image after receiving the first instruction information sent by the second electronic device, so that the first camera can be started on demand, avoiding the continuous operation of the first camera, thereby reducing the power consumption of the electronic device.

[0272] In some implementations, before determining the first motion information based on the motion trajectory of the second electronic device relative to the first electronic device and acquiring the first media file using the third camera, the method 600 further includes:

[0273] The first electronic device determines that the first limb in the first image acquired by the first camera includes the limb required in a preset action, wherein the first camera is a low-power camera.

[0274] In this way, the first electronic device only starts using the second camera to acquire the first action information when the first limb in the first image acquired by the first electronic device through the low-power first camera includes the limb required in the preset action, and starts using the third camera to acquire the first media file at the second moment. This avoids the second and third cameras from being constantly activated, thereby reducing the power consumption of the first electronic device.

[0275] In some implementations, a first animation corresponding to the target action is added to the first media file to obtain the second media file, including:

[0276] The first electronic device adds a first motion effect to the body movements included in the first media file to obtain a second media file.

[0277] In this way, since the user's body movements will not be exactly the same each time, the resulting second media file will be different. This can improve the user's interactive experience.

[0278] In some implementations, the first display interface of the first electronic device in an unlocked state plays a second media file, including:

[0279] The first electronic device plays a second media file on a first display interface.

[0280] For example, the display area of ​​the second media file can cover part of the first display interface, such as being located in the middle or upper part of the first display interface.

[0281] Based on the embodiments of this application, a second media file is played on a first display interface, so that the user can both browse the second media file and know the current specific display interface.

[0282] In some implementations, the first electronic device is in an unlocked state with a second display interface, which is the display interface of a third-party application. After the second media file finishes playing, the first operation is performed, including:

[0283] After the first electronic device finishes playing the second media file using a third-party application, it performs the first operation using the third-party application.

[0284] For example, the third-party application can be a live streaming application. For instance, the first operation could be adding background music, adding special effects, etc. This allows users to conveniently control the first electronic device to perform operations through body language interaction, even when the user is far away from the device.

[0285] In this way, third-party applications can also use the above-mentioned interactive functions, thereby increasing the application diversity of this human-computer interaction.

[0286] In some implementations, before using the first camera to acquire a first image of the user's limbs, the method 600 further includes:

[0287] In response to the user's settings, the first electronic device associates the target action with the first operation.

[0288] Based on the embodiments of this application, users can set actions and associated first operations in the first electronic device, thereby greatly enhancing users' independent creativity and improving the subsequent interaction experience with users.

[0289] In some implementations, the target action is either preset by the system or added based on the user's operation; the first operation is either preset by the system or added based on the user's operation.

[0290] In other examples, the target action and the first operation could also be something the user downloaded from an app store.

[0291] In some implementations, method 600 also includes:

[0292] Play a demonstration animation of the target action;

[0293] Use a depth camera to acquire secondary motion information of the user's body movements;

[0294] When it is determined that the action indicated by the second action information matches the target action, a second prompt message is displayed. The second prompt message is used to prompt the user that the target action learning has been completed.

[0295] Alternatively, if it is determined that the action indicated by the second action information does not match the target action, a third prompt message is displayed to remind the user that the limb movement needs to be corrected.

[0296] Based on the embodiments of this application, the first electronic device can also play a demonstration of the target action to allow the user to learn, and can verify the user's learning results and provide guidance for the target action.

[0297] In some implementations, displaying a first prompt message based on a first limb in a first image includes:

[0298] When the first electronic device matches the first limb with any limb included in a preset action, it displays a first prompt message.

[0299] It should be understood that the first limb here matches the limbs included in any preset action, which can be understood as the first limb including the limbs required in any preset action.

[0300] For example, if the first limb is the head and hand, and the preset action 1 requires the head as the limb, and action 2 requires the hand as the limb, then the first limb includes the limbs required for both action 1 and action 2. This first prompt information can be the head and / or hand, or a uniform icon, etc.

[0301] Based on the embodiments of this application, when it is determined that the first limb matches a limb included in any preset action, it can be determined that the user may have a need for human-computer interaction. Therefore, a first prompt message can be displayed to prompt the user to start performing a limb movement. This avoids the user performing an invalid action because the first electronic device has not started recognizing the limb.

[0302] In some implementations, the first prompt message includes an icon or animation of the first limb.

[0303] In other examples, the first prompt message includes an icon or animation of a limb included in any of the aforementioned preset actions.

[0304] In this way, the first electronic device provides users with more accurate action guidance.

[0305] For example, Figure 8 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Figure 8As shown, the electronic device 900 includes one or more processors 910; one or more memories 920; the one or more memories 920 storing one or more instructions that, when executed by one or more processors 910, cause the method described in any of the possible implementations above to be performed.

[0306] For example, the electronic device 900 can be the electronic device 100, electronic device 200 mentioned above, the electronic device in the embodiment, the first electronic device, the second electronic device, etc.

[0307] The electronic device 900 can be used to perform methods 500 and 600 mentioned above.

[0308] This application also provides an electronic device, including a processor, a memory, and a communication interface, wherein the communication interface is used to receive signals, the memory is used to store signals, the communication interface is also used to transmit signals to the processor, and the processor processes the signals so that the human-computer interaction method as described in any of the possible implementations above is executed.

[0309] This application also provides a device for switching between display interfaces, including a processor and a communication interface. The communication interface is used to receive signals and transmit the signals to the processor. The processor processes the signals so that the human-computer interaction method described in any of the possible implementations above is executed.

[0310] The device can be a chip. For example, the chip can be a chip system or a standalone chip.

[0311] This application also provides a readable storage medium (also known as a computer-readable storage medium) storing instructions that, when executed on an electronic device, cause the electronic device to perform the aforementioned method steps to implement the human-computer interaction method described in the above embodiments.

[0312] This application also provides a program product (also known as a computer program product) that, when run on an electronic device, causes the electronic device to perform the aforementioned steps to realize the human-computer interaction method described in the above embodiments.

[0313] This application also provides an apparatus including a module for implementing the human-computer interaction method as described in any of the foregoing embodiments.

[0314] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store instructions, and when the apparatus is running, the processor may execute the instructions stored in the memory to cause the apparatus to perform the human-computer interaction methods in the above-described method embodiments.

[0315] In this embodiment, the device, readable storage medium, program product or apparatus are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0316] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0317] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0318] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0319] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0320] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0321] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0322] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for human-computer interaction, characterized in that, The method is applied to a first electronic device, and the method includes: Use the first camera to capture the first image of the user's first limb; Based on the first limb in the first image, a first prompt message is displayed, which prompts the user to start performing a limb movement. Acquire first action information corresponding to the user's limb movement and a first media file of the user's limb movement, wherein the first action information includes three-dimensional information of the user's limbs; If the first action information can match a preset target action, add a first motion effect corresponding to the target action to the first media file to obtain a second media file; Play the second media file.

2. The method according to claim 1, characterized in that, The target action corresponds to the first operation, and the method further includes: After the second media file has finished playing, perform the first operation.

3. The method according to claim 1 or 2, characterized in that, The step of obtaining the first action information corresponding to the user's physical action and the first media file of the user's physical action includes: The first motion information is acquired using a second camera, and the first media file is acquired using a third camera, wherein the second camera is a depth-sensing camera and the third camera is a non-depth-sensing camera.

4. The method according to claim 3, characterized in that, The first camera is part or all of the second camera or the third camera.

5. The method according to any one of claims 1-4, characterized in that, The first camera is in low power mode, or the first camera is a low power camera.

6. The method according to any one of claims 3-5, characterized in that, The step of using a second camera to acquire the first motion information and using a third camera to acquire the first media file includes: The first action information is acquired using a second camera at a first moment, and the first media file is acquired using a third camera at a second moment, wherein the first moment is the same as the second moment, or the second moment is before or after the first moment, or the second moment is the moment when the third camera receives the instruction information indicating that it should start working.

7. The method according to claim 6, characterized in that, Before using the second camera to acquire the first motion information at the first moment and using the third camera to acquire the first media file at the second moment, the method further includes: It is determined that the first limb in the first image acquired by the first camera includes the limb required in a preset action, wherein the first camera is a low-power camera.

8. The method according to claim 1 or 2, characterized in that, The step of obtaining the first action information corresponding to the user's physical action and the first media file of the user's physical action includes: The first action information is determined based on the motion trajectory of the second electronic device relative to the first electronic device, and the first media file is acquired using a third camera, wherein the second electronic device is communicatively connected to the first electronic device.

9. The method according to claim 8, characterized in that, Before acquiring a first image of the user's first limb using the first camera, the method further includes: The device receives a first instruction message sent by the second electronic device, the first instruction message being used to instruct the first camera to be activated.

10. The method according to claim 9, characterized in that, Before determining the first motion information based on the motion trajectory of the second electronic device relative to the first electronic device and acquiring the first media file using the third camera, the method further includes: It is determined that the first limb in the first image acquired by the first camera includes the limb required in a preset action, wherein the first camera is a low-power camera.

11. The method according to any one of claims 1-10, characterized in that, Adding a first animation effect corresponding to the target action to the first media file to obtain a second media file includes: The first motion effect is added to the body movements included in the first media file to obtain the second media file.

12. The method according to any one of claims 1-11, characterized in that, The first display interface of the first electronic device in an unlocked state, the playback of the second media file includes: Play the second media file on the first display interface.

13. The method according to claim 2, characterized in that, The first electronic device is in an unlocked state with a second display interface, which is the display interface of a third-party application. The step of performing the first operation after the second media file has finished playing includes: After playing the second media file using the third-party application, the first operation is performed using the third-party application.

14. The method according to any one of claims 2-13, characterized in that, Before acquiring the first image of the user's limb using the first camera, the method further includes: In response to the user's settings, the target action is associated with the first action.

15. The method according to claim 14, characterized in that, The target action is either preset by the system or added based on the user's operation; the first operation is either preset by the system or added based on the user's operation.

16. The method according to claim 14 or 15, characterized in that, The method further includes: Play a demonstration animation of the target action; Use a depth camera to acquire secondary motion information of the user's body movements; When it is determined that the action indicated by the second action information matches the target action, a second prompt message is displayed to indicate to the user that the target action has been learned; or... When it is determined that the action indicated by the second action information does not match the target action, a third prompt message is displayed, which is used to prompt the user to correct the limb movement.

17. The method according to any one of claims 1-16, characterized in that, Based on the first limb in the first image, display the first prompt information, including: When the first limb matches any limb included in any preset action, the first prompt message is displayed.

18. The method according to any one of claims 1-17, characterized in that, The first prompt message includes an icon or animation of the first limb.

19. An electronic device, characterized in that, include: One or more processors; One or more memories; the one or more memories storing one or more programs that, when executed by one or more processors, cause the method as described in any one of claims 1-18 to be performed.

20. A chip, characterized in that, The chip includes a processor and a communication interface, the communication interface being used to receive signals and transmit the signals to the processor, the processor processing the signals such that the method as described in any one of claims 1-18 is executed.

21. A readable storage medium, characterized in that, The readable storage medium stores instructions that, when executed on an electronic device, cause the method as described in any one of claims 1-18 to be performed.

22. A program product, characterized in that, The program product includes program code that, when run on an electronic device, causes the method as described in any one of claims 1-18 to be performed.

Images