Electronic equipment and gesture control method

Abstract

The embodiment of the invention discloses electronic equipment and a gesture control method. The method comprises the steps of obtaining a scene image collected by an image collector; performing gesture recognition on the scene image; when it is recognized that the scene image contains the target gesture, the target gesture is compared with a previous gesture, and the previous gesture is the gesture recognized in the previous frame of scene image; if the type of the target gesture is the same as that of the previous gesture, matching and executing a target function according to the working state of the electronic equipment, the execution state of the gesture control function and the displacement relation between the target gesture and the previous gesture. According to the gesture control method and the gesture control device, the matched gesture function control process is executed by comprehensively considering the conditions of the equipment working state, the execution state (for example, started or not started) of the gesture control function, whether the gesture is displaced or not and the like, so that the accuracy of static and dynamic gesture control is improved.

Description

Electronic device and gesture control method

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent applications filed with the China Patent Office on March 28, 2023, with application number 202310313940.5, filed with the China Patent Office on June 5, 2023, with application number 202310658455.1, filed with the China Patent Office on June 5, 2023, with application number 202310660088.9, and filed with the China Patent Office on June 6, 2023, with application number 202310660405.7, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of interactive technology, and in particular to an electronic device and a gesture control method. Background Art

[0004] Gesture control of electronic devices can be either static or motion gesture control. Static gesture control involves controlling an electronic device using static gestures. For example, using an "OK" gesture to mute or unmute an electronic device involves mute. If the electronic device is currently playing sound and the user points the "OK" gesture at the camera, the device recognizes the gesture and mutes the device.

[0005] Motion gesture control refers to controlling electronic devices using motion gestures, such as using a "fist" gesture to move left and right to adjust the volume. Assuming that the user keeps the "fist" gesture unchanged and moves the gesture to the right, the electronic device recognizes the motion gesture and increases the volume.

[0006] Static gesture control can result in incorrect control. For example, if the user does not immediately lower their hand after gesture recognition or lowers it slowly, gesture recognition and gesture control commands may be repeated, resulting in inaccurate control. When executing motion gesture control, interference from factors such as user hand jitter and hand and limb motion constraints can lead to inaccurate control or the inability to achieve sustained and stable control.

[0007] Summary of the Invention

[0008] An embodiment of the present application provides an electronic device, comprising: a display for displaying images and / or a user interface; a user interface configured to receive instructions from a user; a communication device configured to communicate with an external device according to a predetermined protocol; a memory configured to store computer instructions and data associated with the display device; an image collector for collecting scene images around the display; at least one processor connected to the display, user interface, communicator and memory, and configured to execute computer instructions so that the display device performs: gesture recognition on the scene image collected by the image collector; when a target gesture is recognized as being included in the scene image, comparing the target gesture with a previous gesture, the previous gesture being a gesture recognized in the previous frame of the scene image; if the target gesture and the previous gesture are of the same category, matching and executing the target function based on the working state of the electronic device, the execution state of the gesture control function and the displacement relationship between the target gesture and the previous gesture.

[0009] An embodiment of the present application also provides a gesture control method, including: acquiring a scene image captured by an image collector; performing gesture recognition on the scene image; when recognizing that the scene image contains a target gesture, comparing the target gesture with a previous gesture, where the previous gesture is a gesture recognized in the previous frame of the scene image; if the target gesture and the previous gesture are of the same category, matching and executing the target function based on the working state of the electronic device, the execution state of the gesture control function, and the displacement relationship between the target gesture and the previous gesture.

[0010] An embodiment of the present application further provides a computer non-volatile storage medium, in which program instructions are stored. When the program instructions are executed on a computer, the computer executes the methods involved in the above aspects and their respective implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] FIG1 illustrates an operation scenario between an electronic device and a control device 100 according to an embodiment of the present application;

[0012] FIG2 is a block diagram of a hardware configuration of a control device 100 according to an embodiment of the present application;

[0013] FIG3 is a hardware configuration block diagram of an electronic device 200 according to an embodiment of the present application;

[0014] FIG4 is a diagram showing a software configuration in an electronic device 200 according to an embodiment of the present application;

[0015] FIG5 is a schematic diagram of a first guide page displayed when an electronic device is connected to a camera for the first time according to an embodiment of the present application;

[0016] FIG6 is a schematic diagram of a second guide page displayed when an electronic device is connected to a camera for the first time according to an embodiment of the present application;

[0017] FIG7 is a schematic diagram of a third guide page displayed when the electronic device is connected to a camera for the first time according to an embodiment of the present application;

[0018] FIG8 is a schematic diagram of a fourth guide page displayed when the electronic device is connected to a camera for the first time according to an embodiment of the present application;

[0019] FIG9 is an exit guide page according to an embodiment of the present application;

[0020] FIG10 is a schematic diagram of an experience page for an automatic audio adjustment function according to an embodiment of the present application;

[0021] FIG11 is a schematic diagram of a fifth guide page displayed when an electronic device is connected to a camera for the first time according to an embodiment of the present application;

[0022] FIG12 is a schematic diagram of an experience page for an automatic brightness adjustment function according to an embodiment of the present application;

[0023] FIG13 is a schematic diagram of a sixth guide page displayed when the electronic device is connected to a camera for the first time according to an embodiment of the present application;

[0024] FIG14 is a schematic diagram of an experience page for a gesture control function according to an embodiment of the present application;

[0025] FIG15 is a schematic diagram of a dynamic gesture recognition process according to an embodiment of the present application;

[0026] FIG16 is a schematic diagram of a gesture setting page according to an embodiment of the present application;

[0027] FIG17 is a schematic diagram showing changes in a gesture setting page according to an embodiment of the present application;

[0028] FIG18 is a schematic diagram of a seventh guide page displayed when the electronic device is connected to a camera for the first time according to an embodiment of the present application;

[0029] FIG19 is a schematic diagram of a first camera function setting page according to an embodiment of the present application;

[0030] FIG20 is a schematic diagram of a second camera function setting page according to an embodiment of the present application;

[0031] FIG21 is a schematic diagram of a third camera function setting page according to an embodiment of the present application;

[0032] FIG22 is a schematic diagram of a fourth camera function setting page according to an embodiment of the present application;

[0033] FIG23 is a schematic diagram of a fifth camera function setting page according to an embodiment of the present application;

[0034] FIG24 is a schematic diagram of a sixth camera function setting page according to an embodiment of the present application;

[0035] FIG25 is a schematic diagram of a seventh camera function setting page according to an embodiment of the present application;

[0036] FIG26 is a schematic diagram of the processing logic of the automatic brightness adjustment function according to an embodiment of the present application;

[0037] FIG27 is a schematic diagram of the processing logic of the gesture control function according to an embodiment of the present application;

[0038] FIG28 is a schematic diagram of displaying a second prompt pop-up window when no camera is detected according to an embodiment of the present application;

[0039] FIG29 is a schematic diagram of a hand gesture jitter according to an embodiment of the present application;

[0040] FIG30 is a schematic diagram of a system architecture of an electronic device according to an embodiment of the present application;

[0041] FIG31 is a flowchart of a first gesture control method according to an embodiment of the present application;

[0042] FIG32 is a schematic diagram showing the principle of reducing jitter of a motion gesture according to an embodiment of the present application;

[0043] FIG33 is a schematic diagram showing the principle of continuous motion gesture control according to an embodiment of the present application;

[0044] FIG34 is a schematic diagram showing an effect of an electronic device controlling a browser using a cursor of a remote controller according to an embodiment of the present application;

[0045] FIG35 is a schematic diagram showing another effect of an electronic device controlling a browser using a cursor of a remote controller according to an embodiment of the present application;

[0046] FIG36 is a schematic diagram illustrating an electronic device switching tabs according to a remote control cursor according to an embodiment of the present application;

[0047] FIG37 is a schematic diagram of a process of an electronic device executing a browser control based on gesture recognition according to an embodiment of the present application;

[0048] FIG38 is a schematic diagram of a process for an electronic device to determine an operation category for controlling a browser according to an embodiment of the present application;

[0049] FIG39 is a schematic diagram illustrating determining the operation category of controlling a browser by means of a switch control according to an embodiment of the present application;

[0050] FIG40 is a schematic diagram showing an effect of an electronic device displaying a gesture control category reminder message according to an embodiment of the present application;

[0051] FIG41 is a schematic diagram of a process of generating dynamic control instructions from gesture categories by an electronic device according to an embodiment of the present application;

[0052] FIG42 is a schematic diagram of a scenario in which an electronic device generates dynamic control instructions from gesture categories according to an embodiment of the present application;

[0053] FIG43 is a schematic diagram of a flow chart of an electronic device generating a control instruction according to an embodiment of the present application;

[0054] FIG44 is a schematic diagram of a flow chart of executing a static control instruction for display settings according to an embodiment of the present application;

[0055] FIG45 is a schematic diagram showing the principle of determining the direction of a gesture movement by an electronic device according to an embodiment of the present application;

[0056] FIG46 is a flowchart of a method for controlling a browser based on gesture recognition according to an embodiment of the present application;

[0057] FIG47 is a schematic diagram showing a comparison of interfaces for compressing an original image into a target image according to an embodiment of the present application;

[0058] FIG48 is a schematic diagram of gesture pixels before compression according to an embodiment of the present application;

[0059] FIG49 is a schematic diagram of compressed gesture pixels according to an embodiment of the present application;

[0060] FIG50 is a schematic diagram of software configuration of a computer device according to an embodiment of the present application;

[0061] FIG51 is a schematic diagram showing an icon control page display of an application program of a computer device according to an embodiment of the present application;

[0062] FIG52 is a flowchart of a method for long-distance gesture recognition according to an embodiment of the present application;

[0063] FIG53 is a second flow chart of a method for long-distance gesture recognition according to an embodiment of the present application;

[0064] FIG54 is a third flow chart of a method for long-distance gesture recognition according to an embodiment of the present application;

[0065] FIG55 is a fourth flow chart of a method for long-distance gesture recognition according to an embodiment of the present application;

[0066] FIG56 is a fifth flowchart of a method for long-distance gesture recognition according to an embodiment of the present application;

[0067] FIG57 is a schematic diagram showing the interface comparison of compressing an original image and cropping it to obtain a target image according to an embodiment of the present application;

[0068] FIG58 is a schematic diagram of gesture pixels before the second compression according to an embodiment of the present application;

[0069] FIG59 is a schematic diagram of gesture pixels after a second compression according to an embodiment of the present application;

[0070] FIG60 is a schematic structural diagram of a computer device according to an embodiment of the present application;

[0071] FIG61 is a sixth flowchart of a method for long-distance gesture recognition according to an embodiment of the present application;

[0072] FIG62 is a seventh flowchart of a method for long-distance gesture recognition according to an embodiment of the present application;

[0073] FIG63A is an eighth flowchart of a method for long-distance gesture recognition according to an embodiment of the present application;

[0074] FIG63B is a ninth flowchart of the long-distance gesture recognition method according to an embodiment of the present application. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical implementation and advantages of the exemplary embodiments of the present application clearer, the technical implementation in the exemplary embodiments of the present application will be clearly and completely described below in combination with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0076] All other embodiments derived by persons of ordinary skill in the art based on the exemplary embodiments described in this application without inventive effort are within the scope of protection of this application. Furthermore, while the disclosure herein is presented based on one or more exemplary embodiments, it should be understood that each aspect of the disclosure may constitute a complete technical implementation.

[0077] It should be understood that the terms "first," "second," "third," etc., in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, for example, enabling implementation in an order other than that shown or described in the embodiments of this application.

[0078] The electronic devices provided in some embodiments of the present application may have various implementation forms, such as televisions, smart TVs, laser projection devices, monitors, electronic bulletin boards, electronic tables, etc. Figures 1 and 2 illustrate a specific implementation of the electronic device of the present application.

[0079] Figure 1 is a schematic diagram of an operation scenario between an electronic device and a control device provided in some embodiments of the present application. As shown in Figure 1, a user can operate an electronic device 200 through a smart device 300 or a control device 100.

[0080] In some implementations, the control device 100 may be a remote controller, and communication between the remote controller and the electronic device may include infrared protocol communication, Bluetooth protocol communication, or other short-range communication methods, to control the electronic device 200 wirelessly or wiredly. The user may control the electronic device 200 by inputting user commands through buttons on the remote controller, voice input, control panel input, and the like.

[0081] In some implementations, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) may also be used to control the electronic device 200. For example, the electronic device 200 may be controlled using an application running on the smart device.

[0082] In some implementations, the electronic device may not use the aforementioned smart device or control device to receive instructions, but may receive user control through touch or gestures.

[0083] In some implementations, the electronic device 200 can also be controlled in ways other than the control device 100 and the smart device 300. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the electronic device 200, or the user's voice command control can be received through a voice control device set outside the electronic device 200.

[0084] In some implementations, the electronic device 200 also communicates data with the server 400. The electronic device 200 may be connected to the server 400 via a local area network (LAN), a wireless local area network (WLAN), or other networks. The server 400 may provide various content and interactions to the electronic device 200. The server 400 may be a single cluster or multiple clusters, and may include one or more types of servers.

[0085] Figure 2 is a block diagram of a control device 100 provided in some embodiments of the present application. As shown in Figure 2, the control device 100 includes a processor 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive user input commands and convert them into commands that the electronic device 200 can recognize and respond to, acting as an intermediary for interaction between the user and the electronic device 200.

[0086] FIG3 is a block diagram of the hardware configuration of an electronic device 200 provided in some embodiments of the present application. As shown in FIG3 , the electronic device 200 includes at least one of a tuner / demodulator 210, a communicator 220, a detector 230, an external device interface 240, a processor 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0087] In some implementations, the processor includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and first to nth interfaces for input / output.

[0088] The display 260 includes a display screen component for presenting images, a driving component for driving image display, a component for receiving image signals output from a processor, and a component for displaying video content, image content, and a menu control interface and a user control UI interface.

[0089] The display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0090] Communicator 220 is a component used to communicate with external devices or servers using various communication protocols. For example, the communicator may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, or other network communication protocol chip, a near-field communication protocol chip, and an infrared receiver. Electronic device 200 can use communicator 220 to send and receive control signals and data signals with control device 100 or server 400.

[0091] The user interface can be used to receive control signals from the control device 100 (such as an infrared remote controller, etc.).

[0092] Detector 230 is used to collect signals from the external environment or external interactions. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or detector 230 includes an image collector, such as a camera, for collecting external environmental scenes, user attributes, or user interaction gestures; or detector 230 includes a sound collector, such as a microphone, for receiving external sounds.

[0093] The external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by multiple of the above interfaces.

[0094] The tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals and EPG data signals from a plurality of wireless or wired broadcast television signals.

[0095] In some implementations, the processor 250 and the tuner / demodulator 210 may be located in different separate devices, that is, the tuner / demodulator 210 may be located in an external device of the main device where the processor 250 is located, such as an external set-top box.

[0096] Processor 250 controls the operation of the electronic device and responds to user operations through various software control programs stored in the memory. Processor 250 controls the overall operation of electronic device 200. For example, in response to receiving a user command to select a UI object to be displayed on display 260, processor 250 may perform operations related to the object selected by the user command.

[0097] In some implementations, the processor includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM Random Access Memory (RAM), ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus, etc.

[0098] The user may input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command through a specific voice or gesture, and the user input interface may recognize the voice or gesture through a sensor to receive the user input command.

[0099] FIG4 is a diagram of the software configuration of the electronic device 200 provided in some embodiments of the present application. As shown in FIG4 , in some implementations, the system is divided into four layers, namely, from top to bottom, the application layer (hereinafter referred to as the "application layer"), the application framework layer (hereinafter referred to as the "framework layer"), the Android runtime and system library layer (hereinafter referred to as the "system runtime library layer"), and the kernel layer.

[0100] In some implementations, at least one application runs in the application layer. These applications can be window programs, system settings programs, clock programs, etc. that come with the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0101] The framework layer provides applications with an application programming interface (API) and programming framework. The application framework layer includes predefined functions. The application framework layer acts as a processing center, determining the actions taken by applications in the application layer. Through the API, applications can access system resources and services during execution.

[0102] As shown in Figure 4, the application framework layer in the embodiment of the present application includes managers, content providers, etc., wherein the manager includes at least one of the following modules: an activity manager (Activity Manager) is used to interact with all activities running in the system; a location manager (Location Manager) is used to provide system services or applications with access to system location services; a package manager (Package Manager) is used to retrieve various information related to the application package currently installed on the device; a notification manager (Notification Manager) is used to control the display and clearing of notification messages; a window manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers and desktop components on the user interface.

[0103] In some implementations, the activity manager manages the lifecycle of each application and common navigation back functions, such as controlling application exit, opening, and backing. The window manager manages all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling display window changes (such as shrinking the display window, shaking the display, distorting the display, etc.).

[0104] In some implementations, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0105] In some implementations, the kernel layer is a layer between hardware and software. As shown in FIG4 , the kernel layer includes at least one of the following drivers: an audio driver, a display driver, a Bluetooth driver, a camera driver, a Wi-Fi driver, a USB driver, an HDMI driver, a sensor driver (such as a fingerprint sensor, a temperature sensor, a pressure sensor, etc.), and a power driver.

[0106] For electronic devices with the same or similar hardware and software configurations as those described above, referring to FIG3 , the detector 230 includes an image collector, which can specifically be a camera. The camera can be hardware configured in the electronic device or a hardware device externally connected by a user via an external device interface 240 (e.g., a USB interface). The camera can be placed at a designated location on the electronic device, such as at the top of the frame of the electronic device, with the lens facing the front of the screen of the electronic device to capture images of scenes within the photographic area in front of the screen.

[0107] When a user controls an electronic device through interactive operations such as hand gestures, the camera can capture a user image that includes an image of the user's hand gesture. The processor 250 can then identify the gesture category indicated by the user's hand gesture image, generate a gesture control instruction that matches the gesture category, and send the gesture control instruction to a designated module. For example, if the electronic device is currently playing sound and the processor recognizes the user making an "OK" gesture, it can generate and send a first control instruction to the sound control module. In response to the first control instruction, the sound control module mutes the sound player.

[0108] When the electronic device is connected to a camera for the first time via the external device interface 240 (eg, a USB interface), a function boot process related to the camera may be started, which will be described in detail below.

[0109] Figure 5 is a schematic diagram of a first boot page displayed when an electronic device is first connected to a camera, according to some embodiments of the present application. When processor 250 detects that the electronic device is first connected to a camera, it can control display 260 to display first boot page 50, as shown in Figure 5. First boot page 50 includes a first prompt message 51, a function preview image 52, a first immediate experience control 53, and a later experience control 54.

[0110] The first prompt 51 is used to prompt the user to connect a camera to experience more TV functions. The function preview image 52 includes at least one preview image of a camera-related function. The multiple preview images can be displayed in different layouts, such as carousel, side-by-side, and matrix. If the carousel mode is used, a preview image can be switched every certain time interval (for example, 2 seconds). The camera-related functions include but are not limited to automatic audio adjustment, automatic brightness adjustment, and gesture control. In response to the user triggering the "Experience Later" control 54, the processor 250 controls the display 260 to close the first guide page 50.

[0111] Figure 6 is a schematic diagram of the second guide page displayed when the electronic device provided by some embodiments of the present application is connected to the camera for the first time. In response to the user triggering the operation of the first immediate experience control 53, the processor 250 can control the display 260 to display the second guide page 60 as shown in Figure 6. The second guide page 60 includes a second prompt message 61, a first confirmation control 62 and a first return control 63. Among them, the second prompt message 61 is used to ask whether the use of the camera is allowed, and prompts the user to click "Confirm" to experience the function of the camera, and no images that need to be identified will be collected. In response to the user triggering the operation of the first return control 63, the processor controls the display to switch to the first guide page 50.

[0112] Figure 7 is a schematic diagram of the third guide page displayed when the electronic device provided by some embodiments of the present application is connected to the camera for the first time. In response to the user triggering the operation of the first confirmation control 62, the processor 250 can control the display 260 to display the third guide page 70 as shown in Figure 7. The third guide page 70 includes a shooting preview window 71, a third prompt message 72 and a next control 73. Among them, the shooting preview window 71 is used to display the image captured by the camera in real time. The third prompt message 72 is used to prompt that the recognizable range is within 3.5 meters of the camera. For the best experience, place the camera in the center of the TV and ensure that the user's face and upper body are visible inside the shooting preview window.

[0113] Figure 8 is a schematic diagram of the fourth guide page displayed when the electronic device provided by some embodiments of the present application is connected to the camera for the first time. In response to the user triggering the operation of the next step control 73, the processor 250 can control the display 260 to display the fourth guide page 80 as shown in Figure 8. The fourth guide page 80 includes a fourth prompt message 81, a second immediate experience control 82, a first jump control 83 and a second jump control 84. Among them, the fourth prompt message 81 is used to prompt that the audio automatic adjustment function automatically adjusts the sound balance according to the position to obtain a better audio experience, and prompts that clicking "Experience Now" will play the audio. At this time, the selector is on the second immediate experience control 82 by default, that is, the second immediate experience control 82 has focus.

[0114] When the display 260 displays the fourth guide page 80, the processor 250 controls the display to display the exit guide page 90 shown in FIG9 in response to a return operation input by the user, such as when the user clicks the return key of the control device 100. The exit guide page 90 includes a fifth prompt message 91, a cancel control 92, and an exit control 93. The selector is on the cancel control 92 by default. The fifth prompt message 91 is used to confirm with the user whether to exit the guide and to prompt that the settings cannot be restored after exiting. In response to the user triggering the cancel control 92 or the return operation, the processor closes the exit guide page 90 and returns to the fourth guide page 80. In response to the user triggering the exit control 93, the processor closes the exit guide page 90, ending the camera-related function guidance process. The completed settings are not saved and do not take effect.

[0115] Figure 10 is a schematic diagram of the experience page of the audio automatic adjustment function provided in some embodiments of the present application. In response to the user triggering the operation of the second immediate experience control 82, the processor 250 can control the display 260 to display the experience page 101 of the audio automatic adjustment function as shown in Figure 10, and the experience page 101 includes a sixth prompt information 102 and at least two first guide images 103. Among them, the sixth prompt information 102 is used to prompt the user to stay on the left or right side of the TV and experience the audio automatic adjustment function by changing positions. The first guide image 103 can be a dynamic image, which is used to present the difference in audio output when the user moves in different directions in front of the electronic device (for example: left or right), that is, the electronic device can recognize the actual position of the user relative to the TV based on the captured image, thereby adjusting the output of the left and right audio.

[0116] When the display 260 displays the experience page 101, the processor 250 controls the display to switch back to the fourth guide page 80 in response to the user's input return operation, and moves the selector to the position of the first jump control 83. In this way, the processor 250 records the automatic audio adjustment function as being on in response to the user's operation of triggering the first jump control 83, and controls the display 260 to jump to the next guide page. In response to the user's operation of triggering the second jump control 84, the processor 250 records the automatic audio adjustment function as being off, and controls the display 260 to jump to the next guide page.

[0117] Figure 11 is a schematic diagram of the fifth guide page displayed when the electronic device provided by some embodiments of the present application is connected to the camera for the first time. In response to the user triggering the operation of the first jump control 83 or the second jump control 84, the processor 250 can control the display 260 to display the fifth guide page 111 as shown in Figure 11. The fifth guide page 111 includes a seventh prompt message 112, a third immediate experience control 113, a third jump control 114 and a fourth jump control 115. Among them, the seventh prompt message 112 is used to prompt that the automatic brightness adjustment function automatically reduces the screen brightness to maximize energy saving after detecting that the user has left the electronic device, and prompts the default timing time (for example: 5 minutes), and more options are in the settings. At this time, the selector is on the third immediate experience control 113 by default.

[0118] When the display 260 displays the fifth guide page 111 , the processor 250 controls the display to switch to the fourth guide page 80 in response to a return operation input by the user.

[0119] Figure 12 is a schematic diagram of an experience page for the automatic brightness adjustment function provided in some embodiments of the present application. In response to the user triggering the operation of the third immediate experience control 113, the processor 250 can control the display 260 to display the experience page 120 of the automatic brightness adjustment function as shown in Figure 12, and the experience page 120 includes an eighth prompt message 121 and at least two second guide images 122. Among them, the eighth prompt message 121 is used to prompt the user to leave the TV to experience the automatic brightness adjustment function. The second guide image 122 can be a dynamic image, which is used to present the changes in screen brightness when the user is in front of the TV and when the user leaves the TV, so as to form a sharp contrast in screen brightness, so that the user can know that the electronic device automatically reduces the screen brightness to reduce power consumption when it detects that no one is watching in front of the screen, and restores the brightness when the user returns to the electronic device.

[0120] When the display 260 displays the experience page 120, the processor 250 controls the display to switch back to the fifth guide page 111 in response to the user's input of a return operation, and moves the selector to the position of the third jump control 114. Thus, in response to the user's operation of triggering the third jump control 114, the processor 250 records that the automatic brightness adjustment function is in the on state and controls the display 260 to jump to the next guide page. In response to the user's operation of triggering the fourth jump control 115, the processor 250 records that the automatic brightness adjustment function is in the off state and controls the display 260 to jump to the next guide page.

[0121] Figure 13 is a schematic diagram of the sixth guide page displayed when the electronic device provided by some embodiments of the present application is connected to the camera for the first time. In response to the user triggering the operation of the third jump control 114 or the fourth jump control 115, the processor 250 can control the display 260 to display the sixth guide page 131 as shown in Figure 13. The sixth guide page 131 includes a ninth prompt message 132, a fourth immediate experience control 133, a setting control 134, and a fifth jump control 135. Among them, the ninth prompt message 132 is used to prompt the user to use gestures to directly open any installed application and quickly enable functions, and the gesture does not work when the application is using the camera. At this time, the selector defaults to the fourth immediate experience control 133.

[0122] When the display 260 displays the sixth guide page 131 , the processor 250 controls the display to switch to the fifth guide page 111 in response to a return operation input by the user.

[0123] Figure 14 is a schematic diagram of an experience page for the gesture control function provided in some embodiments of the present application. In response to a user triggering the fourth immediate experience control 133, the processor 250 can control the display 260 to display an experience page 141 for the gesture control function, as shown in Figure 14. This experience page 141 includes at least one standard gesture image 142 and a tenth prompt message 143. The tenth prompt message 143 prompts the user to perform the gesture indicated by the standard gesture image 142, keep their hand facing the camera, and ensure that the environment is not too dark. The standard gesture image 142 shows the user a gesture, such as an "OK" gesture or a "clenched fist" gesture. In this way, the processor 250 can obtain user gesture images captured by the camera and use a gesture recognition algorithm or model to identify gesture categories. Recognition is not sequential. When the user gesture matches any standard gesture, the dynamic gesture recognition process can be displayed.

[0124] Figure 15 is a schematic diagram of the dynamic gesture recognition process provided by some embodiments of the present application. As shown in view (a) of Figure 15 , taking the dynamic recognition of the "fist" gesture as an example, a progress ring 142a can be dynamically loaded around the standard gesture image 142 corresponding to the "fist" gesture, and recognition status information 142b can be displayed at an associated position of progress ring 142a (e.g., below the progress ring). Recognition status information 142b includes, but is not limited to, "Recognizing" or "Recognizing in Progress," "Recognition Successful," "Recognition Failed," etc. The dynamic gesture recognition process will last for a certain period of time (e.g., 2 seconds).

[0125] As shown in view (b) of FIG15 , if recognition is successful, the progress bar 142a is fully drawn, and the processor 250 can also control the display 260 to display a recognition success indicator on the standard gesture image 142 corresponding to the "fist" gesture. The recognition success indicator can be expressed in various forms, such as a "√" or a text prompt. A gesture that has already been successfully recognized will not be recognized again.

[0126] When the processor 250 recognizes a user gesture, it can match the user gesture with the standard gesture and calculate the degree of match between the two. If the degree of match is lower than a preset threshold, as shown in view (c) of Figure 15, the progress ring 142a appears to be not fully drawn, and a prompt "Recognition failed, please try again" is displayed at the associated position of the progress ring 142a (for example, below the progress ring). The processor 250 can control the display 260 to restore to the experience page 141 of the gesture control function as shown in Figure 14 after a preset time interval (for example, 2 seconds), thereby starting a new round of gesture recognition.

[0127] When the display 260 displays any user interface as shown in FIG. 14 or FIG. 15 , the processor 250 controls the display 260 to switch back to the sixth guide page 131 and moves the selector to the position of the setting control 134 in response to a return operation input by the user.

[0128] Figure 16 is a schematic diagram of a gesture settings page provided in some embodiments of the present application. In response to a user triggering operation of settings control 134, processor 250 controls display 260 to display gesture settings page 160 as shown in Figure 16. Gesture settings page 160 provides at least one gesture category, and the user can customize the control function mapped to each gesture category.

[0129] In some embodiments, referring to FIG16 , the electronic device may preset the control function mapped to each gesture category (hereinafter referred to as: gesture-function mapping relationship 161 ), and set a switch control 162 corresponding to each gesture-function mapping relationship 161 in the gesture setting page 160 , so that the user can turn on or off the control functions mapped to certain gesture categories by adjusting the switch control 162 . A gesture category can be mapped to only one function, or multiple gesture categories can be mapped to one function at the same time. A static gesture category is not allowed to map multiple functions, otherwise it will cause the electronic device to be unsure which function to execute when it recognizes the gesture category.

[0130] In some exemplary embodiments, referring to Figure 16, it is assumed that the gesture setting page 160 includes three gesture-function mapping relationships 161: gesture A (for example, the "scissors" gesture) maps the mute / unmute function, gesture B (for example, the "fist" gesture) maps the pause play / play function, gesture C (for example, the sliding "OK" gesture) maps the volume adjustment function, etc. Initially, the switch controls 162 corresponding to the three gesture-function mapping relationships 161 can be defaulted to be off (off state).

[0131] Figure 17 is a schematic diagram of the changes in the gesture setting page provided in some embodiments of the present application. Taking gesture A as an example, in response to the user's operation of turning on the switch control 162 corresponding to gesture A, the processor 250 can control the display to refresh the gesture setting page, so that the gesture setting page presents an effect shown in Figure 17. Referring to Figure 17, the changed gesture setting page 160 may include two display areas: display area one 160-1 is used to display the gesture-function mapping relationship 161 and its corresponding switch control 162, and display area two 160-2 displays a function list 163, and the function list 163 includes at least one function type that can be defined by the current gesture A for the user to select. The positions of display area one and display area two are not limited. For example, in Figure 17, display area one is on the left side of the gesture setting page, and display area two is on the right side of the gesture setting page.

[0132] In some embodiments, the function list 163 may include, but is not limited to: mute / unmute, TV shutdown, play / pause, open / close file, launch application A, launch application B, etc. Among them, the selector in the display area 2 160-2 is at the "mute / unmute" option position by default, that is, gesture A is mapped to the mute / unmute function by default. If the user changes the function type mapped by gesture A, for example, if the user moves the selector to the "TV shutdown" option position, the processor 250 controls the display 260 to close the function list 163, and changes the function type mapped by gesture A in the first gesture-function mapping relationship 161 in the gesture setting page 160 from "mute / unmute" to "TV shutdown", and the switch control 162 corresponding to gesture A remains in the open state.

[0133] As shown in Figure 16, the gesture setting page 160 may also include a save setting control 164. In response to the user clicking the save setting control 164, the processor 250 stores the setting status of the switch control 162 corresponding to each current gesture-function mapping relationship 161, and enables the enabled gesture control function. For example, the user sets the switch control 162 corresponding to gestures A and C to on (on state), and the switch control 162 corresponding to gesture B remains off. After the user clicks the save setting control 164, the processor 250 records the switch status of each gesture-function mapping relationship, and enables the mute / unmute function of gesture A to be effective, the media volume adjustment function of gesture C to be effective, the media play / pause function of gesture B to be ineffective, and the control display 260 to jump to the next guide page.

[0134] When the display 260 displays the gesture setting page 160, the processor 250 controls the display to switch back to the sixth guide page 131 in response to the return operation input by the user, and keeps the selector at the position of the setting control 134, without saving or taking effect the settings of the gesture function.

[0135] In response to the user clicking the fifth jump control 135 , the processor 250 records that the gesture control function is in the off state, and controls the display 260 to jump to other guide pages.

[0136] Figure 18 is a schematic diagram of the seventh navigation page displayed when an electronic device, provided by some embodiments of the present application, is first connected to a camera. In response to a user clicking the Save Settings control 164 on the Gesture Settings page 160, the processor 250 controls the display 260 to display the seventh navigation page 180 shown in Figure 18 . The seventh navigation page 180 includes an eleventh prompt message 181, at least one application option switch control 182, and an End control 183. The eleventh prompt message 181 indicates that the app on the page supports gesture control and that supported gestures can be defined within the app.

[0137] The application option switch control 182 consists of two parts: one for the application type and the other for the switch control corresponding to the application option. Initially, all application option switch controls 182 are off by default. The application type can be the application name of a single app (e.g., YouTube, Browser, etc.), and / or the app's tag category (e.g., Video App, Social App, etc.), and / or all installed apps on the device. Users can adjust the state of the switch control corresponding to each application type based on their preferences for gesture control functions and applications, thereby determining which app or apps have gesture control permissions.

[0138] In some embodiments, referring to FIG18 , application types include, but are not limited to, all applications, video applications, and browsers. Initially, the selector is located at the "all applications" position by default. In response to the user triggering the end control 183, the processor 250 records the current setting status of each application option switch control 182 and makes the setting content effective. For example, if the user adjusts the switch control corresponding to the video application from off to on and then triggers the end control 183, the processor 250 will turn on the gesture control function of all video applications installed on the machine.

[0139] The processor 250 can start the guided configuration process of the camera functions including automatic audio adjustment, automatic brightness adjustment, gesture control, etc. when it detects that the camera is connected to the electronic device through the external device interface 240. In some embodiments, the guided configuration process of the camera function can also be executed through system settings.

[0140] FIG19 is a schematic diagram of a first camera function setting page provided in some embodiments of the present application. In response to a user triggering a camera function setting option on the system settings page, the processor 250 controls the display 260 to display the camera function setting page 190 shown in FIG19 . The camera function setting page 190 includes a first area 191 and a second area 192 . The first area 191 is used to display at least one function setting option 191a related to camera configuration. Function setting options 191a may include, but are not limited to, "Custom Gestures," "In-App Gestures," "Auto-Audio Adjustment," and "Auto-Brightness Adjustment." The second area 192 is used to display the next-level settings menu corresponding to the function setting option 191a where the selector is located.

[0141] Figure 20 is a schematic diagram of a second camera function settings page provided in some embodiments of the present application. In response to a user selecting the "Customize Gesture" option control on camera function settings page 190, processor 250 controls display 260 to display a first settings menu 201 in second area 192 of camera function settings page 190. First settings menu 201 includes a fifth immediate experience control 201a, a select all control 201b, and at least one gesture settings control 201c.

[0142] In response to the user triggering the fifth immediate experience control 201a, the processor 250 can control the display 260 to jump to the gesture experience page shown in FIG14 and the related page of the gesture dynamic recognition process shown in FIG15, which will not be repeated here.

[0143] In some embodiments, the select all control 201b can be configured as a switch control, so that the processor 250 can turn on or off all gesture mapping control functions with one click in response to the user's operation of setting the switch state of the select all control 201b, thereby improving user operation efficiency.

[0144] The gesture settings control 201c can include two parts: one is the description of the gesture category and the mapped function. For example, if gesture A is mapped to mute / unmute, a standard gesture icon can be used to present the specific gesture category to the user. The other is the first function switch control corresponding to each gesture category, so that the user can turn on or off any gesture function. When all gesture settings controls 201c are set to off, the select all control 201b is also set to off. Conversely, when all gesture settings controls 201c are on, the select all control 201b is also set to on.

[0145] In some embodiments, when the selector is moved to the position of any gesture setting control 201c, if the gesture setting control 201c is in the off state, the display 260 does not display the next level setting menu corresponding to the gesture setting control 201c.

[0146] Figure 21 is a schematic diagram of a third camera function setting page provided in some embodiments of the present application. When the selector is moved to any gesture setting control 201c, if the gesture setting control 201c is in the on state, the processor 250 can control the display 260 to display the camera function setting page shown in Figure 21. The first area of ​​the camera function setting page displays the first setting menu 201, and the second area of ​​the camera function setting page displays the function list shown in Figure 17 for the user to set the function of any gesture category mapping. The details will not be repeated here.

[0147] Figure 22 is a schematic diagram of a fourth camera function settings page provided in some embodiments of the present application. In response to a user selecting the "In-App Gestures" option control on camera function settings page 190, processor 250 controls display 260 to display a second settings menu 221 in second area 192 of camera function settings page 190. Second settings menu 221 can be configured with reference to the seventh navigation page 180 illustrated in Figure 18 , including at least one application option switch control, which will not be further described here.

[0148] Figure 23 is a schematic diagram of a fourth camera function settings page provided in some embodiments of the present application. In response to a user selecting the "Auto Audio Adjustment" option control on camera function settings page 190, processor 250 controls display 260 to display a third settings menu 231, shown in the example of Figure 23, in the second area of ​​camera function settings page 190. Third settings menu 231 includes a sixth immediate experience control 231a and a second function switch control 231b.

[0149] In response to the user triggering the sixth immediate experience control 231a, the processor 250 controls the display 260 to jump to the experience page 101 of the audio automatic adjustment function shown in Figure 10. While displaying this experience page, the processor 250 controls the display 260 to display the camera function setting page 190 shown in Figure 23 in response to a return operation input by the user.

[0150] In response to the user turning on the second function switch control 231b, the processor 250 records that the audio automatic adjustment function is in the on state and enables the audio automatic adjustment function. In response to the user turning off the second function switch control 231b, the processor 250 records that the audio automatic adjustment function is in the off state, that is, the audio automatic adjustment function is not enabled.

[0151] Figure 24 is a schematic diagram of a sixth camera function settings page provided in some embodiments of the present application. In response to a user selecting the "Auto Brightness Adjustment" option control on camera function settings page 190, processor 250 controls display 260 to display a fourth settings menu 241, shown in Figure 24, in second area 192 of camera function settings page 190. Fourth settings menu 241 includes a seventh immediate experience control 241a, a time settings control 241b, and a third function switch control 241c.

[0152] In response to the user triggering the seventh immediate experience control 241a, the processor 250 controls the display 260 to jump to the experience page 120 of the automatic brightness adjustment function shown in Figure 12. While displaying this experience page, the processor 250 controls the display 260 to display the camera function setting page 190 shown in Figure 24 in response to a return operation input by the user.

[0153] In some embodiments, referring to Figure 24, the time setting control 241b also includes time information, which indicates the currently set time threshold, so that the processor 250 starts the timer when it detects that the user has left the TV, and when the time recorded by the timer reaches the time threshold, controls the display to reduce the screen brightness to save energy.

[0154] Figure 25 is a schematic diagram of a seventh camera function settings page provided in some embodiments of the present application. In response to a user triggering the time settings control 241b in the fourth settings menu 241, the processor 250 controls the display 260 to display the camera function settings page shown in Figure 25. The first area of ​​the camera function settings page displays the fourth settings menu 241 shown in Figure 24, and the second area displays the fifth settings menu 251, which is the next level settings menu corresponding to the time settings control 241b.

[0155] The fifth settings menu 251 includes a time list, which includes at least one optional time control 251a, such as "3 minutes," "5 minutes," and "10 minutes." For example, if the current time information is 5 minutes, the selector defaults to the time control corresponding to "5 minutes." If the user moves the selector to the time control corresponding to "10 minutes," the display 260 needs to refresh the fourth settings menu 241 to change the time threshold indicated by the time setting control 241b from 5 minutes to 10 minutes. When the display 260 displays the camera function settings page shown in FIG. 25 , the processor 250 controls the display to display the camera function settings page shown in FIG. 24 in response to a return operation input by the user.

[0156] In response to the user turning on the third function switch control 241c, the processor 250 records that the automatic brightness adjustment function is in the on state and enables the automatic brightness adjustment function. In response to the user turning off the third function switch control 241c, the processor 250 records that the automatic brightness adjustment function is in the off state, that is, the automatic brightness adjustment function is not enabled.

[0157] FIG26 is a schematic diagram of the processing logic of the automatic brightness adjustment function provided by some embodiments of the present application. In conjunction with FIG26 , the processing logic of the automatic brightness adjustment function includes: logic 1 and logic 3 executed by the processor 250, and logic 2 executed by the display 260.

[0158] Logic 1 of the automatic brightness adjustment function includes: the processor 250 performs human body recognition based on the scene image captured by the camera, starts the timer when it detects that the scene image does not contain a human body, and determines whether the timing time is greater than the time threshold; if the timing time is not greater than the time threshold, maintains the current display state of the display and continues to recognize the human body; if the timing time is greater than (including greater than or equal to) the time threshold, sends a first control instruction to the display 260, so that the display 260 executes logic 2 after responding to the first control instruction.

[0159] The second logic of the automatic brightness adjustment function includes: display 260 displays a first prompt pop-up window 261. Referring to Figure 26, first prompt pop-up window 261 includes brightness adjustment prompt information 261a, a first cancel button 261b, and a disable button 261c. The selector defaults to the position of first cancel button 261b. The brightness adjustment prompt information 261a indicates that the electronic device is currently unattended and the screen brightness will be reduced to save power.

[0160] The third logic of the automatic brightness adjustment function includes: in response to the user clicking the first cancel button 261b within a preset time, the processor 250 controls the display 260 to close the first prompt pop-up window 261 without changing the screen brightness. The preset time is not limited and can be set to 15 seconds, for example.

[0161] The third logic for the automatic brightness adjustment function also includes: in response to the user clicking the disable button 261c within a preset time, the processor 250 controls the display 260 to close the first prompt pop-up window 261 and suspend the automatic brightness adjustment function for the preset disable time. For example, if the preset disable time is one hour, the processor 250 will not execute the human presence detection and other logic within one hour after the user clicks the disable button 261c.

[0162] The logic three of the automatic brightness adjustment function also includes: when the processor 250 detects that the user has not clicked any button in the first prompt pop-up window 261 within a preset time, that is, a response timeout occurs, the display 260 is controlled to close the first prompt pop-up window 261 and reduce the screen brightness.

[0163] In some embodiments, the electronic device can preset a normal brightness mode and a low brightness mode, where the normal brightness mode corresponds to a first brightness and the low brightness mode corresponds to a second brightness, where the second brightness is less than the first brightness, and the second brightness can be a static value or a dynamically adjustable value (e.g., gradually dimming). In this way, when the processor 250 detects that the user has not clicked any button in the first prompt pop-up window 261 within a preset time, it switches from the normal brightness mode to the low brightness mode, thereby reducing the screen brightness and achieving the purpose of reducing the power consumption of the electronic device.

[0164] FIG27 is a schematic diagram of the processing logic of the gesture control function provided by some embodiments of the present application. In conjunction with FIG27 , the processing logic of the gesture control function includes: logic four and logic six executed by the processor 250, and logic five executed by the display 260.

[0165] Logic 4 of the gesture control function includes: processor 250 obtains the scene image captured by the camera and performs gesture recognition. In a specific implementation, processor 250 can calculate the gesture matching degree to determine whether the scene image contains a valid gesture and the specific gesture category (hereinafter referred to as the "target gesture category"). If processor 250 obtains the target gesture category, it can issue a second control instruction to display 260 based on the target gesture category, so that display 260 executes logic 5 after responding to the second control instruction. If the gesture is not recognized, processor 250 does not send the second control instruction.

[0166] Logic 5 of the gesture control function includes: display 260 displays a first layer 271 on the current user interface. Referring to FIG. 27 , first layer 271 can be set at a predetermined position on the user interface (e.g., at the top center of the interface). First layer 271 includes a gesture animation area 272 and a function description control 273. The relative positions of gesture animation area 272 and function description control 273 within first layer 271 are not limited. For example, FIG. 27 provides an example in which function description control 273 is set below gesture animation area 272.

[0167] In some embodiments, the gesture animation area 272 may include two components: a user gesture image 272a extracted from the scene image captured by the camera, and a recognition progress ring 272b positioned around the user gesture image 272a. The recognition progress ring 272b dynamically indicates the progress of gesture recognition. Dynamic gesture recognition typically lasts for a preset duration (e.g., 2 seconds), requiring the user's gesture to also last for that duration.

[0168] The function description control 273 is used to describe the function triggered by the user gesture. The processor 250 queries the function mapped to the target gesture category and generates function description information based on the current working status of the electronic device. The function description information can be uniformly described using the "progressive tense", such as "muting...", "turning off the TV...", etc., and then controls the display 260 to add the function description information to the function description control 273.

[0169] In one example, when the processor 250 recognizes that the user makes gesture D, it queries the function mapped by gesture D. Assuming that the function mapped by gesture D is to start APP1, the processor 250 further queries that the electronic device is not currently running APP1, thereby generating function description information such as "Opening APP1..." and causing the function description control 273 to display the function description information.

[0170] The sixth logic of the gesture control function includes determining whether the gesture function is successfully recognized within a preset time period. If the user releases or quickly switches the gesture within the preset time period, resulting in a gesture function recognition failure, the first layer 271 is de-displayed and the function mapped to the target gesture category is not executed. If the gesture function is successfully recognized, the first layer 271 is de-displayed and the function mapped to the target gesture category is immediately executed.

[0171] Figure 28 is a schematic diagram of displaying a second prompt pop-up window when no camera is detected, provided in some embodiments of the present application. When a user is experiencing camera-related functions based on any of the pages shown in Figures 5 to 25, or when at least one of the automatic audio adjustment function, the automatic brightness adjustment function, and the gesture control function is enabled, or when gesture control is enabled for certain applications, if processor 250 detects that no camera is available, it can control display 260 to display second prompt pop-up window 281 shown in Figure 28.

[0172] Referring to Figure 28 , the second prompt pop-up window 281 includes a twelfth prompt message 281a, a retry button 281b, and a second cancel button 281c. The selector is currently positioned at the retry button 281b. Twelfth prompt message 281a primarily indicates that no camera is currently available, and may also prompt the user to connect a camera to use related functions.

[0173] In response to the user triggering the second cancel button 281 c , the processor 250 controls the display 260 to close the second prompt pop-up window 281 .

[0174] In response to the user triggering the retry button 281b, the processor 250 can control the display 260 to close the second prompt pop-up window 281 and re-detect whether the electronic device has an available camera. If the electronic device has an available camera, the user interface can indicate that the camera connection is successful and perform camera-related functions normally. If the electronic device still does not have an available camera, the display 260 can be controlled to display the second prompt pop-up window 281 again.

[0175] In some embodiments, the aforementioned gesture control function may specifically include: static gesture control and motion gesture control.

[0176] Static gesture control refers to controlling electronic device functions using static gestures, and is generally applicable to controlling switch functions such as mute / unmute, play / pause, etc. In one example: the electronic device is currently in sound playback mode (not muted), and the user makes gesture A towards the camera and maintains this gesture for a preset duration (e.g., 2 seconds). The processor 250 recognizes the gesture and executes the mute function.

[0177] Motion gesture control refers to the use of moving gestures to control the functions of electronic devices. It is generally applicable to scenarios where the size of function parameters needs to be dynamically adjusted. In one example, moving gesture C left and right to adjust the volume, the amount of volume adjustment is generally related to the direction, distance, and speed of the movement of gesture C. Assuming that the user maintains gesture C unchanged and moves it to the right, the processor 250 recognizes the motion gesture and dynamically increases the audio volume according to a preset rule. During the research process, the applicant found that the gesture control function has the following problems:

[0178] Problem 1: Static gesture control may cause miscontrol and inaccurate gesture control.

[0179] When the processor 250 dynamically recognizes a gesture function, referring to FIG. 27 , it controls the display 260 to display a first layer 271 on the user interface. Upon successful recognition, it cancels first layer 271 and generates a control instruction corresponding to the target gesture category. This process generally lasts for a preset duration. For example, if the processor 250 recognizes gesture A and checks to see if the electronic device is in a sound playback state, it sends control instruction A to the sound control module, which responds to control instruction A and turns on the mute mode. If the user still maintains gesture A, the processor 250 repeats the gesture function recognition process and sends control instruction B to the sound control module, which exits the mute mode in response to control instruction B. Thus, when the preset duration is reached, because the user does not immediately release gesture A, the processor 250 issues at least two control instructions (including control instruction A and control instruction B), which does not conform to the user's intention to mute the electronic device through gesture A, resulting in erroneous control.

[0180] Problem 2: Due to jitter in the user's gestures, motion gesture control is inaccurate.

[0181] Figure 29 is a schematic diagram of gesture jitter provided in some embodiments of the present application. As shown in Figure 29, after the user makes a gesture, due to unstable hands or slight shaking, the gesture will cause a small displacement. Especially for motion gesture control, this displacement caused by gesture jitter may be regarded as motion, which is converted into an incorrect adjustment of functional parameters (such as volume, clarity, etc.), resulting in inaccurate motion gesture control.

[0182] Problem 3: Due to the motion constraints of the user’s hands and limbs, continuous and stable motion gesture control cannot be achieved.

[0183] Taking gesture C moving left and right to adjust the volume as an example, suppose the user makes gesture C with their right hand and pushes their right arm to the right, so that gesture C moves to the right side of the body, thereby increasing the volume. Once the user moves gesture C to the extreme position corresponding to the length of their right arm, they cannot move gesture C further to the right. At this time, if the user pulls their right arm back to the left and moves gesture C in the opposite direction (i.e., to the left), the volume will be reversed, resulting in the user being unable to continuously increase the volume through gesture C. If the user changes the gesture (for example, switching to gesture D) when gesture C reaches the extreme position corresponding to the length of their right arm, the processor 250 will stop controlling the volume based on the movement of gesture C and will recognize and control the function of gesture D. This shows that due to the user's physical constraints, the processor 250 cannot continuously and stably dynamically adjust the function parameters based on the same gesture, and the function parameters may fluctuate unstably (for example, the volume increases and then decreases).

[0184] First, in order to solve the problems of possible miscontrol and inaccurate gesture control in the static gesture control mentioned in the first problem, some embodiments of the present application provide a static gesture control implementation method, which is described in detail below.

[0185] Figure 30 is a schematic diagram of a system architecture for an electronic device provided in some embodiments of the present application. As shown in Figure 30, the system architecture may include, but is not limited to, an application layer, an SDK layer, a Framework layer, and a system. The system includes, but is not limited to, a processor, a GPU, and an AI processing chip. The processor is used to control the application layer, SDK layer, Framework layer, and other software and hardware functions in the system.

[0186] The application layer can include a first acquisition module, an interface call module, a second acquisition module, an instruction generation module, and an instruction execution module. The SDK layer can configure the algorithms required for gesture control, including but not limited to image processing algorithms, human body recognition algorithms, gesture recognition algorithms, and gesture tracking algorithms. Electronic devices can train gesture recognition models and encapsulate these models in the SDK layer.

[0187] The first acquisition module is used to acquire a scene image captured by an image collector (e.g., a camera). The interface call module is used to call the SDK layer interface to pass the scene image to the SDK layer. The SDK layer can then use an algorithm or model to perform gesture recognition on the scene image, obtain gesture recognition results, and then send these gesture recognition results to the second acquisition module.

[0188] In some embodiments, an example of the gesture recognition result received by the second acquisition module is:

[0189] The results of gesture recognition include but are not limited to: the number of gestures and gesture sets. Among them, the gesture set includes gesture categories, gesture similarity corresponding to each gesture category, and gesture position. The number of gestures is used to characterize the number of gestures contained in each frame of the scene image recognized by the SDK layer. The gesture category is used to characterize which gesture is recognized. The SDK layer can preset identifiers to indicate gesture categories, for example, identifier 1 indicates gesture A, identifier 2 indicates gesture B, etc. Gesture similarity is used to characterize the similarity between the target gesture recognized in the scene image and the standard gesture. Gesture position is used to indicate the position of the target gesture. The gesture position can be expressed in coordinates. The gesture position of a static gesture basically does not change (small displacements can be ignored), and the gesture position of a moving gesture changes dynamically.

[0190] The second acquisition module is configured to, after receiving the gesture recognition result, send the gesture recognition result to the instruction generation module to initiate the gesture function control process. The gesture function control process may include: the instruction generation module controls the display 260 to display the first layer 271 shown in FIG. 27 on the current user interface, wherein the gesture animation area 272 is used to display the user gesture image and recognition progress ring corresponding to the target gesture, and the function description control 273 is used to display the function description information matching the target gesture. After the first layer 271 is displayed for a preset period of time, the display is controlled to cancel the first layer 271. Based on the target gesture category and the current operating state of the electronic device, the instruction generation module generates and issues a first control instruction to the instruction execution module, so that the instruction execution module responds to the first control instruction and executes the corresponding target gesture function. The instruction execution module may include at least one functional module, such as a mute module, a volume adjustment module, a playback control module, or the like. It should be noted that the hardware and software configuration of the system architecture is not limited to the examples in the embodiments of this application.

[0191] On this basis, one concept of the present application is to detect whether the gesture category and gesture position have changed after the first layer 271 is displayed for a preset period of time. If neither the gesture category nor the gesture position has changed, the gesture function control process will not be activated. If the gesture category has changed and / or the gesture position has changed, and the two adjacent gestures are not considered to be the same static gesture, the gesture function control process can be activated based on the currently newly recognized gesture category. This can avoid the problem of multiple control instructions being sent due to the user's failure to release the target gesture in time, thereby eliminating the erroneous control of static gestures and improving the accuracy and stability of static gesture control.

[0192] FIG31 is a flow chart of a first gesture control method provided in some embodiments of the present application. As shown in FIG31 , the method may be executed by the processor 250 and includes:

[0193] Step S311: Acquire a gesture recognition result based on the scene image acquired by the image collector.

[0194] The gesture recognition result can include: the number of gestures and the gesture set. If the number of gestures is equal to 0, the gesture set is an empty set. If the number of gestures is greater than 0, the gesture set is a non-empty set.

[0195] Step S312: judging whether the target gesture exists in the scene image according to the gesture recognition result.

[0196] In some embodiments, if the gesture set is an empty set, indicating that the gesture recognition results do not include any gestures, there is naturally no category corresponding to the target gesture (hereinafter referred to as the "target gesture category"). When the gesture set is a non-empty set, the target gesture category is searched from the gesture set, and the gesture similarity mapped to the target gesture category is greater than a first threshold. The higher the gesture similarity, the greater the reliability of gesture recognition. Therefore, the first threshold can be reasonably set according to the usage requirements, for example, the first threshold can be set to 0.8.

[0197] In one example, assuming the first threshold is 0.8, the gesture set includes gesture category 1 and gesture category 2, the gesture similarity mapped to gesture category 1 is 0.9, and the gesture similarity mapped to gesture category 2 is 0.5, then gesture category 1 is determined to be the target gesture category. If the gesture recognition result does not include the target gesture category, that is, the target gesture does not exist in the scene image, step S313 is executed; if the gesture recognition result includes the target gesture category, that is, the target gesture exists in the scene image, step S314 is executed.

[0198] Step S313: discard the gesture recognition result.

[0199] When the target gesture category does not exist in the gesture recognition result, that is, the SDK layer returns an invalid gesture, it indicates that the user may have put down the hand performing the gesture, or the user has changed the gesture. If the gesture function control process has been started, that is, the display is displaying the first layer 271, or the processor is parsing or responding to the control instruction corresponding to the gesture, the gesture function control process needs to be stopped immediately to ensure the accuracy of gesture control.

[0200] In some embodiments, the processor may maintain a preset flag, which may be set to a first state value (e.g., True or 1) and a second state value (e.g., False or 0), wherein the first state value is used to indicate that the gesture function control process is currently activated, and the second state value is used to indicate that the gesture function control process is currently not activated. Before executing step S313, or while executing step S313, the processor may access the preset flag. If the preset flag is set to the first state value, the gesture function control process is stopped and the second state value is set for the preset flag. If the preset flag is set to the second state value, indicating that the gesture function control process is not currently executed, the gesture recognition result is discarded.

[0201] In some embodiments, after discarding the gesture recognition result, the process may return to step S311 .

[0202] Step S314: Obtain the target gesture function mapped by the target gesture.

[0203] In some embodiments, based on the mapping relationship between each gesture category and function saved in the aforementioned guidance or setting process, the processor can query the function mapped to the target gesture category (referred to as: target gesture function in this application).

[0204] Step S315 , determining whether the category of the target gesture is the same as that of the previous gesture.

[0205] The previous gesture may be a gesture recognized in the previous frame of the scene image. If the current target gesture is of a different category than the previous gesture, i.e., the category of the static gesture has changed, then step S316 is executed. If the target gesture is of the same category as the previous gesture, then step S317 is executed.

[0206] Step S316 , starting the gesture function control process corresponding to the target gesture to implement the target gesture function.

[0207] In some embodiments, before executing step S316, the processor may access a preset flag. If the preset flag is set to the first state value, the gesture function control process corresponding to the previous gesture is stopped and the preset flag is set to the second state value. Thus, when the preset state value is changed to the second state value, the processor may execute step S316, start the gesture function control process corresponding to the target gesture, and set the preset flag to the first state value.

[0208] In some embodiments, before executing step S316, if it is determined that the preset flag is set to the second state value, the gesture function control process corresponding to the current target gesture is started normally, and the preset flag is set to the first state value. In this way, when the categories of two consecutive gestures change, the corresponding function can be quickly executed according to the current latest gesture category, improving the accuracy and responsiveness of static gesture control.

[0209] Step S317: determine whether there is an activated gesture function control process.

[0210] In some embodiments, when executing step S317, the processor may access a preset flag. If the preset flag is set to the second state value, indicating that there is no currently activated gesture function control process (i.e., the gesture function control process corresponding to the previous gesture), step S316 is executed. If the preset flag is set to the first state value, step S318 is executed.

[0211] Step S318: Calculate the distance between the target gesture and the previous gesture.

[0212] In some embodiments, assuming that the position coordinates of the previous gesture are Rect(x1, y1) and the position coordinates of the target gesture are Rect(x2, y2), the distance d between two adjacent gestures can be calculated using the following formula:

[0213] In some embodiments, to simplify calculations, the distance value d can also be calculated as follows: d = |x2 - x1|, or d = |y2 - y1|. By calculating the distance value d, it is possible to determine whether the positions of two consecutive identical gestures have changed, and the specific amount of position change.

[0214] Step S319: determine whether the distance value is greater than a second threshold.

[0215] When two consecutive gestures have the same category, the following three situations may occur:

[0216] Case 1: If the distance d is equal to or close to 0, it can be assumed that there is no displacement between two consecutive identical gestures, indicating that the user maintains the same gesture type and position. To avoid incorrect control caused by repeated gesture control commands, the processor can maintain the gesture function control process corresponding to the previous gesture. This way, the processor will not activate the target gesture function corresponding to the target gesture.

[0217] Case 2: 0<distance value d≤second threshold, where the second threshold can be an empirical value set based on the size of the user's arms, hands and other limbs, indicating that actual displacement occurs between two adjacent identical gestures. Considering that the user may be performing a motion gesture, the gesture function control process corresponding to the motion gesture has a certain degree of continuity, and it is necessary to dynamically adjust the corresponding function parameters according to the position change of the gesture. For example, the user moves gesture C left and right to reduce or increase the volume. Therefore, the processor can record the category and position of the target gesture, and continue to execute the started gesture function control process to achieve dynamic adjustment of the function parameters.

[0218] Case three: The distance value d is greater than the second threshold, indicating that actual displacement occurs between two adjacent identical gestures, but the displacement of the motion gesture is out of the limit of the user's gesture operable range, and abnormal problems such as gesture misrecognition may occur. In this case, it is necessary to interrupt the gesture function control process corresponding to the motion gesture, that is, the processor needs to stop the currently started gesture function control process. In some embodiments, a second state value can also be set for the preset flag.

[0219] Based on the above three situations, if the distance value is not greater than (including less than or equal to) the second threshold, the processor executes step S3110; if the distance value is greater than the second threshold, the processor executes step S3111.

[0220] Step S3110 , recording the category and position of the target gesture, and continuing to execute the activated gesture function control process.

[0221] Step S3111, stop the currently started gesture function control process.

[0222] Secondly, in order to solve the problem of inaccurate motion gesture control due to jitter of the user's gesture involved in the aforementioned problem 2, some embodiments of the present application also provide a first motion gesture control implementation method. Motion gesture control is to calculate the displacement of two adjacent gestures of the same category in a specified direction (for example: X-axis direction, corresponding to the left and right directions), and convert the displacement into a change in the function parameter. Since there may be a small amplitude jitter when the user performs a gesture action, the jitter may also be converted into a change in the function parameter, resulting in unstable fluctuations in the function parameter, affecting the user experience.

[0223] In some embodiments, the processor can calculate the displacement of two consecutive gestures of the same type in the direction of movement, where the movement direction is the X-axis direction (reflected as left and right movement) or the Y-axis direction (reflected as up and down movement). The electronic device can preset the relationship between the displacement and the function parameters in the motion gesture control function.

[0224] In some embodiments, it can be set that when the displacement is greater than 0, the function parameter is increased, and when the displacement is less than 0, the function parameter is decreased. The larger the absolute value of the displacement is, the greater the change in the function parameter is.

[0225] In some embodiments, it is also possible to set the function parameter to increase when the displacement is less than 0 and to decrease when the displacement is greater than 0. The greater the absolute value of the displacement, the greater the change in the function parameter.

[0226] In some embodiments, the displacement is equal to 0, indicating that the gesture is stationary and the function parameters can remain unchanged.

[0227] In some embodiments, the displacement amount and the change amount of the functional parameter may present a linear relationship, and of course may also present other change rules required to meet the motion gesture control function.

[0228] In some embodiments, the displacement generated by hand shaking is relatively small, so the third threshold can be set based on the allowable shaking amplitude. During the movement of the same gesture along a specified direction, each position corresponds to a fixed-point gesture (hereinafter referred to as a "position gesture"). Thus, a motion gesture can be considered a gesture sequence consisting of multiple position gestures of the same category, i.e., a motion gesture can include at least one position gesture.

[0229] For the sake of convenience of description, the first position gesture and the second position gesture are used to express two adjacent position gestures in a gesture sequence, that is, the first position gesture and the second position gesture are of the same category, the second position gesture is generated after the first position gesture moves along the specified direction, and the processor calculates the absolute value of the displacement between the first position gesture and the second position gesture (hereinafter referred to as: the first displacement).

[0230] In some embodiments, if the first displacement is not greater than a third threshold, indicating that the second-position gesture is generated after the first-position gesture has jittered, the processor may determine that the second-position gesture is a jitter gesture and delete the second-position gesture from the gesture sequence. The processor may also discard the recognition result of the second-position gesture so that the second-position gesture does not participate in the adjustment calculation of the function parameters, thereby achieving gesture anti-jitter and improving the accuracy of gesture function control. In this way, the processor will not adjust the function parameters based on the first displacement between the second-position gesture and the first-position gesture, thereby eliminating the problem of inaccurate function parameter control caused by gesture jitter.

[0231] Afterwards, if the processor obtains a third position gesture, the third position gesture is added to the gesture sequence, and the adjacent position gesture before the third position gesture is the first position gesture, not the second position gesture. In this way, the processor continues to execute the jitter logic and function parameter adjustment according to the displacement between the third position gesture and the first position gesture.

[0232] In some embodiments, if the first displacement is greater than the third threshold, the processor can determine that the second position gesture is not caused by human hand shaking, but is caused by the user moving the first position gesture normally. The processor then adjusts the function parameters according to the displacement between the first position gesture and the second position gesture (which may be positive or negative).

[0233] Figure 32 is a schematic diagram of the principle of motion gesture jitter reduction provided by some embodiments of the present application. Referring to Figure 32, the user currently makes a gesture B (gesture B is shown as a "scissor hand" in Figure 27). Due to hand jitter, gesture B moves from position 1 to position 1'. The processor calculates that the absolute value of the displacement of gesture B between position 1 and position 1' is less than a third threshold, indicating that gesture B has moved to position 1' after jitter. Therefore, the second position gesture corresponding to gesture B recognized at position 1' is ignored, that is, the second position gesture is deleted from the gesture sequence, and the first position gesture corresponding to gesture B recognized at position 1 is retained in the gesture sequence, and the first position gesture is marked as the baseline gesture.

[0234] Referring to Figure 32, the user continues to move gesture B to position 2. The gesture B recognized at position 2 corresponds to the third position gesture. At this time, the gesture sequence includes {first position gesture, third position gesture}. The processor calculates that the absolute value of the displacement of gesture B between position 1 and position 2 is greater than the third threshold, and determines that the third position gesture is a valid gesture (the third position gesture is not deleted in the gesture sequence). The processor adjusts the functional parameters according to the displacement between the third position gesture and the first position gesture (reference gesture).

[0235] Afterwards, the third position gesture is marked as a reference gesture, and the processor uses the displacement between the next position gesture and the third position gesture (reference gesture) to perform anti-shake logic and function parameter adjustment, which will not be repeated here.

[0236] In some embodiments, when executing motion gesture control, the processor may collect position gestures at a fixed frequency. A greater displacement between two adjacent position gestures indicates a faster gesture movement. To expedite the adjustment of function parameters, it is desirable that the greater the gesture movement speed, the faster the function parameter changes. To this end, a second motion gesture control implementation method is provided as an example in the following embodiment.

[0237] In some embodiments, assuming that the first displacement between two adjacent position gestures is Diff, since the processor can call the algorithm model at a fixed frequency to obtain gesture recognition results, the time length for the processor to obtain the gesture recognition results is fixed. Assuming that the time length is t, the movement speed of the gesture along the specified direction V = Diff / t.

[0238] In order to measure the speed of movement, a speed threshold can be preset. If the movement speed V of the gesture is not greater than the speed threshold, the processor can adjust the function parameter according to the first adjustment coefficient p1 between the displacement and the function parameter. The first adjustment coefficient p1 can be a preset fixed value.

[0239] If the movement speed of the gesture is greater than the speed threshold, the processor can adjust the function parameters according to the second adjustment coefficient p2, wherein the second adjustment coefficient p2 can be greater than the first adjustment coefficient p1. The second adjustment coefficient p2 can be a preset fixed value, or it can be a dynamically adjustable value related to the movement speed V of the gesture. For example, the greater the movement speed V, the greater the second adjustment coefficient p2 is set. Compared with conventional motion gesture control, the embodiment of the present application realizes the acceleration of the rise and fall of the function parameters according to the movement speed of the gesture, and realizes the rapid and continuous adjustment of the function parameters. In this way, when the gesture moves the same distance, the change of the function parameters in the present application is faster and more significant.

[0240] Taking gesture C moving left and right along the X-axis direction, and the processor linearly adjusting the volume change (Volume) as an example: in conventional motion gesture control, Volume1 = Diff(X)*p1, where Diff(X) represents the displacement generated when the gesture moves along the X-axis direction (which may be positive or negative); in the embodiment of the present application, when the movement speed of the gesture is greater than the speed threshold, Volume2 = Diff(X)*p2, which achieves an increase in the change of functional parameters under the same displacement amount, thereby improving the user experience.

[0241] To address the aforementioned issue of the third problem, which is the inability to achieve continuous and stable motion gesture control due to the motion constraints of the user's hands and limbs, some embodiments of the present application further provide a third motion gesture control implementation method, which is conceived as follows:

[0242] The user can preset the transition gesture associated with each motion gesture, so that when the current motion gesture (hereinafter referred to as: the first gesture) moves in the first direction, if the processor detects that the gesture category switches to the transition gesture associated with the first gesture, it records the current function parameter value (hereinafter referred to as: the first function parameter value), does not stop the gesture function control process corresponding to the first gesture, and does not adjust the function parameters of the first gesture control according to the displacement of the transition gesture, that is, at this time the function parameter remains unchanged at the first function parameter value.

[0243] If the processor detects that the gesture category switches from the transition gesture back to the first gesture, there is no need to repeat the process of displaying the first layer 271. The processor calculates the target change amount of the function parameter based on the displacement amount of the first gesture, and adjusts the function parameter according to the first function parameter value and the target change amount. The changed function parameter = the first function parameter value + the target change amount.

[0244] In this way, when the user moves the first gesture to the extreme position in the first direction, the user can pull his arm back without stopping the control logic of the current motion gesture and without reversing the functional parameters by switching to the transition gesture and moving the transition gesture in the second direction opposite to the first direction. In this way, after the user restores the first gesture, the first gesture has sufficient movable space when moving in the first direction. Through the coordination between the first gesture and its associated transition gesture, the motion gesture control function is not terminated before and after the transition, and the functional parameters are continuously adjusted from the first functional parameter value recorded before the transition period, thereby improving the continuity, stability and accuracy of the motion gesture control.

[0245] Figure 33 is a schematic diagram of the principle of continuous control of motion gestures provided by some embodiments of the present application. As shown in Figure 33, the first gesture (the example in Figure 33 is "scissor hands") moves to the right, causing the volume to gradually increase. When the first gesture moves from position a to position a', it reaches the extreme position of the right arm. At this time, the user switches the gesture to the transition gesture corresponding to the first gesture (the example in Figure 33 is "fist"), and the processor records the current volume as the first volume. The transition gesture moves in the opposite direction (to the left) from position b to position b'. During this transition, the logic of the first gesture controlling the volume does not stop and remains unchanged at the first volume. Afterwards, the user switches the gesture from the transition gesture back to the first gesture, so that the first gesture can continue to move to the right from position c, and the processor can continue to increase the volume from the first volume. And so on. The user can repeat this process multiple times until the volume is increased to a satisfactory effect. The embodiments of the present application free the motion gesture control function from the limitations of the user's limbs.

[0246] Through the above embodiments, for static gesture control, this application detects the validity of gesture actions, combines the changing state of gesture type and gesture position, and executes matching static gesture control logic, thereby avoiding the problem of erroneous control caused by repeatedly sending the same gesture control instruction, and improving the accuracy and interaction efficiency of static gesture control.

[0247] Through the above embodiments, for motion gesture control, this application can solve the problems of jitter and control deviation of the controlled function caused by gesture jitter, thereby improving the accuracy of motion gesture control. In addition, this application detects the speed of gesture movement and increases the adjustment coefficient to speed up the change rate of functional parameters, thereby achieving fast and accurate motion gesture control. Through the linkage between the first gesture and the transition gesture, the limitation of the user's arms and other limbs on the range of gesture movement can be eliminated, and the changes of functional parameters can be controlled continuously, stably and accurately through motion gestures.

[0248] In addition to the above embodiments, this application also provides some other embodiments for controlling a browser based on gesture recognition, which are described in detail as follows:

[0249] In some embodiments, various types of media can be played through the electronic device 200. When playing media such as video and audio, the electronic device 200 can present them through a browser. Multiple applications (Application, APP) can be deployed in the electronic device 200, and the application APP can also display media data through the browser. For example, the electronic device 200 can present an application interface for media playback through a browser. The media playback application interface can include multiple media options. The user can control the browser to play the corresponding media content by clicking on any media option.

[0250] When accessing a web page or viewing media through a browser in the electronic device 200, it is necessary to perform operations such as turning pages, moving up and down pages, or switching tabs. To achieve these functions, a cursor can be controlled by a remote control. For example, the electronic device can be provided with a matching remote control, and all browser controls can be performed by controlling the cursor movement through the remote control.

[0251] Figure 34 is a schematic diagram of an effect of an electronic device provided by some embodiments of the present application controlling a browser according to a remote control cursor. As shown in Figure 34, after entering content in the browser input box and clicking search, the cursor is located at the top of the web page, as shown by the dotted line. If you need to turn the page down or navigate the browser web page content downward, you need to press the "down" button on the remote control directional keys to move the cursor to the bottom of the web page. Referring to Figure 35, Figure 35 is another schematic diagram of an effect of an electronic device provided by some embodiments of the present application controlling a browser according to a remote control cursor. After moving the cursor multiple times, the cursor is moved to the bottom of the web page shown by the dotted line. In the process of moving the cursor, each time the electronic device receives a directional key signal from the remote control, it moves the cursor position by a preset distance. In this way, operations such as turning pages, moving pages up and down, or switching tabs can be performed by moving the cursor.

[0252] However, the above process of moving the cursor requires repeatedly pressing the down button to move to the bottom of the webpage. Similarly, when you need to turn the browser page up, you also need to frequently press the "up" button on the remote control to move the cursor to the top of the browser. When the user opens multiple tabs, when the user wants to switch to a different tab, they also need to move the cursor to the top and click the arrow keys to switch between tabs.

[0253] For example, Figure 36 illustrates how an electronic device, provided by some embodiments of the present application, switches tabs using a remote control cursor. As shown in Figure 36 , when multiple tabs are open, a user needs to move the cursor to the top and click the desired tab to switch to. During this process, a long press or multiple presses of a direction key are required to switch tabs. For example, a long press of the left or right button, or multiple presses of the left or right button, are required. To bookmark a webpage, the user also needs to repeatedly move the cursor to the bookmark button. It is understood that in either of these operations, frequent cursor movement requires frequent arrow key presses. Consequently, when using a remote control to scroll through pages or switch between web pages in a browser, frequent arrow key presses are required, consuming a significant portion of the user's time. Consequently, performing browser functions such as page turning, scrolling, or switching tabs is cumbersome and inefficient.

[0254] In order to solve the problem that the operation process is cumbersome and the operation efficiency is low when performing functions such as turning pages, moving pages up and down, or switching tabs through a browser, some embodiments of the present application provide an electronic device 200. The electronic device 200 includes a display 260, an image acquisition interface 290, and a processor 250. Among them, the display 260 is configured to display a user interface, and the image acquisition interface 290 is configured to connect to a camera 2901 to capture the user's gesture image. In order to facilitate the understanding of the technical implementation methods in some embodiments of the present application, each step is described in detail below in combination with some specific embodiments and drawings. Figure 37 is a schematic diagram of the process of an electronic device provided in some embodiments of the present application to perform a browser control based on gesture recognition. As shown in Figure 37, in some embodiments, when the electronic device performs a method for controlling a browser based on gesture recognition, it may include the following steps S1-S4, the specific contents are as follows:

[0255] Step S1: In response to an operation event of starting a browser, a gesture image captured by the camera 2901 is obtained.

[0256] In some embodiments, before obtaining the gesture image, the electronic device 200 can obtain the operation category of the browser controlled by the user input, and the operation category of the browser controlled can be divided into a gesture control category and a non-gesture control category. The gesture control category refers to controlling browser-related operations through gesture movements or gesture postures, such as turning pages, switching tabs, and collecting. The non-gesture control category refers to controlling the browser in a non-gesture manner. For example, the browser's page turning, switching tabs, and collecting operations can be controlled by the remote control cursor. If the operation category is a gesture control category, the electronic device 200 can start the camera 2901 and capture the gesture image through the camera 2901.

[0257] In some embodiments, the electronic device 200 can determine the browser's operation category by capturing control instructions received from the browser. For example, if the control instruction received by the browser is issued by the directional keys of a remote control, the browser's operation category can be determined to be a non-gesture control category. If the control instruction received by the browser is an instruction converted from an image containing gesture motion, the browser's operation category can be determined to be a gesture control category. The browser's operation category can also be determined by other methods, which are not specifically limited in this application.

[0258] For example, FIG38 is a flowchart of an electronic device determining an operation category for controlling a browser according to some embodiments of the present application. As shown in FIG38 , in some embodiments, the process specifically includes the following steps:

[0259] S3801. Obtain the browser control operation category input by the user;

[0260] S3802: Monitor operation events on an operation control for selecting an operation category;

[0261] S3803. Read the input value entered in the operation control;

[0262] S3804: Determine the operation category for controlling the browser according to the input value.

[0263] That is, when determining the browser control operation category, the electronic device 200 can first monitor the operation events on the operation control used to select the operation category. For example, before operating the browser, the electronic device 200 can be provided with an operation control for the user to select the browser control operation mode. The user can select the corresponding operation mode according to their needs. For example, the operation mode can include gesture control and non-gesture control. After the operation mode is selected, the input value entered into the operation control can be read based on the operation events on the operation control. It can be understood that the input value is the value of the operation mode corresponding to the operation control. For example, if the user clicks the gesture control category, the input value corresponding to the operation control is the gesture control category. Conversely, if the user clicks the non-gesture control category, the input value corresponding to the operation control is the non-gesture control category. Finally, after the input value of the operation control is determined, the browser control operation category is determined based on the input value. In this way, different browser control processes can be executed according to the different operation modes set by the user.

[0264] In some embodiments, the operation category of controlling the browser can also be determined by turning on or off a switch control. Figure 39 is a schematic diagram of determining the operation category of controlling the browser by using a switch control in some embodiments of the present application. As shown in Figure 39, the electronic device 200 can be provided with a switch control for the application to enable gesture operations. When the switch is off, the gesture control category will not be called and gesture recognition will not be performed. Only when the switch control is on will the gesture control category be opened and the gesture recognition process be executed.

[0265] When the user operates the browser in a non-gesture control category, the electronic device 200 may also provide the user with a gesture control category reminder message. Figure 40 is a schematic diagram of the effect of an electronic device displaying a gesture control category reminder message provided by some embodiments of the present application. As shown in Figure 40, in some embodiments, if the operation category is a non-gesture control category, the electronic device 200 may control the display 260 to display a gesture control reminder message on the current browser page. The gesture control reminder message is used to remind the user that the browser page supports gesture recognition.

[0266] For example, when the gesture control category is turned on, the gesture category can be recognized when performing operations on the browser. For example, a thumb gesture indicates adding the currently displayed webpage to favorites, an upward palm gesture indicates controlling the webpage to flip upward or navigate upward, a downward palm gesture indicates controlling the webpage to flip downward or navigate downward, and a left-right palm gesture indicates controlling the browser to switch tabs, etc. Specific gesture categories and operation rules can be pre-set and are not specifically limited in this application.

[0267] When the operation category of the browser operation selected by the user is the gesture control category, the electronic device 200 can start the camera 2901 and obtain the gesture image captured by the camera 2901. In some embodiments, after the gesture control category is started, the gesture image can be collected by the camera 2901 and the gesture image can be passed to an artificial intelligence algorithm such as an AI algorithm. In this way, the AI ​​algorithm can return the calculation results to the browser, and the electronic device can execute subsequent processes based on the returned calculation results. For example, the calculation results can be converted into control instructions, and the browser can be controlled according to the control instructions. After step S1 is completed, the following step S2 can be executed.

[0268] Step S2: the electronic device 200 identifies the gesture category in the gesture image.

[0269] In some embodiments, when a user selects an operation mode for manipulating a browser through an operation control, the operation control may include using a gesture control and not using a gesture control. When the electronic device 200 performs the identification of gesture categories in a gesture image, it may be implemented as follows. First, the electronic device 200 may obtain an input value, the input value including a first input value for performing an operation using a gesture control and a second input value for performing an operation not using a gesture control. Afterwards, if the input value is detected to be the first input value, an artificial intelligence algorithm is started, and the gesture image is identified through the artificial intelligence algorithm to return the gesture category contained in the gesture image. If the input value is detected to be the second input value, a direction key command sent by the remote control is received, and the direction key command is used to control the movement direction of the browser page, the tab switching of the browser page, and the page collection operation of the browser.

[0270] Exemplarily, the input value is the result of selecting the operation control. When the input value is the first input value, that is, the user chooses to use the gesture control, the electronic device 200 can activate an artificial intelligence algorithm, such as an AI algorithm, to recognize the gesture image, so that the gesture type contained in the gesture image can be obtained. When the input value is detected to be the second input value, that is, the user chooses not to use the gesture control, the browser can receive the direction key command sent by the remote control and control the movement direction of the browser page, tab switching, page collection, and other operations according to the cursor movement method. After step S2 is completed, the following step S3 can be executed.

[0271] Step S3: the electronic device 200 generates a control instruction according to the gesture category, and locks the control direction of the control instruction.

[0272] In some embodiments, the control instructions include dynamic control instructions, which can be understood as instructions generated by dynamic gesture operations. Figure 41 is a flow chart of how an electronic device generates dynamic control instructions from gesture categories in some embodiments of the present application, as shown in Figure 41, specifically including the following steps:

[0273] S4101, recording the initial position and target position of the user's palm according to the center point of the user's palm in the gesture image;

[0274] S4102, monitoring gesture change information based on the initial position and the target position;

[0275] S4103, determining the gesture category according to the gesture change information;

[0276] S4104, extracting the displacement direction and displacement distance of the center point in the gesture category;

[0277] S4105, parsing the displacement direction and displacement distance into dynamic control instructions;

[0278] S4106, dynamic control instructions.

[0279] Specifically, when generating dynamic control instructions based on a gesture category, electronic device 200 may first record the initial and target positions of the user's palm based on the center point of the user's palm in the gesture image. Then, based on the initial and target positions, it monitors gesture change information. Finally, based on this gesture change information, it determines the gesture category. Once the gesture category is determined, it extracts the displacement direction and distance of the center point within the gesture category, and finally interprets the displacement direction and distance as a dynamic control instruction.

[0280] Exemplarily, FIG42 is a schematic diagram of a scenario in which an electronic device provided by some embodiments of the present application generates dynamic control instructions from gesture categories. As shown in FIG42 , the electronic device 200 can identify the center point of the user's palm based on the gesture image, and then record the initial position and target position of the user's palm based on the center point. Gesture change information can be obtained based on the initial position and target information. For example, the gesture moves in the direction of the dotted arrow in FIG42 . The gesture category can be determined based on the gesture change information. For example, the gesture changes in the vertical direction, that is, it moves up and down, and the displacement direction and displacement distance can be obtained based on the center point of the user's palm. For example, the displacement distance can be calculated by the change in the coordinates of the center point of the palm. Then, based on the displacement direction and displacement distance, it can be parsed into a dynamic control instruction so that the user can operate the browser through the dynamic control instruction.

[0281] FIG43 is a flow chart of an electronic device generating a control instruction according to some embodiments of the present application. As shown in FIG43 , in some embodiments, the electronic device 200 may further perform the following process:

[0282] S4301: The electronic device 200 may first monitor the displacement direction and displacement distance;

[0283] S4302a: If the displacement direction is the first direction and the displacement distance is within the first distance range, the electronic device 200 generates a page-up instruction;

[0284] S4302b: If the displacement direction is the second direction and the displacement distance is within the first distance range, the electronic device 200 generates a page-down instruction; wherein the second direction is opposite to the first direction;

[0285] S4302c: If the displacement direction is the third direction and the displacement distance is within the second distance range, the electronic device 200 generates an instruction to switch tabs to the left, wherein the third direction is perpendicular to the first direction and the second direction;

[0286] S4302d: If the displacement direction is the fourth direction and the displacement distance is within the second distance range, the electronic device 200 generates an instruction to switch tabs to the right, wherein the fourth direction is opposite to the third direction.

[0287] For example, the browser's page turning, web browsing, and tab switching are resolved into dynamic control instructions after identifying the gesture category and monitoring the gesture change information. In some embodiments, the first distance range can be 5-12 cm, and the second distance range can also be 5-12 cm. The first direction can be an upward direction, the second direction can be a downward direction, the third direction can be a left direction, and the fourth direction can be a right direction. The second distance range can be the same as the first distance range or different. The specific set range can be set according to the actual operation needs of the user, and this application does not make specific limitations on this.

[0288] After the user's palm is recognized for the first time, the initial position of the palm can be recorded. Afterwards, the target position returned by the subsequent gesture image is checked. If the displacement distance of the upward movement of the gesture is monitored to be within 5-12 cm, an upward page-turning instruction is generated; if the displacement distance of the downward movement of the gesture is monitored to be within 5-12 cm, a downward page-turning instruction is generated. If the displacement distance of the leftward movement of the gesture is monitored to be within 5-12 cm, a left-tab-switching instruction is generated; if the displacement distance of the rightward movement of the gesture is monitored to be within 5-12 cm, a right-tab-switching instruction is generated. In this way, the user's gesture action can be parsed into a dynamic control instruction, thereby achieving the purpose of controlling the browser through gestures. In the process of controlling the browser through gestures, there is no need to press the direction keys of the remote control multiple times. After recognizing the user's gesture, the electronic device 200 will execute the corresponding control process. Therefore, it can solve the problem of cumbersome operation process and low operation efficiency when performing functions such as turning pages, moving pages up and down, or switching tabs through the browser.

[0289] In some embodiments, when calculating the displacement direction and displacement record, a certain point in the corner can also be used as the initial position. For example, the coordinates of the upper, lower, left and right rectangular areas of the user's palm can be obtained, and then the vertex in the upper left corner can be taken as the initial position of the palm. Then, when calculating the target position, the target position is also calculated based on the upper left corner as the base point. It can be selected according to the actual needs of the user, and this application does not make specific limitations. It can be understood that whether the center point of the palm or a vertex in the corner is used as the initial position, it has no effect on the calculation results of the displacement direction and displacement distance. After step S3 is completed, the following step S4 can be executed.

[0290] Step S4: The electronic device 200 controls the browser to perform a page operation based on the control instruction and the control direction, where the page operation is one of page movement, tab switching, and page collection.

[0291] After the dynamic control instruction is generated, the electronic device 200 can control the movement direction of the browser page, the tab switching of the browser page, and other operations based on the dynamic control instruction. For control instructions that do not detect gesture movement, such as instructions for triggering a favorite page through a preset gesture, the gesture does not actually move. For this type of control instruction, it can be called a static control instruction.

[0292] FIG44 is a flow chart of a display setting execution static control instruction according to some embodiments of the present application. As shown in FIG44 , the flow chart specifically includes the following steps:

[0293] S4401: In response to the control instruction being a static control instruction, the electronic device 200 may analyze the gesture posture corresponding to the gesture image;

[0294] S4402: If the gesture posture is a preset posture, generate a collection instruction according to the preset posture;

[0295] S4403: The electronic device 200 recognizes that the gesture postures in the first frame gesture image and the second frame gesture image are both preset gestures, and executes the collection instruction, wherein the second frame gesture image is a key frame located after the first frame gesture image.

[0296] Exemplarily, the preset gesture can be set according to actual needs. In some embodiments, the preset gesture can be a thumbs-up gesture. During the browser operation, after receiving the thumbs-up gesture, the electronic device 200 can detect that the gesture indicates that the page is to be collected.

[0297] In order to prevent the current page from being mistakenly added to the favorites or the instruction corresponding to the gesture from being repeatedly executed to be added to the favorites, in some embodiments, after receiving the thumbs-up gesture, the electronic device 200 will not immediately execute the static control instruction corresponding to the gesture, and will execute the favorite instruction when the gesture image in the next frame is still the thumbs-up gesture. It should be noted that the next frame should be continuous with the previous frame. In this way, on the one hand, the user's gesture can be confirmed, and on the other hand, repeated favorites of the page can be avoided. When other subsequent frames are also the same gesture, since the page has been favorited in the early stage, the electronic device 200 will not perform the same function, that is, the electronic device 200 will ignore the static control instruction until the gesture changes. For example, until the gesture is empty, or the thumbs-up gesture is switched to other gestures.

[0298] When the user is executing a dynamic control instruction, the gesture may not be completely horizontal or completely vertical. The human hand may not be absolutely vertical or absolutely horizontal during movement, and may have a certain tilt. When the user's gesture is tilted, in order to accurately analyze the user's intention, the electronic device 200 can execute the following process. In some embodiments, the electronic device 200 can first obtain the first absolute value of the displacement distance in the vertical direction and the second absolute value in the horizontal direction; if the first absolute value is greater than the second absolute value, a dynamic control instruction is generated according to the vertical direction and the displacement distance; if the first absolute value is less than the second absolute value, a dynamic control instruction is generated according to the horizontal direction and the displacement distance.

[0299] For example, if the first absolute value of the vertical displacement distance is larger, it indicates that the user may want to perform vertical movement. Therefore, the electronic device 200 can generate a dynamic control instruction based on the vertical displacement distance and direction. Conversely, if the second absolute value of the horizontal displacement distance is larger, it indicates that the user may want to perform horizontal movement. Therefore, the electronic device 200 can generate a dynamic control instruction based on the horizontal displacement distance and direction. In other words, selecting the value with the larger absolute value as the basis for converting the control instruction can be more consistent with the user's actual choice.

[0300] When the user's gesture direction is tilted, the tilt angle may be different each time. This may cause errors each time the browser is dynamically controlled, that is, the electronic device 200 may not recognize the user's true intention. For example, the user wants to keep turning pages up and down, but after several times of turning up and down, the tilt angle changes, and the comparison result of the first absolute value and the second absolute value will also change. In this scenario, the browser may be misoperated.

[0301] In order to prevent misoperation of the browser during gesture control, in some embodiments, the electronic device 200 will lock the direction of the initial movement of the palm during the gesture control process, that is, the control direction is the direction of the initial movement of the gesture for controlling the page through gestures. The electronic device 200 can first identify the initial displacement distance of the center point of the user's palm; wherein the initial displacement distance includes a first displacement distance in the vertical direction and a second displacement distance in the horizontal direction. If the first displacement distance is greater than the second displacement distance, the electronic device 200 locks the direction of the gesture movement to the vertical direction, and does not calculate the gesture movement in the horizontal direction; if the second displacement distance is greater than the first displacement distance, the electronic device 200 locks the direction of the gesture movement to the horizontal direction, and does not calculate the gesture movement in the vertical direction.

[0302] For example, FIG45 is a schematic diagram of the principle of determining the direction of gesture movement of an electronic device provided by some embodiments of the present application. As shown in FIG45, the initial coordinates of the center point of the user's palm are (X center ,Y center ), the coordinates of the target center point after movement are (X, Y), then the displacement of the palm in the X-axis direction is diffX=XX center , the displacement in the Y-axis direction is diffY=YY center If diffX is greater than 0, it means the gesture is moving to the right, otherwise it means the gesture is moving to the left. If diffY is greater than 0, it means the gesture is moving downward, otherwise it means the gesture is moving upward.

[0303] After the movement direction of the gesture is recognized for the first time, if the first gesture movement direction is up and down, the electronic device 200 will lock the vertical direction, only consider the displacement in the vertical direction, and ignore the displacement in the horizontal direction, so as to prevent the control instructions for left and right movement from being parsed, and prevent the situation of switching tabs by mistake. When the first gesture movement direction is left and right, the electronic device 200 will lock the horizontal direction, only consider the displacement in the horizontal direction, and ignore the displacement in the vertical direction, so as to prevent the control instructions for up and down movement from being parsed, and prevent the situation of turning pages by mistake or roaming by mistake. In this way, by locking the movement direction during the gesture control process, the electronic device 200 can accurately parse the user's true intention, avoid the situation of misoperation of the browser, and improve the accuracy of browser operation.

[0304] As can be seen from the above technical implementation methods, the above embodiment provides an electronic device 200, which, in response to an operation event of starting a browser, obtains the operation category of controlling the browser input by the user; if the operation category is a gesture control category, starts a camera, and obtains a gesture image captured by the camera; identifies the gesture category in the gesture image; generates a control instruction according to the gesture category, and locks the control direction of the control instruction; controls the movement direction of the browser page, the tab switching of the browser page, and the page collection operation of the browser based on the control instruction and the control direction. In the process of controlling the browser through gestures, the electronic device does not need to press the direction keys of the remote control multiple times once, and will execute the corresponding control process when recognizing the user's gesture. Therefore, it can solve the problem of cumbersome operation process and low operation efficiency when executing functions such as turning pages, moving pages up and down, or switching tabs through the browser.

[0305] Some embodiments of the present application also provide a method for controlling a browser based on gesture recognition, which can be applied to the electronic device 200 in the above embodiment. Figure 46 is a flow chart of a method for controlling a browser based on gesture recognition provided by some embodiments of the present application. As shown in Figure 46, in some embodiments, the method for controlling a browser based on gesture recognition can include the following steps S1-S4, the specific contents of which are as follows:

[0306] Step S1: In response to an operation event of starting a browser, the electronic device 200 obtains a gesture image captured by the camera 2901 .

[0307] In some embodiments, browser control operations can be categorized into gesture control and non-gesture control. Gesture control refers to controlling browser-related operations through gesture movements or gesture postures, such as turning pages, switching tabs, and adding to favorites. Non-gesture control refers to controlling the browser through non-gesture methods. For example, you can use the remote control cursor to control browser operations such as turning pages, switching tabs, and adding to favorites.

[0308] In some embodiments, the electronic device 200 can determine the browser's operation category by capturing control instructions received by the browser. For example, if the control instruction received by the browser is issued by a directional key on a remote control, the browser's operation category can be determined to be non-gesture control. If the control instruction received by the browser is converted from an image containing gesture motion, the browser's operation category can be determined to be gesture control.

[0309] In some embodiments, the browser control operation category can also be determined by turning on or off a switch control. The electronic device 200 can be provided with a switch control for the application to enable gesture operations. When the switch is off, the gesture control category is not called and gesture recognition is not performed. Only when the switch control is on will the gesture control category be opened and the gesture recognition process be executed.

[0310] When the user selects the gesture control category for operating the browser, the electronic device 200 can activate the camera 2901 and obtain the gesture image captured by the camera 2901. In some embodiments, after the gesture control category is activated, the gesture image can be captured by the camera 2901 and transmitted to an artificial intelligence algorithm, such as an AI algorithm. In this way, the AI ​​algorithm can return the calculation results to the browser, and the electronic device can execute subsequent processes based on the returned calculation results. For example, the calculation results can be converted into control instructions, and the browser can be operated according to the control instructions.

[0311] Step S2: the electronic device 200 identifies the gesture category in the gesture image.

[0312] In some embodiments, when a user selects an operation mode for manipulating a browser through an operation control, the operation control may include using a gesture control and not using a gesture control. When the electronic device 200 performs the identification of gesture categories in a gesture image, it may be implemented as follows. First, the electronic device 200 may obtain an input value, the input value including a first input value for performing an operation using a gesture control and a second input value for performing an operation not using a gesture control. Afterwards, if the input value is detected to be the first input value, an artificial intelligence algorithm is started, and the gesture image is identified through the artificial intelligence algorithm to return the gesture category contained in the gesture image. If the input value is detected to be the second input value, a direction key command sent by the remote control is received, and the direction key command is used to control the movement direction of the browser page, the tab switching of the browser page, and the page collection operation of the browser.

[0313] Step S3: the electronic device 200 generates a control instruction according to the gesture category, and locks the control direction of the control instruction.

[0314] In some embodiments, control instructions include dynamic control instructions, which can be understood as instructions generated through dynamic gesture operations. When generating dynamic control instructions based on gesture categories, the electronic device 200 can first record the initial and target positions of the user's palm based on the center point of the user's palm in the gesture image. Then, based on the initial and target positions, it monitors gesture change information; and then determines the gesture category based on the gesture change information. Once the gesture category is determined, the displacement direction and distance of the center point in the gesture category can be extracted, and finally, the displacement direction and distance can be parsed into dynamic control instructions.

[0315] In some embodiments, the electronic device 200 can also perform the following process. The electronic device 200 can first monitor the displacement direction and displacement distance. If the displacement direction is the first direction and the displacement distance is within the first distance range, the electronic device 200 generates an upward page turning instruction; if the displacement direction is the second direction and the displacement distance is within the first distance range, the electronic device 200 generates a downward page turning instruction; wherein the second direction is opposite to the first direction; if the displacement direction is the third direction and the displacement distance is within the second distance range, the electronic device 200 generates a left tab switching instruction, wherein the third direction is perpendicular to the first direction and the second direction; if the displacement direction is the fourth direction and the displacement distance is within the second distance range, the electronic device 200 generates a right tab switching instruction, wherein the fourth direction is opposite to the third direction. In this way, the user's gesture action can be parsed into a dynamic control instruction, thereby achieving the purpose of controlling the browser through gestures. In the process of controlling the browser through gestures, there is no need to press the direction keys of the remote control multiple times. After recognizing the user's gesture, the electronic device 200 will execute the corresponding control process. Therefore, it can solve the problem of cumbersome operation process and low operation efficiency when performing functions such as turning pages, moving pages up and down, or switching tabs through the browser.

[0316] Step S4: The electronic device 200 controls the browser to perform a page operation based on the control instruction and the control direction, where the page operation is one of page movement, tab switching, and page collection.

[0317] In order to prevent the current page from being mistakenly added to the favorites or the instruction corresponding to the gesture from being repeatedly executed to be added to the favorites, in some embodiments, after receiving the thumbs-up gesture, the electronic device 200 will not immediately execute the static control instruction corresponding to the gesture, and will execute the favorite instruction when the gesture image in the next frame is still the thumbs-up gesture. It should be noted that the next frame should be continuous with the previous frame. In this way, on the one hand, the user's gesture can be confirmed, and on the other hand, repeated favorites of the page can be avoided. When other subsequent frames are also the same gesture, since the page has been favorited in the early stage, the electronic device 200 will not perform the same function, that is, the electronic device 200 will ignore the static control instruction until the gesture changes. For example, until the gesture is empty, or the thumbs-up gesture is switched to other gestures.

[0318] When the user is executing a dynamic control instruction, the gesture may not be completely horizontal or completely vertical. The human hand may not be absolutely vertical or absolutely horizontal during movement, and may have a certain tilt. When the user's gesture is tilted, in order to accurately analyze the user's intention, the electronic device 200 can execute the following process. In some embodiments, the electronic device 200 can first obtain the first absolute value of the displacement distance in the vertical direction and the second absolute value in the horizontal direction; if the first absolute value is greater than the second absolute value, a dynamic control instruction is generated according to the vertical direction and the displacement distance; if the first absolute value is less than the second absolute value, a dynamic control instruction is generated according to the horizontal direction and the displacement distance.

[0319] In order to prevent misoperation of the browser during gesture control, in some embodiments, the electronic device 200 will lock the initial movement direction of the palm during gesture control. After the gesture movement direction is first recognized, if the initial gesture movement direction is up and down, the electronic device 200 will lock the vertical direction, only consider the displacement in the vertical direction, and ignore the displacement in the horizontal direction, so as to prevent the control instructions for left and right movement from being parsed, and prevent the situation of mistakenly switching tabs. When the initial gesture movement direction is left and right, the electronic device 200 will lock the horizontal direction, only consider the displacement in the horizontal direction, and ignore the displacement in the vertical direction, so as to prevent the control instructions for up and down movement from being parsed, and prevent the situation of mistakenly turning pages or mistakenly roaming. In this way, by locking the movement direction during gesture control, the electronic device 200 can accurately parse the user's true intention, avoid misoperation of the browser, and improve the accuracy of browser operation.

[0320] It can be seen from the above technical implementation methods that the above embodiments provide a method for controlling a browser based on gesture recognition. In the process of controlling the browser through gestures, there is no need to press the direction keys of the remote control one or more times. Once the user's gesture is recognized, the corresponding control process will be executed. Therefore, it can solve the problem of cumbersome operation process and low operation efficiency when performing functions such as turning pages, moving pages up and down, or switching tabs through the browser.

[0321] In addition to the above embodiments, this application also provides other embodiments of electronic devices and remote gesture recognition methods, which are described in detail below. Gesture interaction is based on the principle of vision. The camera on the TV captures the user's movements and determines whether they are preset movements, thereby executing the corresponding function.

[0322] As chip performance improves, AI computing power also increases, providing technical support for the use of AI algorithms for gesture recognition in smart TVs. Gesture control refers to interpreting human gestures in images captured by a camera as machine control commands. Because the AI ​​(artificial intelligence) computing power of smart TVs or mobile devices is lower than that of chips in computers, lightweight AI models are used in TVs or mobile devices. To reduce TV memory usage and shorten the time it takes for the TV to recognize human gestures, the images from the camera need to be compressed.

[0323] In existing technology, when using AI algorithms to implement gesture control in smart TVs, the original pixel images captured by the camera are typically compressed to target pixels. Typically, the original image captured by the camera has a resolution of 1920*1080 or 960*540. This original image is proportionally compressed several times to obtain an image with a resolution of 512*288 or 320*320, thereby reducing the TV's memory usage and shortening recognition time. However, when the human gesture is far from the camera, the gesture is displayed in a smaller area in the image. After image compression, the gesture becomes blurry, resulting in less accurate gesture recognition.

[0324] For example, Figure 47 shows a person taking a photo about 5 meters from a TV. The 1920*1080 pixel image was compressed to a 512*288 pixel image to reduce the TV's memory usage and shorten recognition time. Figure 48 shows the gesture before compression, while Figure 49 shows the gesture after compression. The compressed gesture clearly blurs the edges.

[0325] For gesture recognition algorithms, the input image is affected by the resolution. The higher the resolution, the clearer the image, and vice versa. Usually, the AI ​​algorithm recognition rate is greater than 90% within 3 meters. When the distance between the human gesture and the camera is greater than 3 meters, the image is compressed and becomes blurred, and the recognition rate is significantly reduced. It cannot be recognized beyond 6 meters, especially for large-screen TVs. When watching a TV larger than 70 inches, the distance between the person and the TV is generally greater than 3 meters; when watching a 100-inch laser TV, the distance between the person and the TV is generally greater than 5 meters; and for these larger TVs, the probability of gestures not being recognized is higher.

[0326] Based on the above problems, an embodiment of the present application provides a long-distance gesture recognition method, which obtains an original image of the scene in which the electronic device is located; obtains a first flag, wherein the first flag is used to indicate whether to intercept a target area on the original image. When the first flag is true, a target area of ​​a target size is intercepted on the original image to obtain a target image, and the target image is recognized based on a gesture recognition algorithm to obtain a gesture recognition result. Since the target size is the image size supported by the gesture recognition algorithm, and the target image of the target area coincides with the center position of the original image, when the user is far away from the electronic device, the center of the original image is intercepted according to the size required by the gesture algorithm to obtain the target image. This image interception method can avoid the problem of inaccurate gesture recognition caused by compression of the original image in the prior art, and further improve the accuracy of gesture recognition.

[0327] In some embodiments, the electronic device 200 can receive user control through touch or gestures, etc. For example, when the electronic device 200 is a smart TV, the user controls the electronic device 200 by touching the display or performing gesture control.

[0328] The long-distance gesture recognition method provided in the embodiments of the present application can be implemented based on a computer device, or a functional module or functional entity in the computer device.

[0329] The computer device may be a personal computer (PC), a server, a mobile phone, a tablet computer, a laptop computer, a mainframe computer, etc., and the embodiments of the present application do not specifically limit this.

[0330] Figure 50 is a software configuration diagram of a computer device according to one or more embodiments of the present application. As shown in Figure 50, the system is divided into four layers, from top to bottom, namely, the application layer (referred to as "application layer"), the application framework layer (referred to as "framework layer"), the data acquisition (Software Development Kit) and the system library layer (referred to as "system runtime layer").

[0331] In the embodiment of the present application, the APP application layer is responsible for calling the video acquisition device such as the camera, calling the SDK data acquisition layer interface to pass the acquired image to the SDK layer for AI calculation, receiving the gesture recognized by the SDK, converting the gesture into a control instruction, and calling the control instruction to perform the corresponding action. The application layer calls the AI ​​algorithm through the interface and returns the AI ​​operation result. Among them, the application layer encapsulates the image acquisition unit, the interface call unit, the image conversion unit, the instruction conversion unit, the result receiving unit, and the instruction execution unit, and the SDK layer encapsulates the image processing unit, the image recognition model unit, and the result processing unit.

[0332] Figure 51 is a schematic diagram showing the icon control interface display of an application included in a smart device (mainly a smart playback device, such as a smart TV, a digital cinema system, or an audio and video server, etc.) according to one or more embodiments of the present application. As shown in Figure 51, the application layer includes at least one application that can display a corresponding icon control on the display, such as: a live TV application icon control, a video on demand VOD application icon control, a media center application icon control, an application center icon control, a game application icon control, etc. A live TV application can provide live TV through different signal sources. A video on demand VOD application can provide videos from different storage sources. Unlike a live TV application, video on demand provides video display from certain storage sources. A media center application can provide applications for playing various multimedia content. An application center can provide storage for various applications.

[0333] In order to explain this embodiment in more detail, the following will be explained in an exemplary manner in conjunction with Figure 52. It can be understood that although the various steps in the flowchart of Figure 52 are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in Figure 52 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps. It shall be based on the ability to implement the long-distance gesture recognition method provided in the embodiment of the present invention.

[0334] As shown in FIG52 , the method specifically includes the following steps:

[0335] S41: Acquire an original image of the scene where the electronic device is located.

[0336] The electronic device may be a smart TV that supports gesture control. The electronic device may have a camera function or be connected to a camera device; the image resolution captured by the camera may be 1920*1080 or 960*640, etc.

[0337] The camera captures an image of a scene in which the electronic device is located. The scene can be within a preset angle range directly in front of the smart TV. The preset angle can be -135° to +135°, -180° to +180°, or other reasonable values, and is not specifically limited here.

[0338] The settings interface of the electronic device may be provided with a camera switch control. When the camera control is turned on, the electronic device may use the camera to capture images. The camera captures images at a frequency of 3 frames per second, 5 frames per second, or other reasonable values, which are not specifically limited here. The original image can have a pixel size of 1920*1080, 960*640, etc.

[0339] S42: Obtain the first flag bit.

[0340] The first flag is used to indicate whether to capture a target area of ​​a target size on the original image, where the target image corresponding to the target area coincides with the center of the original image. The target size is the image size supported by the gesture recognition algorithm.

[0341] The first flag bit is used to determine whether the image needs to be processed by a preset process. Here, the preset process is determined based on the pixels of the acquired original image. When the original image has a pixel size of 960*640, the preset process involves capturing the target area from the original image. When the original image has a pixel size of 1920*1080, the preset process includes first compressing the original image by a factor of 2 to obtain a 960*640 image, and then capturing the target area from the original image.

[0342] Exemplarily, the first flag bit may be set to IsDealPicture. When IsDealPicture is false, it indicates that the gesture is still not recognized after the preset processing.

[0343] It is understandable that when IsDealPicture is false, for the two different original pixel resolutions mentioned above, if the gesture is still not recognized after different preset processing, it means that the target area may not contain a gesture. Therefore, when processing the image captured in the next frame, the first compression processing method can be first used, that is, the original image can be proportionally compressed to 512*288 pixels to roughly estimate whether the original image contains a gesture.

[0344] S43: When the first flag is true, intercept a target area of ​​a target size on the original image to obtain a target image.

[0345] When the first flag is true, it is determined that a target area needs to be captured on the original image. The target area is sized as the target size, which is the image size supported by the gesture recognition algorithm. The target image is obtained by capturing the target area at the center of the original image at the target size.

[0346] For example, because the AI ​​(artificial intelligence) computing power of smart TVs or mobile devices is lower than that of chips in computers, lightweight AI models are used in these devices. However, lightweight AI models are limited in the size of images they can recognize; generally, the target size can be 512*288.

[0347] S44: Recognize the target image based on the gesture recognition algorithm to obtain a gesture recognition result.

[0348] Specifically, the input of the gesture recognition algorithm is a target image, and the target size of the target image is the image size supported by the gesture recognition algorithm. In the embodiment of the present application, the target size is 512*288.

[0349] The output of the gesture recognition algorithm includes the gesture type, the similarity corresponding to the gesture type, and the gesture's location coordinates in the target image. For example, gesture types can include, but are not limited to, "0" through "9"; the similarity corresponding to each gesture type, for example, the gesture "ok" (i.e., gesture "0") has a predicted similarity of 90%; and the gesture's location coordinates in the target image.

[0350] In an embodiment of the present application, an original image of a scene in which an electronic device is located is obtained; a first flag is obtained, wherein the first flag is used to indicate whether a target area is to be captured on the original image. When the first flag is true, a target area of ​​a target size is captured on the original image to obtain a target image. The target image is then recognized based on a gesture recognition algorithm to obtain a gesture recognition result. Since the target size is an image size supported by the gesture recognition algorithm, and the target image of the target area coincides with the center of the original image, when the user is far away from the electronic device, the center of the original image is captured according to the size required by the gesture algorithm to obtain the target image. This image capture method can avoid the problem of inaccurate gesture recognition caused by compression of the original image in the prior art, further improving the accuracy of gesture recognition.

[0351] The gesture recognition method based on the above steps may have a drawback: if the original image does not contain the target gesture, even if you crop the center of the original image, you will not be able to obtain gesture recognition results. Therefore, you need to crop the target area of ​​the original image to the target size and perform detection on the original image before obtaining the target image.

[0352] FIG53 is a flow chart of another method for remote gesture recognition provided by an embodiment of the present application. This embodiment is a further expansion and optimization based on FIG52. Optionally, in some embodiments, the remote gesture recognition method may include the following steps:

[0353] S51: Acquire an original image of the scene where the electronic device is located.

[0354] S52: Obtain the first flag bit.

[0355] The first flag is used to indicate whether to capture a target area of ​​a target size on the original image, where the target image corresponding to the target area coincides with the center of the original image. The target size is the image size supported by the gesture recognition algorithm.

[0356] S53: Detect whether there is a gesture of the target object in the target area.

[0357] If there is a gesture of the target object in the target area, it is determined that the first flag is true, that is, step S54 is executed.

[0358] The human body key point gesture recognition technology can be used to detect the target image and determine whether the target area contains the gesture of the target object.

[0359] Among them, human key point posture recognition is an important branch of human posture recognition technology. By detecting the positions of key parts of human posture in each frame during the expression of human behavior, human posture is simplified into human key points, and the semantics of human posture expression are classified and recognized through these key points.

[0360] Human posture recognition based on key points can be divided into two aspects, namely static human posture recognition and dynamic human posture recognition. In the embodiment of the present application, the camera continuously captures the original image of the scene in which the electronic device is located, and the electronic device continuously processes the captured image, which can be understood as dynamic human posture recognition; and for a single frame image captured by the camera, it can be understood as static human posture recognition.

[0361] S54: When the first flag is true, intercept a target area of ​​a target size on the original image to obtain a target image.

[0362] S55: Recognize the target image based on the gesture recognition algorithm to obtain a gesture recognition result.

[0363] Among them, the implementation method of steps S51-S52 and S54-S55 is the same as the implementation method of the above steps S41-S44, and will not be repeated here.

[0364] The above step S53 avoids the defect that when the original image does not contain the gesture of the target object, the gesture recognition result cannot be obtained by cutting out the center image of the original image of the target size, thereby further improving the accuracy of gesture recognition.

[0365] In an embodiment of the present application, an original image of a scene in which an electronic device is located is obtained; a first flag is obtained, wherein the first flag is used to indicate whether a target area is to be captured on the original image. When the first flag is true, a target area of ​​a target size is captured on the original image to obtain a target image. The target image is then recognized based on a gesture recognition algorithm to obtain a gesture recognition result. Since the target size is an image size supported by the gesture recognition algorithm, and the target image of the target area coincides with the center of the original image, when the user is far away from the electronic device, the center of the original image is captured according to the size required by the gesture algorithm to obtain the target image. This image capture method can avoid the problem of inaccurate gesture recognition caused by compression of the original image in the prior art, further improving the accuracy of gesture recognition.

[0366] FIG54 is a flow chart of another method for remote gesture recognition provided by an embodiment of the present application. This embodiment is a further expansion and optimization based on FIG52. Optionally, in some embodiments, the remote gesture recognition method may further include the following steps:

[0367] S61: Acquire an original image of the scene where the electronic device is located.

[0368] S62: Obtain the first flag bit.

[0369] The first flag is used to indicate whether to capture a target area of ​​a target size on the original image, where the target image corresponding to the target area coincides with the center of the original image. The target size is the image size supported by the gesture recognition algorithm.

[0370] S63: When the first flag is true, intercept a target area of ​​a target size on the original image to obtain a target image.

[0371] S64: Recognize the target image based on the gesture recognition algorithm to obtain a gesture recognition result.

[0372] Among them, the implementation method of steps S61-S64 is the same as the implementation method of the above steps S41-S44, and will not be repeated here.

[0373] S65: When the first flag is false, perform a first compression process on the original image to obtain a first compressed image.

[0374] The first compression process means compressing the original image according to a first compression ratio to obtain a target image of a target size. In this embodiment, the target size can be, but is not limited to, 512*288.

[0375] For example, if the resolution of the original image is 1920*1080, or 960*640, it can be compressed to a target size of 512*288.

[0376] S66: Recognize the first compressed image based on the gesture recognition algorithm.

[0377] S67: Determine whether there is a gesture of the target object in the first compressed image.

[0378] If the gesture of the target object exists in the first compressed image, step S68 is executed.

[0379] S68: Set the first flag bit to false.

[0380] If the gesture of the target object does not exist in the first compressed image, step S69 is executed.

[0381] S69: Cut out a target area of ​​a target size from the original image to obtain a target image.

[0382] For the above original image, if the gesture is still not recognized after the above-mentioned interception processing, it means that the target area may not contain a gesture. Therefore, when processing the image captured by the next frame, you can first use the first compression processing method, that is, compress the original image proportionally to 512*288 pixels to roughly estimate whether the target area contains a gesture. If the target area in the returned result contains a gesture, the returned result is converted to gesture control, and the first flag is set to false, indicating that the image does not need to be intercepted. If the target area in the returned result does not contain a gesture or the similarity corresponding to the gesture is low, it means that the gesture of the target object is not detected, or the distance between the person and the TV exceeds the effective detection range of 3 meters, then the image needs to be intercepted to obtain the target image.

[0383] Since the resolution of the original image collected by commonly used ordinary cameras is generally 1920*1080, when the distance between the person and the TV exceeds the effective detection range of 3 meters, the original image with pixels of 1920*1080 can be compressed by 1 / 2 to obtain an image with pixels of 960*540. Compared with the existing technology that directly compresses the image by 1 / 3.75 times to obtain a target image of 512*288, this can reduce the pixels lost in image compression and is more conducive to gesture recognition.

[0384] FIG55 is a flow chart of another method for remote gesture recognition provided by an embodiment of the present application. This embodiment is a further expansion and optimization based on FIG54. Optionally, in some embodiments, the remote gesture recognition method may further include the following steps:

[0385] S61: Acquire an original image of the scene where the electronic device is located.

[0386] S62: Obtain the first flag bit.

[0387] The first flag is used to indicate whether to capture a target area of ​​a target size on the original image, where the target image corresponding to the target area coincides with the center of the original image. The target size is the image size supported by the gesture recognition algorithm.

[0388] S63: When the first flag is true, intercept a target area of ​​a target size on the original image to obtain a target image.

[0389] Among them, the implementation method of steps S61-S63 is the same as the implementation method of the above steps S41-S43, and will not be repeated here.

[0390] S64: Recognize the target image based on the gesture recognition algorithm to obtain a gesture recognition result.

[0391] In some embodiments, step S64 may further include the following steps:

[0392] S641: When there is a gesture of the target object in the target image, set the first flag to true.

[0393] Since the camera can be set to capture 3-5 images per second, the 3-5 images within 1 second can be considered to have similar movements. That is, when the gesture of the target object exists in the current frame, the first flag is set to true, and the next frame is captured in the same way to obtain the target image. The target image is then input into the gesture recognition algorithm for recognition to obtain the gesture recognition result.

[0394] S642: When there is no gesture of the target object in the target image, set the first flag to false.

[0395] Similarly, since the camera can be set to take 3-5 images per second, the 3-5 images within 1 second can be considered to have little difference in the actions performed by the person. That is, when there is no gesture of the target object in the current frame, the next frame of image does not need to be intercepted and processed according to the above-mentioned interception method, which can reduce the memory usage of the TV. Therefore, the first flag is set to false.

[0396] S65: When the first flag is false, perform a first compression process on the original image to obtain a first compressed image.

[0397] The first compression process means compressing the original image according to a first compression ratio to obtain a target image of a target size. In this embodiment, the target size can be, but is not limited to, 512*288.

[0398] For example, if the resolution of the original image is 1920*1080, or 960*640, it can be compressed to a target size of 512*288.

[0399] S66: Recognize the first compressed image based on the gesture recognition algorithm.

[0400] S67: Determine whether there is a gesture of the target object in the first compressed image.

[0401] If the gesture of the target object exists in the first compressed image, step S68 is executed.

[0402] S68: Set the first flag bit to false.

[0403] If the gesture of the target object does not exist in the first compressed image, step S69 is executed.

[0404] S69: Cut out a target area of ​​a target size from the original image to obtain a target image.

[0405] Among them, steps S65-S69 are implemented in the same way as in the above embodiment Figure 54 and will not be repeated here.

[0406] In addition, since the camera captures images quickly, compression processing can be performed when the next frame of image is obtained, and using the next frame of image does not affect the gesture recognition effect.

[0407] In an embodiment of the present application, an original image of a scene in which an electronic device is located is obtained; a first flag is obtained, wherein the first flag is used to indicate whether a target area is to be captured on the original image. When the first flag is true, a target area of ​​a target size is captured on the original image to obtain a target image. The target image is then recognized based on a gesture recognition algorithm to obtain a gesture recognition result. Since the target size is an image size supported by the gesture recognition algorithm, and the target image of the target area coincides with the center of the original image, when the user is far away from the electronic device, the center of the original image is captured according to the size required by the gesture algorithm to obtain the target image. This image capture method can avoid the problem of inaccurate gesture recognition caused by compression of the original image in the prior art, further improving the accuracy of gesture recognition.

[0408] FIG56 is a flow chart of another method for remote gesture recognition provided by an embodiment of the present application. This embodiment is a further expansion and optimization based on FIG56. Optionally, in some embodiments, the remote gesture recognition method may further include the following steps:

[0409] S71: Acquire an original image of the scene where the electronic device is located.

[0410] S72. Obtain the first flag bit.

[0411] The first flag is used to indicate whether to capture a target area of ​​a target size on the original image, where the target image corresponding to the target area coincides with the center of the original image. The target size is the image size supported by the gesture recognition algorithm.

[0412] Among them, the implementation method of steps S71-S72 is the same as the implementation method of the above steps S41-S42, and will not be repeated here.

[0413] S73: Obtain the resolution of the original image.

[0414] S74. When the resolution of the original image is the preset resolution, perform a second compression process on the original image to obtain a second compressed image.

[0415] The second compression processing means compressing the original image according to a second compression ratio to obtain a target image of a target size.

[0416] S75 . Cut out a target area of ​​a target size from the second compressed image to obtain a target image.

[0417] S76: Recognize the target image based on the gesture recognition algorithm to obtain a gesture recognition result.

[0418] For example, as shown in Figure 57, assuming the original image resolution is 1920*1080, the original 1920*1080 image can be first compressed by 1 / 2 to obtain a second compressed image of 960*540 pixels. The target area of ​​the target size can then be captured on the second compressed image. Referring to Figures 58 and 59, Figure 58 shows the gesture pixels before compression, and Figure 59 shows the gesture pixels after the second compression process. Comparing Figures 58 and 59, it can be seen that although some pixels are lost after the second compression, this does not affect the subsequent recognition results. Furthermore, compared to the first compressed image of 512*288 obtained by directly compressing it by 1 / 3.75 times using the prior art, this second compressed image loses relatively fewer pixels due to image compression, making it more conducive to gesture recognition.

[0419] In an embodiment of the present application, an original image of a scene in which an electronic device is located is obtained; a first flag is obtained, wherein the first flag is used to indicate whether a target area is to be captured on the original image. When the first flag is true, a target area of ​​a target size is captured on the original image to obtain a target image. The target image is then recognized based on a gesture recognition algorithm to obtain a gesture recognition result. Since the target size is an image size supported by the gesture recognition algorithm, and the target image of the target area coincides with the center of the original image, when the user is far away from the electronic device, the center of the original image is captured according to the size required by the gesture algorithm to obtain the target image. This image capture method can avoid the problem of inaccurate gesture recognition caused by compression of the original image in the prior art, further improving the accuracy of gesture recognition.

[0420] An embodiment of the present application provides a computer device, comprising: one or more processors; and a storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement any of the long-distance gesture recognition methods described in the embodiments of the present application.

[0421] Figure 60 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. As shown in Figure 60 , the computer device includes a processor 810 and a storage device 820. The number of processors 810 in the computer device can be one or more, and Figure 60 uses one processor 810 as an example. The processor 810 and storage device 820 in the computer device can be connected via a bus or other means, and Figure 60 uses a bus connection as an example.

[0422] Storage device 820, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the remote gesture recognition method in the embodiments of the present application. Processor 810 executes the software programs, instructions, and modules stored in storage device 820 to execute various functional applications and data processing of the computer device, thereby implementing the remote gesture recognition method provided in the embodiments of the present application.

[0423] The storage device 820 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the terminal, etc. In addition, the storage device 820 may include a high-speed random access storage device and may also include a non-volatile storage device, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some examples, the storage device 820 may further include a storage device remotely located relative to the processor 810, and these remote storage devices may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0424] The computer device provided in this embodiment can be used to execute the long-distance gesture recognition method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0425] In addition to the above embodiments, the present application also provides some other embodiments related to electronic devices and long-distance gesture recognition methods, which are described in detail as follows.

[0426] As shown in FIG52 , a long-distance gesture recognition method provided by the present application specifically includes the following steps:

[0427] S41: Acquire an original image of the scene where the electronic device is located.

[0428] The electronic device may be a smart TV that supports gesture control. The electronic device may have a camera function or be connected to a camera device. The image resolution captured by the camera may be 1920*1080 or 960*640, etc.

[0429] The camera captures an image of a scene in which the electronic device is located. The scene can be within a preset angle range directly in front of the smart TV. The preset angle can be -135° to +135°, -180° to +180°, or other reasonable values, and is not specifically limited here.

[0430] The settings interface of the electronic device may be provided with a camera switch control. When the camera control is turned on, the electronic device may use the camera to capture images. The camera captures images at a frequency of 3 frames per second, 5 frames per second, or other reasonable values, which are not specifically limited here. The original image can have a pixel size of 1920*1080, 960*640, etc.

[0431] For example, it is assumed that the electronic device includes a camera module with a camera function. The camera module acquires original images at a frequency of 3 images per second, and the camera module sends the acquired original images to the processor of the electronic device in real time.

[0432] S42: Obtain the first flag bit.

[0433] The first flag is used to indicate whether the human body recognition algorithm is called to recognize the gesture of the target object contained in the original image.

[0434] The value of the first flag is used to determine whether the human body recognition algorithm has recognized the gesture of the target object contained in the original image. The human body recognition algorithm may be, but is not limited to, a human body key point gesture recognition algorithm.

[0435] The human body key point gesture recognition technology can be used to detect the target image and determine whether the target area contains the gesture of the target object.

[0436] Among them, human key point posture recognition is an important branch of human posture recognition technology. By detecting the positions of key parts of human posture in each frame during the expression of human behavior, human posture is simplified into human key points, and the semantics of human posture expression are classified and recognized through these key points.

[0437] Human posture recognition based on key points can be divided into two aspects, namely static human posture recognition and dynamic human posture recognition. In the embodiment of the present application, the camera continuously captures the original image of the scene in which the electronic device is located, and the electronic device continuously processes the captured image, which can be understood as dynamic human posture recognition; and for a single frame image captured by the camera, it can be understood as static human posture recognition.

[0438] It should be noted that, in some embodiments, considering that the relative position of the user and the electronic device usually changes when the user performs gesture control on the electronic device, the gestures in the acquired image may be located in different positions. For example, for the collected original image, through human body key point posture recognition, some gestures are located on the left side of the original image, some gestures are located in the middle of the original image, and some gestures are located on the right side of the original image.

[0439] S43: When the first flag is true, intercept a target area of ​​a target size on the original image to obtain a target image.

[0440] The target size is an image size supported by a gesture recognition algorithm, and the target area is an area in the original image centered around the gesture of the target object.

[0441] The larger the input image size, the more complex the intermediate calculations required for prediction by the gesture recognition algorithm, resulting in a greater computing burden on the electronic device. To reduce this computing burden, the original image size needs to be resized to the target size of the target image, such as to 512*288. Other reasonable sizes are also possible and are not limited here.

[0442] When the first flag is true, it means that the human body recognition algorithm is called to recognize the gesture of the target object contained in the original image. At this time, no matter where the gesture of the target object is located on the left, right, middle, top, bottom, etc. of the original image, a target area of ​​the target size is captured with the gesture of the target object as the center to obtain the target image.

[0443] S44: Recognize the target image based on the gesture recognition algorithm to obtain a gesture recognition result.

[0444] The gesture recognition result includes: the gesture type, the similarity corresponding to the gesture type, and the position coordinates of the gesture in the target image.

[0445] Specifically, the input of the gesture recognition algorithm is a target image, and the target size of the target image is the image size supported by the gesture recognition algorithm. In the embodiment of the present application, the target size is 512*288.

[0446] The output of the gesture recognition algorithm includes: the gesture type, the similarity corresponding to the gesture type, and the gesture's location coordinates in the target image. For example, gesture types may include, but are not limited to, "0"-"9"; the similarity corresponding to each gesture type, for example, the predicted similarity for gesture "ok" (i.e., gesture "0") is 90%; and the gesture's location coordinates in the target image. The gesture's location coordinates in the target image may include, but are not limited to, the height, width, and center point of the gesture box.

[0447] In an embodiment of the present application, an original image of a scene in which an electronic device is located is obtained, and a first flag is obtained, wherein the first flag is used to indicate whether a human body recognition algorithm is called to recognize a gesture of a target object contained in the original image; when the first flag is true, a target area of ​​a target size is captured on the original image to obtain a target image; and the target image is recognized based on the gesture recognition algorithm to obtain a gesture recognition result. Since the target size is an image size supported by the gesture recognition algorithm, and the target area is an area in the original image centered on the gesture of the target object, when the user is far away from the electronic device, the human body recognition algorithm is first called to obtain the gesture of the target object contained in the original image, and then the target area centered on the gesture of the target object is captured on the original image to obtain the target image. This image capture method is to input an image of the original gesture size into the gesture recognition algorithm without compression loss, thereby avoiding the problem of inaccurate gesture recognition caused by compression of the original image in the prior art, further improving the accuracy of gesture recognition.

[0448] FIG61 is a flow chart of another method for remote gesture recognition provided by an embodiment of the present application. This embodiment further expands and optimizes FIG52. Optionally, in some embodiments, if a human body recognition algorithm is called to recognize a gesture of at least one target object contained in the original image, the remote recognition method may include the following steps:

[0449] S6101: Acquire an original image of the scene where the electronic device is located.

[0450] S6102. Obtain the first flag bit.

[0451] The first flag is used to indicate whether the human body recognition algorithm is called to recognize the gesture of the target object contained in the original image.

[0452] In the embodiment of the present application, the implementation method of steps S6101-S6102 is the same as the implementation method of steps S41-S42, and will not be repeated here.

[0453] S6103 . According to the target size, intercept at least one target area of ​​the target size on the original image to obtain at least one target image.

[0454] In some embodiments, considering that when a user performs gesture control on an electronic device, a situation may occur, that is, multiple people perform gesture control on the electronic device. At this time, an original image obtained may include two or three gestures. The human gesture recognition algorithm based on key points recognizes that the original image contains a gesture of at least one target object, crops the image, and captures a target area of ​​the target size centered on the gesture of each target object to obtain at least one target image.

[0455] S6104: Recognize the target image based on the gesture recognition algorithm to obtain a gesture recognition result.

[0456] Furthermore, FIG62 is a flowchart of another long-distance gesture recognition method provided by an embodiment of the present application. This embodiment is a further expansion and optimization based on FIG61. Optionally, in some embodiments, step S6104 (recognizing the target image based on the gesture recognition algorithm to obtain a gesture recognition result) can be implemented as follows:

[0457] recognizing the at least one target image based on the gesture recognition algorithm, and obtaining a similarity corresponding to a gesture type of at least one target object;

[0458] determining a target gesture according to a similarity corresponding to a gesture type of the at least one target object;

[0459] Execute a preset response action corresponding to the target gesture.

[0460] In some embodiments, the long-distance gesture recognition method may also be implemented by the following steps:

[0461] S6201: Acquire an original image of the scene where the electronic device is located.

[0462] S6202. Obtain the first flag bit.

[0463] The first flag is used to indicate whether the human body recognition algorithm is called to recognize the gesture of the target object contained in the original image.

[0464] S6203: According to the target size, intercept at least one target area of ​​the target size on the original image to obtain at least one target image.

[0465] In the embodiment of the present application, the implementation method of steps S6201-S6203 is the same as the implementation method of steps S6101-S6103, and will not be repeated here.

[0466] S6204: Recognize the at least one target image based on the gesture recognition algorithm, and obtain a similarity corresponding to a gesture type of at least one target object.

[0467] S6205: Determine a target gesture according to the similarity corresponding to the gesture type of the at least one target object.

[0468] S6206: Execute a preset response action corresponding to the target gesture.

[0469] The gesture recognition results include the gesture type, the similarity corresponding to the gesture type, and the gesture's position coordinates in the target image. The gesture types of the target objects are ranked by similarity, and the one with the highest similarity is determined as the target gesture. The preset response action corresponding to the target gesture is then executed.

[0470] Assume that an original image contains three gestures. A target area of ​​the target size is captured with the gesture of each target object as the center, resulting in three target images. The three target images are then input into the gesture recognition algorithm to determine the gesture type corresponding to each gesture and the similarity of each gesture type. For example, the gesture type of gesture 1 is "palm," which corresponds to the number "5." This gesture action indicates pausing the current video data, and its output similarity is 92%. The gesture type of gesture 2 is "1," which indicates playing the next video data, and its output similarity is 70%. The gesture type of gesture 3 is "ok," which corresponds to the number "0." This gesture action indicates starting the current video data, and its output similarity is 50%. The similarities of the gesture types of each target object are sorted by size, and the one with the highest similarity is determined to be gesture 1, thus executing the response action of pausing playback.

[0471] In addition, the method of determining the target gesture is not limited to sorting the similarities of the gesture types of each target object by size. The gesture made by the person in the middle of the original image can also be determined as the target gesture, or other reasonable methods are used. No specific restrictions are made here.

[0472] In an embodiment of the present application, an original image of a scene in which an electronic device is located is obtained, and a first flag is obtained, wherein the first flag is used to indicate whether a human body recognition algorithm is called to recognize a gesture of a target object contained in the original image; when the first flag is true, a target area of ​​a target size is captured on the original image to obtain a target image; and the target image is recognized based on the gesture recognition algorithm to obtain a gesture recognition result. Since the target size is an image size supported by the gesture recognition algorithm, and the target area is an area in the original image centered on the gesture of the target object, when the user is far away from the electronic device, the human body recognition algorithm is first called to obtain the gesture of the target object contained in the original image, and then the target area centered on the gesture of the target object is captured on the original image to obtain the target image. This image capture method is to input an image of the original gesture size into the gesture recognition algorithm without compression loss, thereby avoiding the problem of inaccurate gesture recognition caused by compression of the original image in the prior art, further improving the accuracy of gesture recognition.

[0473] FIG63A is a flow chart of another method for remote gesture recognition provided by an embodiment of the present application. This embodiment is a further expansion and optimization based on FIG52. Optionally, in some embodiments, the remote gesture recognition method may further include the following steps:

[0474] S6301: Acquire an original image of the scene where the electronic device is located.

[0475] S6302. Obtain the first flag bit.

[0476] The first flag is used to indicate whether to capture a target area of ​​a target size on the original image, where the target image corresponding to the target area coincides with the center of the original image. The target size is the image size supported by the gesture recognition algorithm.

[0477] S6303: When the first flag is true, cut out a target area of ​​a target size on the original image to obtain a target image.

[0478] S6304: Recognize the target image based on the gesture recognition algorithm to obtain a gesture recognition result.

[0479] Among them, the implementation method of steps S6301-S6304 is the same as the implementation method of the above steps S41-S44, and will not be repeated here.

[0480] S6305: When the first flag is false, perform a first compression process on the original image to obtain a first compressed image.

[0481] The first compression process means compressing the original image according to a first compression ratio to obtain a target image of a target size. In this embodiment, the target size can be, but is not limited to, 512*288.

[0482] For example, if the resolution of the original image is 1920*1080, or 960*640, it can be compressed to a target size of 512*288.

[0483] S6306: Recognize the first compressed image based on the gesture recognition algorithm.

[0484] S6307: Determine whether there is a gesture of the target object in the first compressed image.

[0485] If the gesture of the target object exists in the first compressed image, step S6308 is executed.

[0486] S6308. Set the first flag bit to false.

[0487] If the gesture of the target object does not exist in the first compressed image, step S6309 is executed.

[0488] S6309: Cut out a target area of ​​a target size from the original image to obtain a target image.

[0489] For the original image above, if no gesture is recognized after calling the human body recognition algorithm, it means that the target area may not contain a gesture. Therefore, when processing the next frame of the captured image, you can first use the first compression processing method, that is, compress the original image proportionally to 512*288 pixels, and roughly estimate whether the target area of ​​the next frame of the image contains a gesture. If the target area in the returned result contains the gesture of the target object, the returned result is converted to gesture control and the first flag is set to false, indicating that the image does not need to be intercepted because the gesture has been captured at this time. If the target area in the returned result does not contain a gesture or the similarity of the gesture is low, it means that the target object's gesture was not detected, or the distance between the person and the TV exceeds the effective detection range of 3 meters, then the image needs to be intercepted to obtain the target image.

[0490] By performing a first compression process on the original image to obtain a first compressed image when the first flag is false, and then recognizing the first compressed image, the purpose is to minimize the use of the human body recognition algorithm and reduce the extension of gesture recognition time caused by adding the algorithm.

[0491] Furthermore, in some embodiments, the above step S6309 (cutting off a target area of ​​a target size on the original image to obtain a target image) can be implemented as follows:

[0492] S63091. Call a human body recognition algorithm to recognize the original image.

[0493] S63092: Determine whether the original image contains a gesture of the target object.

[0494] If the original image contains the gesture of the target object, step S63093 is executed.

[0495] S63093. Cut out a target area of ​​a target size from the original image to obtain a target image.

[0496] If the original image does not contain the gesture of the target object, step S63094 is executed.

[0497] S63094. Set the first flag bit to false.

[0498] The implementation of steps S63091-S63093 is the same as that of steps S42-S43, and will not be described in detail here. The purpose of setting the first flag bit to false in step S63094 is to provide a reference for the processing method of the next frame of image.

[0499] FIG63B is a flow chart of another method for remote gesture recognition provided by an embodiment of the present application. This embodiment is a further expansion and optimization based on FIG63A. Optionally, in some embodiments, the remote gesture recognition method may further include the following steps:

[0500] S6301: Acquire an original image of the scene where the electronic device is located.

[0501] S6302. Obtain the first flag bit.

[0502] The first flag is used to indicate whether to capture a target area of ​​a target size on the original image, where the target image corresponding to the target area coincides with the center of the original image. The target size is the image size supported by the gesture recognition algorithm.

[0503] S6303: When the first flag is true, cut out a target area of ​​a target size on the original image to obtain a target image.

[0504] S6304: Recognize the target image based on the gesture recognition algorithm to obtain a gesture recognition result.

[0505] In some embodiments, step S6304 may further include the following steps:

[0506] S63041. When there is a gesture of the target object in the target image, set the first flag to true.

[0507] Since the camera can be set to capture 3-5 images per second, the 3-5 images within 1 second can be considered to have similar movements. That is, when the gesture of the target object exists in the current frame, the first flag is set to true, and the next frame is captured in the same way to obtain the target image. The target image is then input into the gesture recognition algorithm for recognition to obtain the gesture recognition result.

[0508] S63042: When there is no gesture of the target object in the target image, set the first flag to false.

[0509] Similarly, since the camera can be set to take 3-5 images per second, the 3-5 images within 1 second can be considered to have little difference in the actions performed by the person. That is, when there is no gesture of the target object in the current frame, the next frame of image does not need to be intercepted and processed according to the above-mentioned interception method, which can reduce the memory usage of the TV. Therefore, the first flag is set to false.

[0510] S6305: When the first flag is false, perform a first compression process on the original image to obtain a first compressed image.

[0511] The first compression process means compressing the original image according to a first compression ratio to obtain a target image of a target size. In this embodiment, the target size can be, but is not limited to, 512*288.

[0512] For example, if the resolution of the original image is 1920*1080, or 960*640, it can be compressed to a target size of 512*288.

[0513] S6306: Recognize the first compressed image based on the gesture recognition algorithm.

[0514] S6307: Determine whether there is a gesture of the target object in the first compressed image.

[0515] If the gesture of the target object exists in the first compressed image, step S6208 is executed.

[0516] S6308. Set the first flag bit to false.

[0517] If the gesture of the target object does not exist in the first compressed image, step S6209 is executed.

[0518] S6309: Cut out a target area of ​​a target size from the original image to obtain a target image.

[0519] For the original image above, if no gesture is recognized after calling the human body recognition algorithm, it means that the target area may not contain a gesture. Therefore, when processing the next frame of the captured image, you can first use the first compression processing method, that is, compress the original image proportionally to 512*288 pixels, and roughly estimate whether the target area of ​​the next frame of the image contains a gesture. If the target area in the returned result contains the gesture of the target object, the returned result is converted to gesture control and the first flag is set to false, indicating that the image does not need to be intercepted because the gesture has been captured at this time. If the target area in the returned result does not contain a gesture or the similarity of the gesture is low, it means that the target object's gesture was not detected, or the distance between the person and the TV exceeds the effective detection range of 3 meters, then the image needs to be intercepted to obtain the target image.

[0520] Furthermore, in some embodiments, the above step S6309 (cutting off a target area of ​​a target size on the original image to obtain a target image) can be implemented as follows:

[0521] S63091. Call a human body recognition algorithm to recognize the original image.

[0522] S63092: Determine whether the original image contains a gesture of the target object.

[0523] If the original image contains the gesture of the target object, step S63093 is executed.

[0524] S63093. Cut out a target area of ​​a target size from the original image to obtain a target image.

[0525] If the original image does not contain the target object's gesture, then execute step S63094.

[0526] S63094. Set the first flag bit to false.

[0527] In the embodiment of the present application, except for step S6304, the remaining steps are the same as the implementation method in Figure 63A and will not be repeated here.

[0528] In an embodiment of the present application, an original image of a scene in which an electronic device is located is obtained, and a first flag is obtained, wherein the first flag is used to indicate whether a human body recognition algorithm is called to recognize a gesture of a target object contained in the original image; when the first flag is true, a target area of ​​a target size is captured on the original image to obtain a target image; and the target image is recognized based on the gesture recognition algorithm to obtain a gesture recognition result. Since the target size is an image size supported by the gesture recognition algorithm, and the target area is an area in the original image centered on the gesture of the target object, when the user is far away from the electronic device, the human body recognition algorithm is first called to obtain the gesture of the target object contained in the original image, and then the target area centered on the gesture of the target object is captured on the original image to obtain the target image. This image capture method is to input an image of the original gesture size into the gesture recognition algorithm without compression loss, thereby avoiding the problem of inaccurate gesture recognition caused by compression of the original image in the prior art, further improving the accuracy of gesture recognition.

[0529] An embodiment of the present application provides a computer device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the methods described in the embodiments of the present application.

[0530] In some implementations, the present application also provides a computer non-volatile storage medium that can store a program. When the computer non-volatile storage medium is configured in the electronic device 200, the program, when executed, can include the program steps of any of the methods described in the above embodiments configured with the processor 250. The computer non-volatile storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0531] For ease of explanation, the above description has been provided with reference to specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations are possible. The above embodiments have been selected and described to better explain the content of this application, thereby enabling those skilled in the art to better utilize the embodiments.

Claims

1. An electronic device, include: A display for displaying images and / or a user interface; A user interface configured to receive instructions from a user; a communication device configured to communicate with an external device according to a predetermined protocol; a memory configured to store computer instructions and data associated with a display device; An image collector, used for collecting scene images around the display; At least one processor, connected to the display, user interface, communicator and memory, is configured to execute computer instructions to cause the display device to perform: Performing gesture recognition on the scene image collected by the image collector; When it is recognized that the scene image contains a target gesture, comparing the target gesture with a previous gesture, where the previous gesture is a gesture recognized in a previous frame of the scene image; If the target gesture is of the same category as the previous gesture, the target function is matched and executed according to the working state of the electronic device, the execution state of the gesture control function, and the displacement relationship between the target gesture and the previous gesture.

2. The electronic device according to claim 1, before comparing the target gesture with the previous gesture, the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Acquire gesture recognition results, where the gesture recognition results include the number of gestures, the category of the user gestures, and the gesture similarity and gesture position mapped to each user gesture, where the gesture similarity is the similarity between the user gesture and the standard gesture; If the number of gestures is zero, determining that the scene image does not contain the target gesture, and discarding the gesture recognition result; If the number of gestures is greater than zero, the target gesture is searched from multiple user gestures, and the gesture similarity of the target gesture mapping is greater than a first threshold.

3. The electronic device of claim 2, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: After discarding the gesture recognition result, reading the status value of a preset flag, the preset flag is used to indicate the execution status of the gesture control function, the status value includes a first status value and a second status value, the first status value is used to indicate that the gesture function control process corresponding to the previous gesture has been started, and the second status value is used to indicate that the gesture function control process is not currently started; in, The gesture function control process is configured as follows: the display displays a first layer on the user interface, and after the first layer is displayed for a preset time, a gesture control instruction is generated, so that the function module responds to the gesture control instruction and performs a corresponding function; the first layer is used to display the user gesture image, recognition progress and function description information; If the preset flag records the first state value, the gesture function control process corresponding to the previous gesture is stopped, and a second state value is set for the preset flag.

4. The electronic device according to claim 3, wherein the processor matches and executes the target function. include: If the target gesture is of the same category as the previous gesture, reading the state value of the preset flag; If the preset flag bit records the second state value, matching the function mapped by the target gesture to the target function, starting the gesture function control process corresponding to the target gesture, and setting the first state value for the preset flag bit; If the preset flag records the first state value, the displacement between the target gesture and the previous gesture is calculated, and the target function is matched and executed according to the displacement.

5. The electronic device according to claim 4, wherein the processor matches and executes a target function according to the displacement amount. include: If the displacement is equal to zero, the function mapped by the previous gesture is matched to the target function, the gesture function control process corresponding to the previous gesture is continued to be executed, and the gesture function control process corresponding to the target gesture is not started; If the displacement is greater than zero and is not greater than a second threshold, matching the function mapped by the motion gesture to the target function, and continuing to execute the gesture function control process corresponding to the motion gesture; The gesture function control process corresponding to the motion gesture includes: adjusting the function parameter of the target function according to the displacement; If the displacement is greater than a second threshold, the gesture function control process corresponding to the previous gesture is stopped, the function matching of the target gesture mapping to the target function is not started, and a second state value is set for the preset flag.

6. The electronic device of claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: If the target gesture is different from the category of the previous gesture, reading the state value of the preset flag; If the preset flag bit records the second state value, matching the function mapped by the target gesture to the target function, starting the gesture function control process corresponding to the target gesture, and setting the first state value for the preset flag bit; If the preset flag records the first state value, the gesture function control process corresponding to the previous gesture is stopped, and the second state value is set for the preset flag; when it is detected that the state value of the preset flag changes to the second state value, the target gesture is mapped to the second state value. The function of the target gesture is matched as the target function, a gesture function control process corresponding to the target gesture is started, and a first state value is set for the preset flag.

7. The electronic device according to claim 5, wherein the processor executes a gesture function control process corresponding to the motion gesture, include: When a second position gesture is detected, the second position gesture is added to a gesture sequence, wherein the gesture sequence is a sequence consisting of position gestures generated at different positions during a gesture movement of the same category; Calculating a first displacement between a first position gesture and a second position gesture; the first position gesture is a gesture that is located in a sequence before the second position gesture in the gesture sequence, and the first displacement is an absolute value of the displacement between the first position gesture and the second position gesture; If the first displacement is not greater than a third threshold, deleting the second position gesture from the gesture sequence, and discarding the gesture recognition result of the second position gesture; If the first displacement is greater than a third threshold, a function parameter of the target function is adjusted according to the displacement between the first position gesture and the second position gesture.

8. The electronic device according to claim 7, wherein the processor executes a gesture function control process corresponding to the motion gesture, include: Calculating the gesture movement speed according to the first displacement and the preset time length; If the gesture movement speed is not greater than the speed threshold, adjusting the function parameter according to the displacement between the first position gesture and the second position gesture and according to a first adjustment coefficient; If the gesture movement speed is greater than the speed threshold, the function parameter is adjusted according to the displacement between the first position gesture and the second position gesture, and according to a second adjustment coefficient; wherein the second adjustment coefficient is greater than the first adjustment coefficient, the second adjustment coefficient is a preset value, or the second adjustment coefficient is a variable related to the gesture movement speed.

9. The electronic device according to claim 5, wherein the processor executes a gesture function control process corresponding to the motion gesture, include: During the movement of the first gesture, when it is detected that the gesture category is changed to a transition gesture associated with the first gesture, a first function parameter value corresponding to a first position is recorded, where the first position is the final position of the first gesture when the gesture category is changed; Maintaining the gesture function control process corresponding to the first gesture, and not adjusting the function parameters of the first gesture control according to the displacement of the transition gesture; When it is detected that the gesture category switches back from the transition gesture to the first gesture, the function parameter is continuously adjusted based on the first function parameter value and according to the displacement of the first gesture.

10. A gesture control method, include: Acquire the scene image collected by the image collector; performing gesture recognition on the scene image; When it is recognized that the scene image contains a target gesture, comparing the target gesture with a previous gesture, where the previous gesture is a gesture recognized in a previous frame of the scene image; If the target gesture is of the same category as the previous gesture, the target function is matched and executed according to the working state of the electronic device, the execution state of the gesture control function, and the displacement relationship between the target gesture and the previous gesture.

Images