Gesture control method, apparatus, device, storage medium and program product

Abstract

The present disclosure relates to a gesture control method, device, equipment, storage medium and program product, the method comprising: acquiring a first hand image; identifying a hand posture of the first hand image; if the hand posture is a first posture, determining position information of a thumb according to the first hand image, wherein the first posture is a posture of four fingers being clenched and the thumb being straightened; determining that the position of the thumb satisfies a preset condition, and executing a first operation. In the embodiment of the present disclosure, when the palm is in a state of four fingers being clenched and the thumb being straightened, and is near the mouth, the operation of a virtual device is triggered, thereby solving the problem of interaction between a biological hand and a virtual object.

Description

Technical Field

[0001] This disclosure relates to the field of virtual reality technology, and in particular to a gesture control method, device, equipment, storage medium, and program product. Background Technology

[0002] Due to technological advancements and the diversification of market demands, virtual reality systems are becoming increasingly prevalent and are being applied in many fields, such as computer games, health and safety, industry, and education and training.

[0003] Currently, the main virtual reality hardware devices on the market are virtual reality headsets, which immerse users in a virtual world. The primary interaction method with VR headsets is through handheld controllers. These controllers connect to the headset via Bluetooth, transmitting their position and button information to the headset, which then responds accordingly. The drawback of handheld controllers is their reliance on the controller for interaction; when the controller runs out of power, the convenient and reliable interaction method is lost. Furthermore, many controllers are quite heavy and bulky, becoming a burden after prolonged use. VR emphasizes immersion, and using handheld controllers hinders this immersive experience. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides a gesture control method, apparatus, device, storage medium, and program product that triggers operations with a virtual device when the hand is in a fist with four fingers clenched and the thumb extended, and is near the mouth, thus solving the problem of interaction between a biological hand and a virtual object.

[0005] In a first aspect, embodiments of this disclosure provide a gesture control method, the method comprising:

[0006] Obtain the first hand image;

[0007] Identify the hand posture in the first hand image;

[0008] If the hand posture is the first posture, the position information of the thumb is determined according to the first hand image, wherein the first posture is a posture in which the four fingers are clenched into a fist and the thumb is extended.

[0009] Once the position of the thumb meets the preset conditions, the first operation is executed.

[0010] Secondly, embodiments of this disclosure provide a gesture control device, the device comprising:

[0011] The first hand image acquisition module is used to acquire the first hand image;

[0012] A hand pose recognition module is used to recognize the hand pose of the first hand image;

[0013] The position information determination module is used to determine the position information of the thumb based on the first hand image if the hand posture is a first posture, wherein the first posture is a posture in which the four fingers are clenched into a fist and the thumb is extended.

[0014] The first operation execution module is used to determine that the position of the thumb meets the preset conditions and execute the first operation.

[0015] Thirdly, embodiments of this disclosure provide an electronic device, the electronic device comprising:

[0016] One or more processors;

[0017] Storage device for storing one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the gesture control method as described in any of the first aspects above.

[0019] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the gesture control method as described in any one of the first aspects above.

[0020] Fifthly, embodiments of this disclosure provide a computer program product comprising a computer program or instructions that, when executed by a processor, implement the gesture control method as described in any of the first aspects above.

[0021] This disclosure provides a gesture control method, apparatus, device, storage medium, and program product. The method includes: acquiring a first hand image; recognizing the hand posture in the first hand image; if the hand posture is a first posture, determining the position information of the thumb based on the first hand image, wherein the first posture is a posture in which four fingers are clenched into a fist and the thumb is extended; determining that the position of the thumb meets a preset condition, and performing a first operation. In this disclosure, when the palm is in a state of clenched four fingers and extended thumb, and is near the mouth, an operation with a virtual device is triggered, solving the problem of interaction between a biological hand and a virtual object. Attached Figure Description

[0022] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0023] Figure 1 This is a flowchart of a gesture control method according to an embodiment of the present disclosure;

[0024] Figure 2 This is a schematic diagram of the gesture for the first operation in an embodiment of this disclosure;

[0025] Figure 3 This is a schematic diagram of the gesture for the second operation in an embodiment of this disclosure;

[0026] Figure 4 This is a schematic diagram of the structure of a gesture control device according to an embodiment of the present disclosure;

[0027] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0028] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0029] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0030] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0031] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0032] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0033] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0034] The gesture control method proposed in the embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0035] Figure 1 This is a flowchart of a gesture control method according to an embodiment of the present disclosure. This embodiment is applicable to human-computer interaction in virtual reality or augmented reality. The method can be executed by a gesture control device, which can be implemented in software and / or hardware. The gesture control device can be configured in a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device.

[0036] like Figure 1 The gesture control method provided in this embodiment mainly includes the following steps:

[0037] S101. Obtain the first hand image.

[0038] The device acquiring the first hand image can be any one or more of VR, AR, and MR devices; this embodiment does not specifically limit the type of virtual device. Optionally, the device detecting the hand state information is a head-mounted virtual reality device.

[0039] The hand image mentioned here refers to a virtual hand image displayed in a virtual device. The position and movement of the virtual hand are determined by the associated biological hand. Specifically, the depth information of the biological hand is acquired in real time through sensors and a depth camera on the virtual device. The biological hand refers to the real palm of a client in the real world. Specifically, firstly, the positional relationship between the biological hand and the sensors in the virtual device is obtained through the depth camera, including the actual position and virtual coordinates of the sensors. The actual position of the biological hand is calculated based on the actual position of the sensors. Then, based on the known virtual coordinates of the sensors, the actual position of the palm and the actual position of the fingertips can be mapped into the virtual space. This obtains the spatial coordinates of the palm and the fingertips. Since the positional relationships between the joints of the biological hand (hand joint information) are known, the coordinates of each joint in the virtual space can be calculated and derived from the spatial coordinates of the palm and the fingertips, thus filling in and forming the virtual hand, and finally obtaining the spatial coordinates of the entire virtual hand within the VR system.

[0040] S102. Identify the hand posture of the first hand image.

[0041] Here, hand posture refers to the action being performed or the posture being maintained by the palm in the first hand image. For example, the first posture in this embodiment is a posture with four fingers clenched into a fist and the thumb extended.

[0042] In one embodiment of this disclosure, identifying the hand posture of the first hand image includes: identifying the fingertip of the thumb, index finger, middle finger, ring finger, little finger, and palm in the first hand image; and determining the distance between the fingertip of the thumb, index finger, middle finger, ring finger, little finger, and palm.

[0043] In one embodiment of this disclosure, identifying the hand posture of the first hand image further includes: determining the hand posture as the first posture when the distance between the fingertips of the index finger, middle finger, ring finger, and little finger and the palm is less than a first distance threshold, and the distance between the fingertips of the thumb and the palm is greater than a second distance threshold.

[0044] Furthermore, the spatial coordinates of the virtual hand within the VR system are used to determine whether the palm is in a state of four fingers clenched into a fist and the thumb extended.

[0045] The first distance threshold is used to determine whether the four fingers are clenched into a fist. When the hand is normally clenched, the distance between the fingertips of the index, middle, ring, and little fingers and the palm is very close. Considering the different hand sizes of different users, the first distance threshold should have a certain margin. For example, setting the first distance threshold to 2 cm means that the distance between the fingertips of the index, middle, ring, and little fingers and the palm is less than 2 cm to meet the requirement. The relationship between the four is a logical "AND". After the distance between the fingertips and the palm is less than the first distance threshold, it is necessary to determine whether the distance between the thumb tip and the palm tip is greater than the second distance threshold. When the thumb is extended, the distance between the thumb tip and the palm tip is relatively far. Again, considering the different hand sizes of different users, the second distance threshold should not be too large or too small, otherwise it may affect the judgment. For example, setting the second distance threshold to 8 cm.

[0046] The user keeps four fingers clenched into a fist. The distance between the fingertips of the index, middle, ring, and little fingers and the palm is less than the first distance threshold of 2 cm, which meets the requirement of clenching four fingers into a fist. The distance between the fingertips of the thumb and the palm is 10 cm. Assuming the second distance threshold is set to 8 cm, if the distance between the fingertips of the thumb and the palm is greater than the second distance threshold, then the thumb is determined to be in an extended state.

[0047] The distance between the fingertips of the thumb, index finger, middle finger, ring finger, and little finger and the palm indicates the degree of bending of that finger, thereby identifying whether the palm has made a fist. By judging the distance between the fingertips and the palm, the problem of high precision requirements in traditional bare-hand operation is solved, the accuracy of virtual hand positioning is improved, the precision requirements are reduced, and the user's immersive experience is enhanced.

[0048] In one possible implementation, detecting that the palm is in a four-finger fist includes: binding an index fingertip ball to the index finger based on the position information of the index fingertip; binding a middle fingertip ball to the middle finger based on the position information of the middle fingertip; binding a ring fingertip ball to the ring finger based on the position information of the ring fingertip; binding a little fingertip ball to the little finger based on the position information of the little fingertip; and binding a palm ball to the palm based on the position information of the palm. If the middle fingertip ball, the ring fingertip ball, and the little fingertip ball are all tangent to or intersect with the palm ball, then the palm is detected to be in a four-finger fist.

[0049] The aforementioned fingertip ball refers to a spherical model bound to the position information of the fingertip, and the palm ball refers to a spherical model bound to the position information of the palm, wherein the diameter of the palm ball is larger than the diameter of the fingertip ball.

[0050] Furthermore, based on the position information of the virtual middle fingertip, a middle fingertip ball is bound to the virtual middle finger, including: binding the middle fingertip ball to the world coordinates corresponding to the virtual middle fingertip. For example, the center coordinates of the middle fingertip ball can be set to the world coordinates corresponding to the virtual middle fingertip, or the middle fingertip ball can be set at a position tangent to the virtual middle fingertip. The palm ball always moves with the movement of the palm's position information; the fingertip ball always moves with the movement of the fingertip's position information.

[0051] In this embodiment, if the fingertips of the middle finger, ring finger, and little finger are all tangent to or intersect with the palm ball, it indicates that the middle finger, ring finger, and little finger are bent and close to the palm. At this time, it is determined that the palm is in a four-finger fist.

[0052] If any one of the fingertips of the middle finger, ring finger, or little finger is separated from the palm, it indicates that one finger is less bent. In this case, it is determined that the palm does not meet the requirement of a four-finger fist.

[0053] In this embodiment, the distance between the fingertip and the palm can be determined by judging the positional relationship between the palm ball and the fingertip ball, and then the degree of finger bending can be judged to identify whether the whole hand has made a fist. This method of judging the state of the hand by binding small balls solves the problem of high precision requirements in traditional bare hand operation, improves the stability of virtual hand positioning, and reduces the accuracy requirements, thereby improving the user's immersive experience.

[0054] S103. If the hand posture is a first posture, determine the position information of the thumb based on the first hand image, wherein the first posture is a posture in which the four fingers are clenched into a fist and the thumb is extended.

[0055] The thumb's position information can include the position of the thumb's fingertip, and the thumb's normal line is constructed by the position of the thumb's fingertip and the position of the palm.

[0056] Specifically, the thumb normal originates from the center of the palm and extends towards the tip of the thumb, which is the direction the thumb points in on the spatial coordinate system. Establishing the thumb normal provides support for subsequent operations.

[0057] S104. Determine that the position of the thumb meets the preset conditions, and execute the first operation.

[0058] Specifically, the thumb must be within a predetermined range centered on the mouth to meet the preset conditions. Within this range, the detection device can detect hand posture. Detection of hand position and posture is achieved by detecting the fingertips and palm of the hand. If the hand is outside the preset range, it cannot be detected, and further operation cannot proceed. This disclosure does not further define the preset range; it is sufficient to be in front of the face centered on the mouth. A larger or smaller preset range can be set as needed.

[0059] In one possible implementation, determining that the position of the thumb meets a preset condition includes: pre-setting a collision area at the user's head position; constructing a ray along the direction of the thumb tip in an extended state; and determining that the preset condition is met when the ray intersects with the collision area.

[0060] The collision area can be set on the plane of the user's forehead, or on the plane of the wearable device's glasses.

[0061] Furthermore, the process of constructing the thumb normal includes: taking the center of the palm as the starting point and the direction of the thumb tip as the ray direction, constructing the thumb ray.

[0062] Specifically, when the thumb ray intersects the plane of the collision area, the preset conditions are successfully met, such as... Figure 2 As shown, the ray originates from the tip of the thumb and passes through the collision area at an angle. It should be noted that the thumb normal is a ray, and the preset collision area is a circular surface. The thumb is satisfied if the thumb normal can pass through the collision area. The thumb is within the preset range if the thumb normal has an angle with the circular surface of the collision area but still passes through the collision area. Even if it does not pass through perpendicularly, it is still within the preset range and still meets the requirements.

[0063] The first operation can be set by the user. In this embodiment, the first operation is voice input. The user can also set other functions. This disclosure does not make any specific limitations.

[0064] In one possible implementation, performing the first operation includes: turning on the microphone; and acquiring audio signals through the microphone.

[0065] Specifically, such as Figure 2 As shown, when the index, middle, ring, and little fingers are clenched into a fist and the thumb is extended, and the thumb ray intersects with the collision area, the microphone is activated and the user's voice signal is collected. At this time, the wearable device activates the voice input function, and subsequent operations such as microphone activation and audio acquisition proceed.

[0066] In one possible implementation, the process of performing the first operation further includes: acquiring a second hand image at a preset period; determining the positional relationship between the thumb and index finger based on the second hand image; and performing the second operation when it is determined that the positional relationship between the thumb and index finger is that the thumb and index finger are in contact.

[0067] The preset period can be set according to the actual situation, generally a few milliseconds or a fraction of a second, but it is not specifically limited in this embodiment.

[0068] Specifically, the positional change between the thumb and index finger is determined by the distance between the tips of the thumb and index finger. A third distance threshold is set. If the distance between the tips of the thumb and index finger is less than the third distance threshold, it is determined that the thumb and index finger are in contact, and the second operation is performed. Assuming the third distance threshold is set to 1 cm, ... Figure 2 As shown, the distance between the tip of the thumb and the tip of the index finger is too far, exceeding the third distance threshold, so the second operation is not performed; Figure 3 As shown, if the distance between the tips of the thumb and index finger is less than 2 centimeters, which is less than the third distance threshold, then the second operation is performed.

[0069] Furthermore, a thumb ball is bound to the tip of the thumb, and a square index finger is bound to the virtual index finger. The thumb ball moves along with the position information of the virtual thumb. The square index finger can be a cuboid that covers the entire finger and moves along with the position information of the virtual index finger.

[0070] In this embodiment, if the ball of the thumb is tangent to or intersects with the square body of the index finger, it indicates that the thumb is bent and the thumb tip is in contact with the index finger, triggering the second operation. If the ball of the thumb is not tangent to or intersects with the square body of the index finger, that is, they are in a separated state, then the thumb tip is not in contact with the index finger, and the second operation is not triggered.

[0071] In this embodiment, the degree of bending of the thumb is determined by judging the positional relationship between the ball at the tip of the thumb and the square body of the index finger, so as to identify whether the thumb is in contact with the index finger. This method of judging the state of the hand by binding a small ball solves the problem of high precision requirements in traditional bare-hand operation, improves the stability of virtual hand positioning, and reduces the accuracy requirements, thereby improving the user's immersive experience.

[0072] In one possible implementation, performing the second operation includes: sending and / or saving the audio signal. The second operation may be sending the audio input from the first operation or saving the audio input from the first operation; this disclosure does not impose specific limitations on the embodiments.

[0073] Specifically, in this embodiment, the first operation is voice input, and the second operation is to end the voice input and proceed with the audio to the next operation, namely, sending or saving. These two operations complement each other, improving the user experience while resolving the issues associated with virtual gesture interaction.

[0074] Figure 4 This is a schematic diagram of the structure of a human-computer gesture control device according to an embodiment of the present disclosure. This embodiment is applicable to human-computer interaction in virtual reality or augmented reality. The gesture control device can be implemented by software and / or hardware. The gesture control device can be configured in a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device.

[0075] like Figure 4 The gesture control device 40 provided in this embodiment mainly includes: a first hand image acquisition module 41, a hand posture recognition module 42, a position information determination module 43, and a first operation execution module 44.

[0076] The first hand image acquisition module 41 is used to acquire a first hand image;

[0077] The hand pose recognition module 42 is used to recognize the hand pose of the first hand image;

[0078] The position information determination module 43 is used to determine the position information of the thumb based on the first hand image if the hand posture is a first posture, wherein the first posture is a posture in which the four fingers are clenched into a fist and the thumb is extended.

[0079] The first operation execution module 44 is used to determine that the position of the thumb meets the preset conditions and execute the first operation.

[0080] In one possible implementation, the hand posture recognition module 42 includes: a fingertip and palm recognition unit for recognizing the fingertip of the thumb, index finger, middle finger, ring finger, little finger and palm in the first hand image; and a distance determination unit for determining the distance between the fingertip of the thumb, index finger, middle finger, ring finger and little finger and the palm.

[0081] In one possible implementation, the hand posture recognition module 42 further includes: a first posture determination unit, used to determine the hand posture as a first posture when the distance between the fingertips of the index finger, middle finger, ring finger, and little finger and the palm is less than a first distance threshold, and the distance between the fingertips of the thumb and the palm is greater than a second distance threshold.

[0082] In one possible implementation, determining that the position of the thumb meets a preset condition includes: pre-setting a collision area at the user's head position; constructing a ray along the direction of the thumb tip in an extended state; and determining that the preset condition is met when the ray intersects with the collision area.

[0083] In one possible implementation, the first operation execution module 44 includes: a microphone switch unit for turning on the microphone; and an audio acquisition unit for acquiring audio signals through the microphone.

[0084] In one possible implementation, the device further includes: a first hand image acquisition module, configured to acquire a second hand image at a preset period during the execution of the first operation; a finger position relationship acquisition module, configured to determine the positional relationship between the thumb and index finger based on the second hand image; and a second operation execution module, configured to execute a second operation when it is determined that the positional relationship between the thumb and index finger is that the thumb and index finger are in contact.

[0085] In one possible implementation, the second operation execution module is specifically used to send and / or save the audio signal.

[0086] The gesture control device provided in this disclosure can execute the steps performed in the gesture control method provided in this disclosure, and has the execution steps and beneficial effects, which will not be repeated here.

[0087] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. See below for details. Figure 5 The diagram illustrates a structural schematic suitable for implementing the electronic device 500 in the embodiments of this disclosure. The electronic device 500 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable terminal devices, etc., as well as fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Figure 5 The terminal device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0088] like Figure 5 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503 to implement the image rendering method as described in the embodiments of this disclosure. The RAM 503 also stores various programs and data required for the operation of the terminal device 500. The processing device 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0089] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows terminal device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 A terminal device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0090] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the gesture control method as described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0091] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0092] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0093] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0094] The aforementioned computer-readable medium carries one or more programs, which, when executed by the terminal device, cause the terminal device to: acquire a first hand image; identify the hand posture in the first hand image; if the hand posture is a first posture, determine the position information of the thumb based on the first hand image, wherein the first posture is a posture in which four fingers are clenched into a fist and the thumb is extended; determine that the position of the thumb meets a preset condition, and perform a first operation.

[0095] Optionally, when one or more of the above-mentioned programs are executed by the terminal device, the terminal device may also execute other steps described in the above embodiments.

[0096] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0097] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0098] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0099] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0100] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0101] According to one or more embodiments of this disclosure, this disclosure provides a gesture control method, including: acquiring a first hand image; recognizing the hand posture of the first hand image; if the hand posture is a first posture, determining the position information of the thumb based on the first hand image, wherein the first posture is a posture in which four fingers are clenched into a fist and the thumb is extended; determining that the position of the thumb meets a preset condition, and performing a first operation.

[0102] According to one or more embodiments of this disclosure, this disclosure provides a gesture control method, wherein recognizing the hand posture of the first hand image includes: recognizing the fingertip of the thumb, index finger, middle finger, ring finger, little finger and palm in the first hand image; and determining the distance between the fingertip of the thumb, index finger, middle finger, ring finger and little finger and the palm.

[0103] According to one or more embodiments of this disclosure, this disclosure provides a gesture control method, wherein recognizing the hand posture of the first hand image further includes: determining the hand posture as a first posture when the distance between the fingertips of the index finger, middle finger, ring finger, and little finger and the palm is less than a first distance threshold, and the distance between the fingertips of the thumb and the palm is greater than a second distance threshold.

[0104] According to one or more embodiments of this disclosure, this disclosure provides a gesture control method, wherein determining that the position of the thumb satisfies a preset condition includes: pre-setting a collision area at the user's head position; constructing a ray along the fingertip direction of the extended thumb; and determining that the preset condition is satisfied when the ray intersects with the collision area.

[0105] According to one or more embodiments of this disclosure, this disclosure provides a gesture control method, wherein a first operation includes: turning on a microphone; and acquiring audio signals through the microphone.

[0106] According to one or more embodiments of this disclosure, this disclosure provides a gesture control method, wherein, during the execution of a first operation, the method further includes: acquiring a second hand image at a preset period; determining the positional relationship between the thumb and index finger based on the second hand image; and executing a second operation when the positional relationship between the thumb and index finger is determined to be that the thumb and index finger are in contact.

[0107] According to one or more embodiments of this disclosure, this disclosure provides a gesture control method, wherein the second operation includes: sending and / or saving the audio signal.

[0108] According to one or more embodiments of this disclosure, a gesture control device is provided, comprising: a first hand image acquisition module for acquiring a first hand image; a hand posture recognition module for recognizing the hand posture in the first hand image; a position information determination module for determining the position information of a thumb based on the first hand image if the hand posture is a first posture, wherein the first posture is a posture in which four fingers are clenched into a fist and the thumb is extended; and a first operation execution module for determining that the position of the thumb meets a preset condition and executing a first operation.

[0109] According to one or more embodiments of this disclosure, this disclosure provides a gesture control device, wherein, in one possible implementation, the hand posture recognition module includes: a fingertip and palm recognition unit for recognizing the fingertip of the thumb, index finger, middle finger, ring finger, little finger, and palm in the first hand image; and a distance determination unit for determining the distance between the fingertip of the thumb, index finger, middle finger, ring finger, and little finger and the palm.

[0110] According to one or more embodiments of this disclosure, this disclosure provides a gesture control device, wherein the hand posture recognition module further includes: a first posture determination unit, used to determine the hand posture as a first posture when the distance between the fingertip of the index finger, the fingertip of the middle finger, the fingertip of the ring finger, the fingertip of the little finger and the palm is less than a first distance threshold, and the distance between the fingertip of the thumb and the palm is greater than a second distance threshold.

[0111] According to one or more embodiments of this disclosure, this disclosure provides a gesture control device, wherein determining that the position of the thumb meets a preset condition includes: pre-setting a collision area at the user's head position; constructing a ray in the direction of the fingertip of the extended thumb; and determining that the preset condition is met when the ray intersects with the collision area.

[0112] According to one or more embodiments of this disclosure, this disclosure provides a gesture control device, wherein a first operation execution module includes: a microphone switch unit for turning on a microphone; and an audio acquisition unit for acquiring audio signals through the microphone.

[0113] According to one or more embodiments of this disclosure, this disclosure provides a gesture control device, wherein the device further includes: a first hand image acquisition module, configured to acquire a second hand image at a preset period during the execution of a first operation; a finger position relationship acquisition module, configured to determine the positional relationship between the thumb and index finger based on the second hand image; and a second operation execution module, configured to execute a second operation when it is determined that the positional relationship between the thumb and index finger is that the thumb and index finger are in contact.

[0114] According to one or more embodiments of this disclosure, this disclosure provides a gesture control device, wherein a second operation execution module is specifically used to send and / or save the audio signal.

[0115] According to one or more embodiments of this disclosure, this disclosure provides an electronic device, including:

[0116] One or more processors;

[0117] Memory, used to store one or more programs;

[0118] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the gesture control methods provided in this disclosure.

[0119] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements a gesture control method as described in any of the present disclosure.

[0120] This disclosure also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the gesture control method described above.

[0121] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0122] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0123] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A gesture control method, characterized in that, The method includes: Obtain the first hand image; Identify the hand posture in the first hand image; If the hand posture is the first posture, the position information of the thumb is determined according to the first hand image, wherein the first posture is a posture in which the four fingers are clenched into a fist and the thumb is extended. Once the position of the thumb meets the preset conditions, execute the first operation; Determining that the position of the thumb meets preset conditions includes: Pre-define the collision zone at the user's head position; Starting from the center of the palm, draw a ray with the tip of the extended thumb pointing in the direction of the ray. When the ray intersects with the collision region, the preset condition is determined to be met.

2. The method according to claim 1, characterized in that, Recognizing the hand pose of the first hand image includes: Identify the tips of the thumb, index finger, middle finger, ring finger, little finger, and palm in the first hand image; Determine the distance between the tips of your thumb, index finger, middle finger, ring finger, and little finger and the center of your palm.

3. The method according to claim 2, characterized in that, Recognizing the hand pose of the first hand image further includes: When the distance between the fingertips of the index, middle, ring, and little fingers and the palm is less than a first distance threshold, and the distance between the fingertips of the thumb and the palm is greater than a second distance threshold, the hand posture is determined to be the first posture.

4. The method according to claim 1, characterized in that, The first operation includes: Turn on your microphone; Audio signals are collected through the microphone.

5. The method according to claim 4, characterized in that, The process of performing the first operation also includes: Acquire a second hand image at a preset cycle; Based on the second hand image, determine the positional relationship between the thumb and index finger; When it is determined that the positional relationship between the thumb and index finger is such that the thumb and index finger are in contact, the second operation is performed.

6. The method according to claim 5, characterized in that, The second operation includes: sending and / or saving the audio signal.

7. A gesture control device, characterized in that, The device includes: The first hand image acquisition module is used to acquire the first hand image; A hand pose recognition module is used to recognize the hand pose of the first hand image; The position information determination module is used to determine the position information of the thumb based on the first hand image if the hand posture is a first posture, wherein the first posture is a posture in which the four fingers are clenched into a fist and the thumb is extended. The first operation execution module is used to determine that the position of the thumb meets the preset conditions and execute the first operation; The first operation execution module is specifically used for: Pre-define the collision zone at the user's head position; Starting from the center of the palm, draw a ray with the tip of the extended thumb as the ray direction. When the ray intersects with the collision region, the preset condition is determined to be met.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

10. A computer program product comprising a computer program or instructions that, when executed by a processor, implement the method as described in any one of claims 1-6.

Images