Gesture input method and device, equipment, storage medium and program product

By recognizing hand gestures and mapping them to a virtual keyboard, the problem of inefficient text input on large screens is solved, enabling a fast and convenient input method.

CN118939122APending Publication Date: 2024-11-12HANGZHOU VISION TECHNOLOGY CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411040484.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Large screens make it impossible to input text via touch, resulting in slower input speed and a negative impact on user experience.

Method used

By acquiring the image to be detected, recognizing the user's hand posture and determining the target position, the change trajectory of the hand posture is mapped onto the virtual keyboard to form a key selection trajectory, thereby enabling continuous input.

Benefits of technology

It improves input speed, reduces the number of key presses required for confirmation, and enhances the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939122A_ABST
    Figure CN118939122A_ABST
Patent Text Reader

Abstract

The invention discloses a gesture input method and device, equipment, a storage medium and a program product, and relates to the technical field of interface interaction. Identifying a hand posture of a user in the to-be-detected image, and determining a target position where the hand of the user is located; when the hand posture is the input gesture and the target position of the hand is changed, it is indicated that the user selects the keys in the virtual keyboard by moving the hand, so that the selection track of the user on the corresponding keys of the virtual keyboard can be reflected by mapping the change track of the target position on the virtual keyboard, and the user experience is improved. According to the embodiment of the invention, the content to be input by the user can be determined only based on the selection track, so that the user is prevented from selecting another key after selecting one key by clicking confirmation; the problem that the input speed is reduced due to the fact that characters can only be input into a large-size screen in a single-point mode through a remote controller is solved, and the user experience is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of interface interaction technology, and in particular to gesture input methods, devices, equipment, storage media and program products. Background Art

[0002] Currently, large-size screens (for example, screens larger than 27 inches) are generally unable to input text via touch screens, and text can only be typed in a single-point manner via a remote control. For example, after selecting a button, you need to click to confirm before you can select another button, which reduces the overall input speed and affects the user experience. Summary of the invention

[0003] The main purpose of this application is to provide a gesture input method, device, equipment, storage medium and program product, aiming to solve the technical problem that typing text in a single-point manner through a remote control causes the overall input speed to decrease, affecting the user experience.

[0004] To achieve the above objectives, the present application proposes a gesture input method, the method comprising:

[0005] Acquire the image to be detected;

[0006] Identify the hand posture of the user in the image to be detected, and determine the target position of the user's hand;

[0007] When the hand posture is an input gesture and the target position changes, the change trajectory of the target position is mapped on the virtual keyboard to form a selection trajectory for the corresponding key of the virtual keyboard, and the input content is determined based on the selection trajectory.

[0008] In one embodiment, the step of mapping the change trajectory of the target position on the virtual keyboard to form a selection trajectory of a key corresponding to the virtual keyboard includes:

[0009] Mapping the target position to a first position corresponding to the virtual keyboard;

[0010] Based on the relative distance between the changed position of the hand and the target position, mapping the changed position to a second position corresponding to the virtual keyboard;

[0011] Based on the first position and the second position, a selection track for the corresponding key of the virtual keyboard is formed.

[0012] In one embodiment, the second position includes an end position, and the step of forming a selection track for a key corresponding to the virtual keyboard based on the first position and the second position includes:

[0013] When the hand posture is an end gesture, determining the final position of the hand, mapping the final position with the end position on the virtual keyboard, and using the end position as the end point of the selection track;

[0014] Based on the first position and the end point, a selection trajectory for the corresponding key of the virtual keyboard is formed.

[0015] In one embodiment, the step of determining input content based on the selection trajectory includes:

[0016] Determining whether a key mapped to each target position on the virtual keyboard is a key selected by the user based on the dwell time of the hand at each target position in the change trajectory;

[0017] Based on the key selected by the user, the input content is determined.

[0018] In one embodiment, after the step of determining the target position of the user's hand, the method further includes:

[0019] When the hand posture is a start gesture, the initial position of the hand is determined, the initial position is mapped to a preset position of the virtual keyboard, and the preset position is used as the starting point of the selection track.

[0020] In one embodiment, the preset position includes at least one of the following:

[0021] A cursor displayed on the screen when the hand gesture is a start gesture;

[0022] When the hand posture is a start gesture, the position of the key highlighted in the virtual keyboard is located, and the highlighted key is any key in the virtual keyboard.

[0023] In addition, to achieve the above-mentioned purpose, the present application also proposes a gesture input device, the gesture input device comprising:

[0024] An acquisition module, used for acquiring an image to be detected;

[0025] A determination module, used to identify the hand posture of the user in the image to be detected, and determine the target position of the user's hand;

[0026] The input module is used to map the change trajectory of the target position on the virtual keyboard when the hand posture is an input gesture and the target position changes, to form a selection trajectory for the corresponding key of the virtual keyboard, and to determine the input content based on the selection trajectory.

[0027] In addition, to achieve the above objectives, the present application also proposes a gesture input device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the gesture input method described above.

[0028] In addition, to achieve the above objectives, the present application also proposes a storage medium, which is a computer-readable storage medium, and stores a computer program on the storage medium. When the computer program is executed by a processor, the steps of the gesture input method described above are implemented.

[0029] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the gesture input method described above are implemented.

[0030] One or more technical solutions proposed in this application have at least the following technical effects:

[0031] The present application obtains an image to be detected; identifies the hand posture of the user in the image to be detected, and determines the target position of the user's hand; when the identified hand posture is an input gesture and the target position changes, maps the change trajectory of the target position on the virtual keyboard to form a selection trajectory of the corresponding key of the virtual keyboard; it can be understood that when the hand posture is an input gesture and the target position of the hand changes, it means that the user is selecting the keys in the virtual keyboard by moving his hand. Therefore, by mapping the change trajectory of the target position on the virtual keyboard, the selection trajectory of the user for the corresponding key of the virtual keyboard can be reflected. Only based on the selection trajectory, the content to be input by the user can be determined, avoiding the need for the user to click to confirm after selecting a key before selecting another key; it solves the problem that large-size screens can only type text in a single-point manner through a remote control, resulting in a decrease in input speed, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0034] Figure 1A flowchart of the first embodiment of the gesture input method of the present application is provided;

[0035] Figure 2 This is a schematic diagram of the module structure of the gesture input device according to an embodiment of the present application;

[0036] Figure 3 Schematic diagram of the device structure of the hardware operating environment involved in the gesture input method in the embodiment of the present application.

[0037] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0038] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0039] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0040] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a gesture input device, etc. The following takes the gesture input device as an example to illustrate this embodiment and the following embodiments.

[0041] Based on this, the embodiment of the present application provides a gesture input method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the gesture input method of the present application.

[0042] In this embodiment, the gesture input method includes steps S10 to S30:

[0043] Step S10, obtaining an image to be detected;

[0044] It should be noted that the gesture input device may be subordinate to the gesture input device, and the gesture input device may be a device for controlling the display device, or may be the display device itself.

[0045] Among them, the display device is provided with a display screen, and the display screen can be a large-size screen or a small-size screen (for example, a screen smaller than 27 inches), etc. It can be understood that the gesture input method provided in this embodiment can be applied to any type of screen and is not limited here.

[0046] It should also be noted that the gesture input device is provided with a camera, which can capture images or videos within the viewing angle in real time, or can capture images or videos within the viewing angle when the gesture input device is started.

[0047] It can be understood that the image to be detected can be an image captured by a camera, or a video frame extracted based on a video captured by a camera, etc.

[0048] Step S20, identifying the hand posture of the user in the image to be detected, and determining the target position of the user's hand;

[0049] After acquiring the image to be detected, it is identified whether there is a user in the image to be detected. If there is a user, the user's hand posture is identified (the hand posture can be identified by local magnification), and the user's intention is determined by the change of the hand posture.

[0050] It should be noted that different hand gestures correspond to different control instructions. Specifically, when the gesture input device is started or before the user uses gesture input, the user can be prompted with the control instructions corresponding to different hand gestures so that the user can make corresponding hand gestures based on different intentions.

[0051] Specifically, the hand gesture of the user in the image to be detected may be recognized by a gesture recognition model, and the gesture recognition model may be obtained by iteratively training a preset basic model based on training samples and corresponding gesture labels.

[0052] Among them, recognizing hand posture can be performed by recognizing the bending state of the user's fingers. For example, when the five fingers of one hand of the user are all in a bent state, the intermediate recognition result obtained may be (0,0,0,0,0), and the hand posture recognized based on the intermediate result may be a fist, etc.; when the five fingers of one hand of the user are all in a non-bent state, the intermediate recognition result obtained may be (1,1,1,1,1), and the hand posture recognized based on the intermediate result may be five fingers open, etc.

[0053] In the specific implementation process, the posture recognition model extracts features from the image to be detected, and determines whether the extracted features are hand features based on the similarity threshold. Therefore, the recognition sensitivity can be adjusted by adjusting the similarity threshold to avoid false triggering.

[0054] Since the position of the camera is fixed, when the user's hand is recognized and the position of the hand changes, the relative position change of the user's hand within the same viewing angle can be calculated; therefore, the target position is a relative position and can be a randomly set coordinate.

[0055] It should be noted that the above hand gesture may include a start gesture, that is, a gesture for starting input. For example, the start input gesture may be a gesture of opening five fingers, or may be a gesture of extending preset fingers (for example, a gesture of extending the index finger and the middle finger at the same time). After the step of determining the target position of the user's hand, there are the following possible implementations:

[0056] When the hand posture is a start gesture, the initial position of the hand is determined, the initial position is mapped to a preset position of the virtual keyboard, and the preset position is used as the starting point of the selection track.

[0057] Specifically, when the hand posture is a start gesture, the initial position of the hand can be determined. It can be understood that the initial position can be any position.

[0058] By mapping the initial position with the preset position of the virtual keyboard, the preset position can be used as the starting point of the selection track, that is, the user can start to select a key from the preset position.

[0059] Specifically, when the hand posture is a start gesture, a cursor can be displayed on the display screen. The cursor can be on the virtual keyboard, or on the screen outside the virtual keyboard, etc., and the initial position is mapped to the cursor position, that is, the starting point of the selection trajectory is the cursor position.

[0060] In a feasible embodiment, when the hand posture is a start gesture, any key in the virtual keyboard can be highlighted, and the initial position can be mapped to the position of the highlighted key, that is, the starting point of the selection trajectory is the position of the highlighted key.

[0061] In a feasible embodiment, when the hand gesture is a start gesture, any key in the virtual keyboard may be highlighted, and a cursor may be displayed at the highlighted key at the same time.

[0062] Step S30, when the hand posture is an input gesture and the target position changes, the change trajectory of the target position is mapped on the virtual keyboard to form a selection trajectory for the corresponding key of the virtual keyboard, and the input content is determined based on the selection trajectory.

[0063] It can be understood that when the hand posture is an input gesture, it means that the user is ready to start moving fingers and start selecting keys.

[0064] Therefore, when the hand posture is an input gesture and the target position changes, the change trajectory of the target position can be mapped on the virtual keyboard to map the selection trajectory of the user for the corresponding key on the virtual keyboard.

[0065] Specifically, by identifying the change in the relative position of the hand in the image to be detected, that is, the relative distance between the position after the change and the position before the change can be obtained, and the preset position can be changed based on the relative distance to obtain the changed selection trajectory.

[0066] For example, the position of the cursor may be changed, or the highlighted key may be changed based on the relative distance, thereby forming a selection track for the corresponding key of the virtual keyboard.

[0067] It can be understood that there is a mapping relationship between the change trajectory of the target position and the selection trajectory. Specifically, there is a preset proportional relationship between the relative distance of the target position change and the increased length of the key selection trajectory, and the relative direction of the target position change corresponds to the change direction of the selection trajectory.

[0068] In order to enable the user to see the selection track more clearly, the moving path of the selection track may be displayed on the virtual keyboard.

[0069] Based on the above, when the user's hand moves in an input gesture, the cursor in the display screen moves accordingly or the highlighted keys on the virtual keyboard change accordingly; the user can select keys based on the moving direction and distance of the cursor, or based on the changes in the highlighted keys.

[0070] It should also be noted that when the user's hand moves in an input gesture and gradually forms a selection trajectory for the corresponding keys of the virtual keyboard, the letters corresponding to the keys selected by the user can be displayed synchronously, and / or based on the preset input method, the words, text, etc. composed of the keys selected by the user can be displayed synchronously.

[0071] The input gesture may be a preset gesture when the index finger is in a non-bent state and other fingers are in a bent state.

[0072] It can be understood that users can achieve text input by simply moving their fingers in the air with an input gesture, and users do not need to confirm each key by clicking, but only need to move their fingers continuously to form a corresponding selection trajectory (the input content can be obtained based on the selection trajectory), which improves user convenience and reduces the total distance that the fingers need to move while increasing the input speed.

[0073] The present application obtains an image to be detected; identifies the hand posture of the user in the image to be detected, and determines the target position of the user's hand; when the identified hand posture is an input gesture and the target position changes, maps the change trajectory of the target position on the virtual keyboard to form a selection trajectory of the corresponding key of the virtual keyboard; it can be understood that when the hand posture is an input gesture and the target position of the hand changes, it means that the user is selecting the keys in the virtual keyboard by moving his hand. Therefore, by mapping the change trajectory of the target position on the virtual keyboard, the selection trajectory of the user for the corresponding key of the virtual keyboard can be reflected. Only based on the selection trajectory, the content to be input by the user can be determined, avoiding the need for the user to click to confirm after selecting a key before selecting another key; it solves the problem that large-size screens can only type text in a single-point manner through a remote control, resulting in a decrease in input speed, thereby improving the user experience.

[0074] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above description, and no further description will be given later. In this embodiment, the specific implementation method of mapping the change trajectory of the target position on the virtual keyboard to form the selection trajectory of the corresponding key of the virtual keyboard can be:

[0075] The target position is mapped to the first position corresponding to the virtual keyboard; based on the relative distance between the changed position of the hand and the target position, the changed position is mapped to the second position corresponding to the virtual keyboard; based on the first position and the second position, a selection trajectory for the corresponding key of the virtual keyboard is formed.

[0076] It can be understood that mapping the change track of the target position on the virtual keyboard may be mapping each changed position to a position corresponding to the virtual keyboard during the process in which the target position changes.

[0077] For example, when the target position changes, the cursor position is changed or the highlighted key is changed based on the relative distance between the changed position and the target position and the above mapping relationship, thereby forming a selection track for the corresponding key of the virtual keyboard.

[0078] Specifically, the target position can be mapped to the first position corresponding to the virtual keyboard first, for example, the first position is the position of the letter "R" in the virtual keyboard, wherein the target position can be the above-mentioned initial position or any position after the user moves the finger with an input gesture.

[0079] Furthermore, based on the relative distance between the changed position of the hand and the target position, the changed position can be mapped to a second position corresponding to the virtual keyboard. For example, the second position can be the position of the letter "H" on the virtual keyboard.

[0080] It can be understood that when the target position of the user's hand changes again, the position of the letter "H" in the virtual Chinese keyboard can be used as the target position, and the changed position can be mapped to the new second position corresponding to the virtual keyboard. For example, the new second position can be the position of the letter "M" in the virtual keyboard.

[0081] Based on the above, based on the first position and the second position, a specific implementation method for forming a selection trajectory for the corresponding keys of the virtual keyboard can be: when the second position mapped to the changed position of the user's hand is "H", a selection trajectory from key "R" (first position) to key "H" is formed; when the new second position mapped to the changed position of the user's hand is "M", a selection trajectory from key "R" (first position) to "H" (second position) and then to key "M" is formed.

[0082] Specifically, the second position includes an end position, and a specific implementation of forming a selection track corresponding to a key of the virtual keyboard based on the first position and the second position may be:

[0083] When the hand posture is an end gesture, determine the final position of the hand, map the final position with the end position on the virtual keyboard, and use the end position as the end point of the selection trajectory; based on the first position and the end point, form a selection trajectory for the corresponding key of the virtual keyboard.

[0084] It can be understood that when the hand posture is an end gesture, it is considered that the user has ended the input, and the position of the user's hand at this time is the final position, wherein the end gesture can be the same as the start gesture, or it can be another gesture different from the input gesture, that is, when the user's hand posture changes from an input gesture to an end gesture, the user ends the input.

[0085] Furthermore, by mapping the final position with the end position on the virtual keyboard, the end point of the selection track can be obtained, so that a selection track for the corresponding key of the virtual keyboard is formed based on the first position and the end point.

[0086] Specifically, based on the first position and the end point, a specific implementation method for forming a selection trajectory for the corresponding key of the virtual keyboard may be to form a selection trajectory for the corresponding key of the virtual keyboard based on the starting point, the second position and the end point; wherein the starting point may be the first position, the end point may be a new second position, and so on.

[0087] For example, if the first position is the position of the key "R" and the end position is the position of the key "M", then based on the first position and the end point, the selection trajectory for the corresponding key of the virtual keyboard is from key "R" (first position) to "H" (second position) and then to key "M".

[0088] It should be noted that the specific implementation of determining the input content based on the selection trajectory may be:

[0089] Based on the dwell time of the hand at each target position in the change trajectory, it is determined whether the key mapped to each target position on the virtual keyboard is the key selected by the user; based on the key selected by the user, the input content is determined.

[0090] In the process of the user moving his hand (including fingers) with an input gesture, the residence time of each target position in the change trajectory of the target position can be obtained in time sequence, and based on the residence time, it is determined whether the key mapped to the target position in the change trajectory is the key selected by the user.

[0091] Specifically, when the dwell time exceeds a preset dwell time threshold, it is considered that the key mapped to the target position that exceeds the preset dwell time threshold is the key selected by the user.

[0092] Furthermore, based on the key selected by the user, the content that the user wants to input can be determined.

[0093] Specifically, the specific implementation method of determining whether the keys mapped to each target position on the virtual keyboard are the keys selected by the user based on the residence time of the hand at each target position in the change trajectory may be to determine whether the keys corresponding to the first position and the second position in the selection trajectory are the keys selected by the user based on the residence time of the hand at the target position and the changed position respectively.

[0094] In this embodiment, by mapping the target position of the hand and its changed position to the first position and the second position on the virtual keyboard respectively, the change trajectory of the target position is mapped on the virtual keyboard to form a selection trajectory of the corresponding key of the virtual keyboard; there is no need for the user to confirm each single key separately, and the selection trajectory of the corresponding key of the virtual keyboard can be obtained directly based on the continuous change trajectory of the target position of the hand, so as to directly obtain the input content. It can be understood that in the entire input process, the user only makes one confirmation action, that is, confirms the start of input through the starting gesture when starting the input. After the input starts, the user does not need to confirm each selected key, and the content to be input by the user can be determined based on the selection trajectory, thereby avoiding the need for the user to click to confirm after selecting a key before selecting another key; it solves the problem that large-size screens can only type text in a single-point manner through a remote control, resulting in a decrease in input speed, thereby improving user convenience.

[0095] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the gesture input method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0096] This application also provides a gesture input device, please refer to Figure 2 , the gesture input device comprises:

[0097] An acquisition module 10 is used to acquire an image to be detected;

[0098] A determination module 20, used to identify the hand posture of the user in the image to be detected, and determine the target position of the user's hand;

[0099] The input module 30 is used to map the change trajectory of the target position on the virtual keyboard when the hand posture is an input gesture and the target position changes, to form a selection trajectory for the corresponding key of the virtual keyboard, and to determine the input content based on the selection trajectory.

[0100] The gesture input device provided by the present application adopts the gesture input method in the above embodiment, which can solve the technical problem that the overall input speed is reduced and the user experience is affected by typing text in a single-point manner through a remote control. Compared with the prior art, the beneficial effects of the gesture input device provided by the present application are the same as the beneficial effects of the gesture input method provided by the above embodiment, and the other technical features of the gesture input device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0101] The present application provides a gesture input device, which includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the gesture input method in the above-mentioned embodiment 1.

[0102] Reference below Figure 3 , which shows a schematic diagram of the structure of a gesture input device suitable for implementing the embodiment of the present application. The gesture input device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The gesture input device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0103] like Figure 3 As shown, the gesture input device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the gesture input device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the gesture input device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows a gesture input device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0104] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0105] The gesture input device provided by the present application adopts the gesture input method in the above embodiment, which can solve the technical problem that the overall input speed is reduced and the user experience is affected by typing text in a single-point manner through a remote control. Compared with the prior art, the beneficial effects of the gesture input device provided by the present application are the same as the beneficial effects of the gesture input method provided by the above embodiment, and the other technical features in the gesture input device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0106] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0107] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0108] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the gesture input method in the above-mentioned embodiment.

[0109] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0110] The computer-readable storage medium may be included in the gesture input device; or may exist independently without being assembled into the gesture input device.

[0111] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the gesture input device, the gesture input device is enabled to: execute the gesture input method.

[0112] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0113] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0114] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0115] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned gesture input method, and can solve the technical problem that the overall input speed is reduced and the user experience is affected by typing text in a single-point manner through a remote control. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the gesture input method provided by the above-mentioned embodiment, and will not be repeated here.

[0116] The present application also provides a computer program product, including a computer program, which implements the steps of the gesture input method as described above when executed by a processor.

[0117] The computer program product provided by the present application can solve the technical problem that the overall input speed is reduced and the user experience is affected by typing text in a single-point manner through a remote control. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the gesture input method provided by the above embodiment, and will not be elaborated here.

[0118] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A gesture input method, It is characterized in that The method includes: Acquire the image to be detected; Identify the hand posture of the user in the image to be detected, and determine the target position of the user's hand; When the hand posture is an input gesture and the target position changes, the change trajectory of the target position is mapped on the virtual keyboard to form a selection trajectory for the corresponding key of the virtual keyboard, and the input content is determined based on the selection trajectory.

2. The method according to claim 1, It is characterized in that The step of mapping the change track of the target position on the virtual keyboard to form a selection track of the corresponding key of the virtual keyboard includes: Mapping the target position to a first position corresponding to the virtual keyboard; Based on the relative distance between the changed position of the hand and the target position, mapping the changed position to a second position corresponding to the virtual keyboard; Based on the first position and the second position, a selection track for the corresponding key of the virtual keyboard is formed.

3. The method according to claim 2, It is characterized in that The second position includes an end position, and the step of forming a selection track for a key corresponding to the virtual keyboard based on the first position and the second position includes: When the hand posture is an end gesture, determining the final position of the hand, mapping the final position with the end position on the virtual keyboard, and using the end position as the end point of the selection track; Based on the first position and the end point, a selection trajectory for the corresponding key of the virtual keyboard is formed.

4. The method as claimed in claim 1, It is characterized in that The step of determining input content based on the selection track includes: Determining whether a key mapped to each target position on the virtual keyboard is a key selected by the user based on the dwell time of the hand at each target position in the change trajectory; Based on the key selected by the user, the input content is determined.

5. The method according to claim 1, It is characterized in that After the step of determining the target position of the user's hand, the method further includes: When the hand posture is a start gesture, the initial position of the hand is determined, the initial position is mapped to a preset position of the virtual keyboard, and the preset position is used as the starting point of the selection track.

6. The method according to claim 5, It is characterized in that The preset position includes at least one of the following: A cursor displayed on the screen when the hand gesture is a start gesture; When the hand posture is a start gesture, the position of the key highlighted in the virtual keyboard is located, and the highlighted key is any key in the virtual keyboard.

7. A gesture input device, It is characterized in that The device comprises: An acquisition module, used for acquiring an image to be detected; A determination module, used to identify the hand posture of the user in the image to be detected, and determine the target position of the user's hand; The input module is used to map the change trajectory of the target position on the virtual keyboard when the hand posture is an input gesture and the target position changes, to form a selection trajectory for the corresponding key of the virtual keyboard, and to determine the input content based on the selection trajectory.

8. A gesture input device, It is characterized in that The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the gesture input method according to any one of claims 1 to 6.

9. A storage medium, It is characterized in that The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the gesture input method according to any one of claims 1 to 6 are implemented.

10. A computer program product, It is characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the gesture input method according to any one of claims 1 to 6 are implemented.

Citation Information

Cited By

  • Text selection method and augmented reality system

    CN120469612A