Information input methods, devices, electronic devices and readable storage media

By obtaining finger posture to determine input information, the problem of low efficiency of external input devices is solved, realizing a convenient information input method that is suitable for terminal devices such as mobile phones and VR/AR/MR devices.

CN117331429BActive Publication Date: 2026-11-17BEIJING XIAOMI MOBILE SOFTWARE CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202210725365.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-23
Publication Date
2026-11-17
Estimated Expiration
2042-06-23

AI Technical Summary

Technical Problem

In existing technologies, using external input devices such as keyboards or controllers to assist in inputting information is inefficient and inconvenient to carry, especially in virtual reality devices where the buttons are small and difficult to aim at, and the input devices are either unseen or difficult to see.

Method used

By acquiring the extension and bending posture of the user's fingers, the input information is determined directly using the finger posture as input information without the need for external special input devices or controllers.

Benefits of technology

It enables convenient information input without the need for external devices, improving input efficiency and portability, and simplifying user operation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117331429B_ABST
    Figure CN117331429B_ABST
Patent Text Reader

Abstract

This disclosure relates to an information input method, apparatus, and readable storage medium. The method includes: acquiring a finger posture in an image to be detected, wherein the finger posture is a posture formed by the extension and / or bending of the fingers; determining input information based on the finger posture; and using the input information as input information for an electronic device. The method is capable of acquiring an image containing finger postures, determining the finger postures in the image, assigning different information to different finger postures, and thus determining input information based on the finger postures and using the input information as input information for an electronic device. It eliminates the need for external dedicated input devices or controller-assisted input, facilitating user information input.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to image processing, and more particularly to an information input method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] In related technologies, some electronic device applications require external input devices or controllers to assist with input, such as keyboards or gamepads. However, using controllers / gamepads on virtual keyboards is inefficient, and some controllers (such as the gamepads commonly found in virtual reality headsets) have small keys that are difficult to aim at and require space, making them inconvenient for use outdoors. Using external input devices like keyboards requires users to carry them, which is inconvenient, and the input devices are often difficult or impossible to see in the input environment. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides an information input method, apparatus, electronic device, and readable storage medium.

[0004] According to a first aspect of the present disclosure, an information input method is provided, comprising acquiring a finger posture in an image to be detected; the finger posture being a posture formed by the extension and / or bending of a finger; determining input information based on the finger posture; and using the input information as input information for the electronic device.

[0005] Optionally, the finger posture includes: a posture formed by a static finger and a finger with at least one knuckle presenting a set bending angle; wherein, the static state means that the finger is stationary relative to the palm, or the rate of change of the bending angle presented at the knuckle is less than the set rate of change.

[0006] Optionally, the posture formed by a static finger and at least one knuckle presenting a predetermined bending angle includes: a static first finger, a static second finger, and a middle knuckle of the non-static second finger presenting predetermined bending angles within different angle ranges; the middle knuckle being the second knuckle of the second finger with its fingertip pointing towards the palm; and / or, a static second finger and the front knuckle of the first finger presenting predetermined bending angles within different angle ranges; the front knuckle being the first knuckle of the first finger with its fingertip pointing towards the palm; wherein the first finger is the thumb, and the second finger is any finger other than the thumb.

[0007] Optionally, determining the input information based on the finger posture includes: determining the target area selected by the user based on the finger posture; the target area is one of the areas divided on the user's fingers and / or palm; and determining the input information corresponding to the target area based on the correspondence between the areas divided on the user's fingers and / or palm and different input information.

[0008] Optionally, the areas divided on the user's fingers and / or palm include at least one of the following: the area where the finger joints are located, the area between two finger joints, the area where the fingertip is located, the area between the fingertip and the finger joint, the area in the palm near the base of the fingers, the area divided at the gap between the fingers when they are together, and the area divided inside the palm.

[0009] Optionally, obtaining the finger pose in the image to be detected includes: after detecting the user's hand, displaying input information corresponding to different regions in the regions divided on the user's fingers and / or palm; after detecting the user's hand forming the finger pose, obtaining an image of the user's hand as the image to be detected.

[0010] Optionally, before displaying the input information corresponding to different areas in the areas divided on the user's fingers and / or palm after detecting the user's hand, the method further includes: after detecting the user's hand, if a trigger gesture is detected, then displaying the input information corresponding to different areas in the areas divided on the user's fingers and / or palm; wherein the trigger gesture is used to indicate the start of determining input information through finger posture.

[0011] Optionally, determining the input information corresponding to the target area based on the correspondence between regions divided on the user's fingers and / or palm and different input information includes: if the input information corresponding to the target area is one type, then determining the input information corresponding to the target area according to the correspondence; if the input information corresponding to the target area is multiple types, then displaying the multiple types of input information corresponding to the target area according to the correspondence; and determining the input information selected by the user based on the selection operation performed by the user.

[0012] Optionally, the step of displaying multiple input information corresponding to the target area according to the correspondence, and determining the input information selected by the user based on the selection operation performed by the user, includes: displaying different input information corresponding to the target area in different selection areas according to the correspondence; detecting the selection area touched by the user, and using the input information corresponding to the selection area touched by the user as the input information selected by the user; wherein, the selection area is an area in the hand and / or space that does not overlap with the area divided on the user's fingers and / or palm.

[0013] According to a second aspect of the present disclosure, an information input device is provided, comprising: an acquisition module configured to acquire a finger posture in an image to be detected; the finger posture being a posture formed by the extension and / or bending of a finger; a processing module configured to determine input information based on the finger posture; and a determination module configured to use the input information as input information for the electronic device.

[0014] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the steps of the aforementioned information input method.

[0015] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the information input method provided in the first aspect of the present disclosure.

[0016] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: acquiring finger postures in an image to be detected, where finger postures are postures formed by fingers extending and / or bending, determining input information based on finger postures, and using the input information as input information for an electronic device; being able to acquire images containing finger postures, determine the finger postures in the images, assign different information to different finger postures, thus determining input information based on finger postures and using the input information as input information for an electronic device; eliminating the need for external dedicated input devices or controller-assisted input, thus facilitating user input information.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0019] Figure 1 This is a flowchart illustrating an information input method according to an exemplary embodiment of this disclosure.

[0020] Figure 2 This is a schematic diagram of key points of the palm shown in an exemplary embodiment of this disclosure.

[0021] Figure 3 This is a flowchart illustrating sub-step S11 as shown in an exemplary embodiment of this disclosure.

[0022] Figure 4 This is a schematic diagram illustrating the division of a hand region according to an exemplary embodiment of this disclosure.

[0023] Figure 5 This is a diagram illustrating the correspondence between a hand area and input information, as shown in an exemplary embodiment of this disclosure.

[0024] Figure 6 This is a flowchart illustrating sub-step S12 as shown in an exemplary embodiment of this disclosure.

[0025] Figure 7 This is a block diagram illustrating an information input device according to an exemplary embodiment.

[0026] Figure 8 This is a block diagram illustrating an information input device according to an exemplary embodiment. Detailed Implementation

[0027] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0028] It should be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.

[0029] This exemplary embodiment discloses an information input method for use in terminals such as mobile phones, smart glasses, virtual reality (VR) devices, augmented reality (AR) devices, and mixed reality (MR) devices. The method includes: acquiring finger postures from a target image, where finger postures are the postures formed by the extension and / or bending of the fingers. For example, a target image containing the user's palm can be acquired in real time using a camera installed on the terminal, and the finger postures can be obtained using image detection technology; determining input information based on the finger postures, where different information can be pre-assigned to different finger postures, and the information represented by the finger postures can be used as input information; and using the input information as input information for the electronic device. This method can acquire images containing finger postures, determine the finger postures in the images, and assign different information to different finger postures. Therefore, input information can be determined based on the finger postures and used as input information for the electronic device, eliminating the need for external dedicated input devices or controller-assisted input, thus facilitating user information input.

[0030] Figure 1This is a flowchart illustrating an information input method according to an exemplary embodiment, such as... Figure 1 As shown, the information input method is used in terminals such as mobile phones, smart glasses, VR / AR / MR devices, etc., and includes the following steps:

[0031] In step S11, the finger pose in the image to be detected is obtained.

[0032] It should be noted that finger gestures are the postures formed by extending and / or bending the fingers. They can be any specified shape formed by the fingers, such as using the "OK" gesture to indicate "confirm" information input.

[0033] Furthermore, the finger posture includes a posture formed by a static finger and a finger with at least one knuckle presenting a set bending angle; wherein, the static state means that the finger is stationary relative to the palm, or the rate of change of the bending angle presented at the knuckle is less than the set rate of change.

[0034] Finger postures can be formed by the static fingers and the fingers with knuckles exhibiting a predetermined bending angle. The same static fingers, but different predetermined bending angles in the non-static fingers, indicate different finger postures. For example, the finger posture when the thumb, middle finger, ring finger, and little finger are stationary, and the index finger exhibits a predetermined bending angle of 45°, is different from the finger posture when the index finger exhibits a predetermined bending angle of 145°. Alternatively, the same predetermined bending angle can be present in the same fingers, but different static fingers will indicate different gestures. For example, the finger posture when the index finger exhibits a predetermined bending angle of 45° while the other fingers are stationary is different from the finger posture when the ring finger exhibits a predetermined bending angle of 45° while the other fingers are stationary. Or, the same static fingers exhibiting the same predetermined bending angle at different knuckles will indicate different gestures. For example, the finger posture when the index finger's middle knuckle exhibits a predetermined bending angle of 45° while the index finger's front knuckle exhibits a predetermined bending angle of 45° is different from the finger posture when the index finger's front knuckle exhibits a predetermined bending angle of 45°.

[0035] Knuckles refer to the joints of the fingers. Please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram illustrating key points of the palm according to an exemplary embodiment. Figure 2The diagram shows a total of 20 key points. Key points 1-4 are key points on the thumb, key point 4 is the tip of the thumb, and key points 2-3 are the knuckles of the thumb; key points 5-8 are key points on the index finger, key point 8 is the tip of the index finger, and key points 5-7 are the knuckles of the index finger; key points 9-12 are key points on the middle finger, key point 12 is the tip of the middle finger, and key points 9-11 are the knuckles of the middle finger; key points 13-16 are key points on the ring finger, key point 16 is the tip of the ring finger, and key points 13-16 are the knuckles of the ring finger; key points 17-20 are key points on the little finger, key point 20 is the tip of the little finger, and key points 17-19 are the knuckles of the little finger; key point 0 is the bottom of the palm.

[0036] The bending angle of the knuckles refers to the angle formed by the segments of the finger connecting the knuckles, with the knuckles as the apex. A finger segment refers to the segment of the finger that is divided by the knuckles. For example... Figure 2 As shown, the middle position is where 6 is located. The bending angle at the middle knuckle refers to the bending angle formed by taking 6 as the vertex, the finger segment between 6 and 7 as the edge, and the finger segment between 6 and 5 as the other edge.

[0037] Static states can be, for example, the fingers naturally extended, naturally bent, or naturally extended with slight tremors. For instance, when the fingers are naturally bent, there may sometimes be slight tremors. If the angle change of the fingers during these tremors is less than a predetermined range, it is considered that the requirements for forming different hand gestures have not been met, and this state is considered static. Taking the index finger as an example, the fingertip moves within a predetermined range, and the angle change of the index finger's knuckle is less than a preset threshold. In one implementation, this predetermined range can be 0.5 cm, and the preset threshold can be 15°.

[0038] The posture formed by a static finger and at least one phalanx of the finger presenting a predetermined bending angle includes: a static first finger, a static second finger, and a middle phalanx of the non-static second finger presenting predetermined bending angles within different angle ranges; the middle phalanx is the second phalanx of the second finger with the fingertip pointing towards the palm; and / or, a static second finger and the front phalanx of the first finger presenting predetermined bending angles within different angle ranges; the front phalanx is the first phalanx of the first finger with the fingertip pointing towards the palm; wherein, the first finger is the thumb, and the second finger is any finger other than the thumb.

[0039] Taking the input of information by using a second finger each time as an example, when the information is input by the index finger, the thumb, middle finger, ring finger and little finger are all in a static state. Different information is input by using different bending angles of the middle joint of the index finger.

[0040] Taking information input via the thumb as an example, when inputting information via the thumb, the index, middle, ring, and little fingers are all in a static position. Different information is input by varying the bending angle of the thumb's proximal knuckle. By determining the input information from the finger posture in the image to be detected, no external input device or controller is required, making it more convenient for users to input information.

[0041] The angle range includes less than or equal to 90°, greater than 90° and less than 145°, or greater than or equal to 145°. Different angle ranges correspond to different gestures.

[0042] As Figure 2 Taking the index and middle fingers as an example, key point 6 is the middle joint of the index finger. The first finger is the thumb, and the second finger is any finger other than the thumb. For example, the angle range of the middle joint can be less than or equal to 90°, greater than 90° and less than 145°, or greater than or equal to 145°. The middle joint of a human finger is more flexible than the front joints, and its angle is more pronounced with a wider range, thus facilitating angle detection and improving its accuracy.

[0043] It should be noted that step S11 also includes sub-steps S111, S112, and S113. The specific method for obtaining the finger pose in the image to be detected will be described in detail in the sub-steps of step S11. Please refer to [link / reference]. Figure 3 , Figure 3 This is a flowchart illustrating sub-step S11 as shown in an exemplary embodiment of this disclosure.

[0044] An exemplary embodiment of an information input method is provided, wherein determining input information based on the finger posture includes:

[0045] The target area selected by the user is determined based on the finger posture; the target area is one of the regions divided on the user's fingers and / or palm.

[0046] Based on the correspondence between regions defined on the user's fingers and / or palms and different input information, the input information corresponding to the target region is determined.

[0047] To simplify the input method and facilitate the quick determination of input information corresponding to different finger postures, a correspondence between finger postures and different areas of the hand can be established. Then, based on the correspondence between different areas of the hand and different input information, the input information selected by the user can be determined.

[0048] The areas defined on the user's fingers and / or palm include at least one of the following: the area where the finger joints are located, the area between two finger joints, the area where the fingertip is located, the area between the fingertip and the finger joint, the area in the palm near the base of the fingers, the area between the fingers when they are together, and the area within the palm. It is understood that the areas defined on the fingers and / or palm may include only any one of the above, or may include multiple combinations. For example, the defined areas may include not only the area between two finger joints, but also areas defined within the palm. For example, please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a schematic diagram illustrating the division of a hand region according to an exemplary embodiment. Figure 4 The hand area is divided into 12 different zones. Zones 2, 5, 8, and 11 are the areas between the knuckles of two fingers, while zones 1, 4, 7, and 10 are the areas between the fingertips and knuckles. Different zones correspond to different input information; for example, different zones can correspond to different key layouts. Figure 5 Please refer to the nine-grid keypad shown. Figure 5 , Figure 5 This is a diagram illustrating the correspondence between hand regions and input information according to an exemplary embodiment. By dividing the user's hand into different regions, different regions of the hand can be associated with different input information, facilitating the selection of different regions and thus enabling information input.

[0049] An exemplary embodiment of an information input method illustrates that acquiring finger poses in a detection image includes: after detecting a user's hand, displaying input information corresponding to different regions within areas defined on the user's fingers and / or palm; and after detecting finger poses formed by the user's hand, acquiring an image of the user's hand as the detection image. Displaying input information corresponding to different regions within areas defined on the palm allows the user to quickly understand the input information corresponding to different regions, thereby enabling rapid input of information through finger poses, eliminating the need for blind operation. Figure 5 As shown, different areas of the fingers and palm correspond to different keypads, which are used for inputting information. Figure 5 The 3, 6, and 9 areas correspond to the "DEF" keys, "MNO" keys, and "WXYZ" keys on a 9-key keypad, respectively.

[0050] After displaying input information corresponding to different areas on the user's fingers and / or palm, the user can select a specific area using different finger postures, and then sequentially determine the input information corresponding to that area. Therefore, after detecting the user's finger posture, an image of the user's hand is acquired as the detection image to determine the area selected by the user, and further refine the input information. For example, if the detection image detects that the user has formed a finger posture with the middle knuckle of the index finger bent at 90°, then the target area is determined by the areas corresponding to different finger postures, and the input information corresponding to the target area is then used as the final input information.

[0051] An exemplary embodiment of an information input method, after detecting a user's hand and before displaying input information corresponding to different areas within a region defined on the user's fingers and / or palm, further includes: after detecting the user's hand, if a trigger gesture is detected, then displaying input information corresponding to different areas within the region defined on the user's fingers and / or palm; wherein the trigger gesture is used to indicate the start of determining input information through finger posture. For example, the trigger gesture may be, but is not limited to, the "OK" gesture. The trigger gesture can identify whether the user currently needs to perform gesture input, effectively avoiding accidental operations. After detecting the trigger gesture, input information corresponding to different areas is displayed within the region defined on the user's fingers and / or palm. For example, finger keypoint detection technology can be used to detect the user's hand, obtain the positions of multiple key points in the palm using the finger keypoint detection technology, and locate the finger posture based on the positions of the multiple key points. For example, finger keypoint detection technology can be used to detect... Figure 2 Regarding the key points of the middle finger, it should be noted that finger key point detection technology is a relatively mature technology in this field, and this disclosure can adopt any of them, which will not be elaborated here.

[0052] In step S12, the input information is determined based on the finger posture.

[0053] It should be noted that step S12 also includes sub-steps S121 and S122. The specific method for obtaining input information based on finger posture will be described in detail in the sub-steps of step S12. Please refer to [link / reference]. Figure 6 , Figure 6 This is a flowchart illustrating sub-step S12 as shown in an exemplary embodiment of this disclosure.

[0054] In step S121, the target area selected by the user is determined based on the finger posture.

[0055] The target area is one of the regions defined on the user's fingers and / or palm. For example... Figure 4 or Figure 5 The regions are divided in the text.

[0056] In one implementation, the target area selected by the user can be determined by the angle of the middle knuckle of the second finger. For example, the target area selected by the user is determined when the angle of the middle knuckle of the second finger is within different preset ranges; for example, when the angle is within a first preset range, a second preset range, and a third preset range, it is determined that the user has selected different target areas. In one implementation, the first preset range can be equal to 90°, the second preset range can be greater than 90° and less than 145°, and the third preset range can be greater than or equal to 145°. The second finger is at least one of the index finger, middle finger, ring finger, and little finger. Taking the index finger as an example, when the angle of the middle knuckle of the index finger is within the first preset range, the second preset range, and the third preset range, it can be determined that the index finger has selected a target area. Figure 5 The 3, 6, and 9 areas correspond to the first, second, and third keys on a 9-key keypad; the first key can be... Figure 5 The "DEF" key in the middle, the second key can be Figure 5 The "MNO" key in the middle, the third key can be Figure 5 The "WXYZ" keys on the keyboard. If the angle of the middle knuckle of the index finger is less than or equal to 90°, the index finger has triggered the "DEF" keys on the 9-key keyboard; if the angle of the middle knuckle of the index finger is greater than 90° and less than 145°, the index finger has triggered the "MNO" keys on the 9-key keyboard; and so on. This will not be elaborated further here.

[0057] Taking the middle finger as an example, when the angle of the middle joint of the middle finger is within the first preset range, the second preset range, and the third preset range, it can be determined that the middle finger has selected... Figure 5 The 2, 5, and 8 areas correspond to the fourth, fifth, and sixth keys on a 9-key keypad; the fourth key can be... Figure 5 The fifth key can be one of the "ABC" keys in the code. Figure 5 The sixth key in the "JKL" key can be... Figure 5 The "YUV" key. If the angle of the middle knuckle of the middle finger is less than or equal to 90°, it is determined that the middle finger has triggered the "ABC" key on the 9-key keyboard; if the angle of the middle knuckle of the middle finger is greater than 90° and less than 145°, it is determined that the middle finger has triggered the "JKL" key on the 9-key keyboard; and so on, which will not be elaborated here.

[0058] Taking the ring finger as an example, when the angle of the middle joint of the ring finger is within the first preset range, the second preset range, and the third preset range, it can be determined that the ring finger has selected... Figure 5 The 1, 4, and 7 areas correspond to the seventh, eighth, and ninth keys on a 9-key keypad; the seventh key can be... Figure 5 The eighth key can be the ",.?!" key. Figure 5 The "GHI" key, the ninth key can be... Figure 5 The "PQRS" key in the keyboard. For example, if the angle of the middle knuckle of the ring finger is less than or equal to 90°, it indicates that the ring finger has triggered the ",.?!" keys on the keypad; and so on. Different input information is determined by the different bending angles of the finger knuckles.

[0059] In step S122, the input information corresponding to the target area is determined based on the correspondence between the regions divided on the user's fingers and / or palms and different input information.

[0060] For example, the correspondence could be different keypad buttons corresponding to different areas of the fingers and / or palm, such as... Figure 5 Areas 3, 6, and 9 correspond to the "DEF" keys, "MNO" keys, and "WXYZ" keys on a 9-key keypad, respectively. By mapping different areas of the user's hand to different input information, the system determines the input information corresponding to the target area, eliminating the need for external input devices or controllers, thus facilitating user input.

[0061] If the input information corresponding to the target area is of one type, then the input information corresponding to the target area is determined according to the correspondence. For example, if the angle of the middle joint of the little finger is less than or equal to 90°, then the little finger is determined to have selected... Figure 5 The area where the "0" key is located is taken as the target area. The "0" key corresponds to the input information "0". Based on the correspondence, the input information corresponding to the target area is determined to be "0", and "0" is used as the input information selected by the user.

[0062] If there are multiple input options corresponding to the target area, then the multiple input options corresponding to the target area are displayed according to the correspondence, and the input option selected by the user is determined based on the selection operation performed by the user. As mentioned above, if the angle of the middle joint of the index finger is less than or equal to 90°, it is determined that the index finger has selected the correct input option. Figure 5The system uses three regions as the target area, corresponding to the "DEF" keys on a 9-key keyboard. These keys correspond to the input information "D", "E", and "F". Based on this correspondence, different input information corresponding to the target area is displayed in different selection areas. The system then captures the user's selection action (e.g., the user can choose one of "D", "E", or "F"). The system detects the selected area touched by the user and uses the input information corresponding to that area as the user's selected input. For example, if the user selects "D", then "D" is used as the corresponding input. The selection area is a region on the hand and / or in space that does not overlap with the areas defined on the user's fingers and / or palm. Different input information is determined by the bending angle of the finger joints, eliminating the need for external input devices or controllers, making user input more flexible and convenient.

[0063] It should be noted that the second finger includes the index, middle, ring, and little fingers. However, when determining input information based on the posture of the second finger, these four fingers are not always used. In one implementation, the posture of only one finger can be used to determine the input information, such as the posture of the index finger. In this case, target areas corresponding to different index finger postures can be predefined. In another implementation, the posture of only two fingers can be used to determine the input information, such as the postures of the index and middle fingers. In this case, keys corresponding to different index finger postures and keys corresponding to different middle finger postures can be predefined. Finally, the input information is determined based on the time sequence of the images to be detected and the keys triggered by the index, middle, ring, and little fingers.

[0064] In step S13, the input information is used as the input information of the electronic device.

[0065] The aforementioned determined input information is used as the input information of the electronic device.

[0066] In summary, the information input method provided by this disclosure includes: acquiring finger postures in an image to be detected, wherein the finger posture is the posture formed by the extension and / or bending of the fingers; determining input information based on the finger posture; and using the input information as input information for an electronic device. It is capable of acquiring an image containing finger postures, determining the finger postures in the image, assigning different information to different finger postures, and thus determining input information based on the finger posture and using that input information as input information for an electronic device. It eliminates the need for external dedicated input devices or controller-assisted input, facilitating user information input.

[0067] It should be noted that finger posture generally refers to the posture formed by the extension and / or bending of the fingers of a single hand. The single hand refers to either the left or right hand.

[0068] Furthermore, the aforementioned "after detecting the user's hand, if a triggered gesture is detected, then displaying input information corresponding to different areas within the areas defined on the user's fingers and / or palm" may further include:

[0069] Once the user's hand is detected to be a single hand, if a trigger gesture is detected, input information corresponding to different areas will be displayed in the areas divided on the fingers and / or palm of the single hand;

[0070] If the user's hands are detected as left and right, and a gesture is triggered, the input information corresponding to different areas is displayed in the areas defined on the fingers and / or palm of the left hand, and the input information corresponding to different areas is displayed in the areas defined on the fingers and / or palm of the right hand.

[0071] Figure 7 This is a block diagram illustrating an information input device according to an exemplary embodiment. (Refer to...) Figure 7 The device 20 includes an acquisition module 201, a processing module 202, and a determination module 203.

[0072] The acquisition module 201 is configured to acquire finger poses in an image to be detected; the finger poses are the poses formed by the fingers through extension and / or bending.

[0073] The processing module 202 is configured to determine input information based on the finger posture;

[0074] The determining module 203 is configured to use the input information as input information for the electronic device.

[0075] Optionally, the finger posture includes: a posture formed by a static finger and at least one knuckle presenting a set bending angle;

[0076] The static state refers to the fingers being stationary relative to the palm, or the rate of change of the bending angle at the knuckles being less than a set rate of change.

[0077] Optionally, the posture formed by the static fingers and at least one knuckle presenting a set bending angle includes:

[0078] The first finger in a static position, the second finger in a static position, and the middle phalanx of the second finger not in a static position are all in a set bending angle within different angle ranges, forming a posture; the middle phalanx is the second phalanx of the second finger with the fingertip pointing towards the palm.

[0079] And / or, the second finger in the static position, and the front phalanx of the first finger, are in a posture formed by a set bending angle within a different angle range; the front phalanx is the first phalanx of the first finger facing the palm direction;

[0080] The first finger is the thumb, and the second finger is any finger other than the thumb.

[0081] Optionally, the processing module 202 is further configured to determine the target area selected by the user based on the finger posture; the target area is one of the areas divided on the user's fingers and / or palm;

[0082] Based on the correspondence between regions defined on the user's fingers and / or palms and different input information, the input information corresponding to the target region is determined.

[0083] Optionally, the areas divided on the user's fingers and / or palm include at least one of the following: the area where the finger joints are located, the area between two finger joints, the area where the fingertip is located, the area between the fingertip and the finger joint, the area in the palm near the base of the fingers, the area divided at the gap between the fingers when they are together, and the area divided inside the palm.

[0084] Optionally, the acquisition module 201 is configured to, after detecting the user's hand, display input information corresponding to different areas within the areas divided on the user's fingers and / or palm;

[0085] After detecting that the user's hand has formed the finger posture, an image of the user's hand is acquired as the image to be detected.

[0086] Optionally, the acquisition module 201 is configured to, upon detecting a user's hand, display input information corresponding to different areas within a region defined on the user's fingers and / or palm if a trigger gesture is detected; wherein the trigger gesture is used to indicate the start of determining input information through finger posture.

[0087] Optionally, the processing module 202 is further configured to determine the input information corresponding to the target region based on the correspondence if the input information corresponding to the target region is one type.

[0088] If there are multiple types of input information corresponding to the target area, then the multiple types of input information corresponding to the target area are displayed according to the correspondence, and the input information selected by the user is determined based on the selection operation performed by the user.

[0089] Optionally, the processing module 202 is further configured to display different input information corresponding to the target region in different selection areas according to the correspondence;

[0090] Detect the selection area touched by the user, and use the input information corresponding to the selection area touched by the user as the input information selected by the user;

[0091] The selected area is the area in the hand and / or space that does not overlap with the area defined on the user's fingers and / or palm.

[0092] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0093] This disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the steps of the information input method provided in this disclosure.

[0094] Figure 8 This is a block diagram illustrating an apparatus for information input according to an exemplary embodiment. For example, apparatus 800 may be an electronic device equipped with a camera, such as a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, or personal digital assistant.

[0095] Reference Figure 8 The device 800 may include one or more of the following components: a processing component 802, a first memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output interface 812, a sensor component 814, and a communication component 816.

[0096] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more first processors 820 to execute instructions to complete all or part of the steps of the aforementioned information input method. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0097] The first memory 804 is configured to store various types of data to support the operation of the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phonebook data, messages, pictures, videos, etc. The first memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0098] Power supply component 806 provides power to various components of device 800. Power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to device 800.

[0099] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0100] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in first memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0101] Input / output interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0102] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0103] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0104] In an exemplary embodiment, the device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described information input method.

[0105] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a first memory 804 including instructions, which can be executed by a first processor 820 of the device 800 to complete the aforementioned information input method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0106] The aforementioned device can be a standalone electronic device or a part of a standalone electronic device. For example, in one embodiment, the device can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be a single IC or a collection of multiple ICs. The chip can include, but is not limited to, the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), and SoC (System on Chip). The aforementioned integrated circuit or chip can be used to execute executable instructions (or code) to implement the aforementioned information input method. The executable instructions can be stored in the integrated circuit or chip or obtained from other devices or equipment. For example, the integrated circuit or chip may include a second processor, a second memory, and an interface for communicating with other devices. The executable instruction can be stored in the second memory, and when the executable instruction is executed by the second processor, it implements the above-mentioned information input method; or, the integrated circuit or chip can receive the executable instruction through the interface and transmit it to the second processor for execution to implement the above-mentioned information input method.

[0107] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a programmable device, the computer program having a code portion for performing the information input method described above when executed by the programmable device.

[0108] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0109] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An information input method, characterized in that, include: Obtain the finger pose in the image to be detected; The finger posture is a posture formed by extending and / or bending the fingers; wherein, the finger posture includes: a posture formed by a static finger and a finger with at least one knuckle presenting a set bending angle, wherein static means that the finger is stationary relative to the palm, or the range of variation of the bending angle presented at the knuckle is less than the set angle range. The input information is determined based on the finger posture; The input information is used as the input information of the electronic device; The process of determining input information based on the finger posture includes determining different input information by using different bending angles of the finger joints.

2. The method according to claim 1, characterized in that, The posture formed by the static fingers and at least one knuckle presenting a predetermined bending angle includes: The first finger in a static position, the second finger in a static position, and the middle phalanx of the second finger not in a static position are all in a set bending angle within different angle ranges, forming a posture; the middle phalanx is the second phalanx of the second finger with the fingertip pointing towards the palm. And / or, the second finger in the static position, and the front phalanx of the first finger, are in a posture formed by a set bending angle within a different angle range; the front phalanx is the first phalanx of the first finger facing the palm direction; The first finger is the thumb, and the second finger is any finger other than the thumb.

3. The method according to claim 1, characterized in that, The step of determining input information based on the finger posture includes: The target area selected by the user is determined based on the finger posture; the target area is one of the regions divided on the user's fingers and / or palm. Based on the correspondence between regions defined on the user's fingers and / or palms and different input information, the input information corresponding to the target region is determined.

4. The method according to claim 3, characterized in that, The areas defined on the user's fingers and / or palms include at least one of the following: The areas where the fingers are knuckles, the areas between the knuckles of two fingers, the areas where the fingertips are, the areas between the fingertips and the knuckles, the areas in the palm near the base of the fingers, the areas between the fingers when they are together, and the areas within the palm.

5. The method according to claim 3, characterized in that, The process of obtaining the finger pose in the image to be detected includes: After detecting the user's hand, the input information corresponding to different areas is displayed in the areas defined on the user's fingers and / or palm; After detecting that the user's hand has formed the finger posture, an image of the user's hand is acquired as the image to be detected.

6. The method according to claim 5, characterized in that, Before displaying input information corresponding to different areas within the regions defined on the user's fingers and / or palm after detecting the user's hand, the method further includes: After detecting the user's hand, if a trigger gesture is detected, input information corresponding to different areas is displayed in the areas divided on the user's fingers and / or palm; wherein, the trigger gesture is used to indicate the start of determining input information through finger posture.

7. The method according to claim 3, characterized in that, The step of determining the input information corresponding to the target region based on the correspondence between regions divided on the user's fingers and / or palms and different input information includes: If the input information corresponding to the target region is of one type, then the input information corresponding to the target region is determined according to the correspondence. If there are multiple types of input information corresponding to the target area, then the multiple types of input information corresponding to the target area are displayed according to the correspondence, and the input information selected by the user is determined based on the selection operation performed by the user.

8. The method according to claim 7, characterized in that, The step of displaying multiple input information corresponding to the target area according to the correspondence, and determining the input information selected by the user based on the selection operation performed by the user, includes: Based on the correspondence, different input information corresponding to the target area is displayed in different selection areas; Detect the selection area touched by the user, and use the input information corresponding to the selection area touched by the user as the input information selected by the user; The selected area is the area in the hand and / or space that does not overlap with the area defined on the user's fingers and / or palm.

9. An information input device, characterized in that, include: The acquisition module is configured to acquire finger poses in the image to be detected; The finger posture is a posture formed by extending and / or bending the fingers; wherein, the finger posture includes: a posture formed by a static finger and a finger with at least one knuckle presenting a set bending angle, wherein static means that the finger is stationary relative to the palm, or the range of variation of the bending angle presented at the knuckle is less than the set angle range. The processing module is configured to determine input information based on the finger posture; The determining module is configured to use the input information as input information for the electronic device; The processing module is also configured to determine different input information by using different bending angles of the finger joints.

10. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing computer program instructions thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1 to 8.

Citation Information

Patent Citations

  • Information processing method, device and equipment

    CN103793057A

  • Input method and apparatus of portable device

    US20140055343A1

  • Gesture movement recognition method, apparatus, and device

    US20200393911A1