Gesture recognition method, device and equipment, computer storage medium and vehicle
By calculating the distance between the key points of the fingertip and judging the gesture type, establishing a touch hot zone solves the problem of misjudgment of input operation judgments in the prior art, improving the accuracy and interactive experience of gesture recognition.
Patent Information
- Application Number
- CN202311089050.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-28
- Publication Date
- 2025-02-28
AI Technical Summary
The prior art is prone to misjudgment when judging user input operations, resulting in poor interaction experience.
By obtaining the first gesture data of the target keyframe, the distance between each fingertip key point and the first fingertip key point is calculated, the fingertip key point with a distance greater than or equal to the preset threshold is determined as the target key point, and the gesture type is judged based on the number of target key points, thereby establishing a corresponding touch hot zone for operation.
Improve the accuracy of input gesture recognition, reduce misjudgment, and improve the interactive experience between users and electronic devices.
Smart Images
Figure CN119536608A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of computer technology, and in particular, relates to a gesture recognition method, device, equipment, computer storage medium and vehicle. Background Art
[0002] Touch interaction brings great convenience to people's work and life. In order to make it more convenient for users to interact with electronic devices, air touch came into being. Air touch involves capturing hand movement images through a camera, creating a virtual touch hot zone in space for touch operation, and controlling the corresponding electronic device according to the user's operation in the touch hot zone.
[0003] The related technology involves presetting a touch plane according to the user's hand position, and determining that the finger has performed an input operation when the distance between the finger joint point and the preset touch plane is less than a preset distance threshold. However, in some cases, the user may accidentally touch the touch plane, resulting in a misjudgment of the input operation.
[0004] With the increasing demand for interactivity, how to accurately judge user input operations has become a problem that needs to be solved urgently. Summary of the invention
[0005] The embodiments of the present application provide a gesture recognition method, apparatus, device, computer storage medium, and vehicle, which can improve the accuracy of input gesture recognition, thereby improving the interaction experience between the user and the electronic device.
[0006] In a first aspect, an embodiment of the present application provides a gesture recognition method, comprising:
[0007] Acquire first gesture data corresponding to the target key frame, the first gesture data including the spatial coordinates of a first fingertip key point in the target key frame, and the spatial coordinates of a plurality of second fingertip key points corresponding to the first fingertip key point, the second fingertip key points being fingertip key points belonging to the same palm as the first fingertip key point;
[0008] Based on the spatial coordinates of the first fingertip key point and the spatial coordinates of the plurality of second fingertip key points, respectively calculating a first distance between each second fingertip key point and the corresponding first fingertip key point;
[0009] When the first distance is greater than or equal to a preset first distance threshold, determining the corresponding second fingertip key point as the target key point;
[0010] When the number of target key points corresponding to the first fingertip key point is within a preset number range, it is determined that the gesture type corresponding to the first fingertip key point is an input gesture.
[0011] In an optional implementation, after determining that the gesture type corresponding to the first fingertip key point is an input gesture, the method further includes:
[0012] Establish touch hotspots;
[0013] An operation corresponding to the first gesture data is performed in the touch hot zone.
[0014] In an optional implementation, the target key frame includes 1 or 2 first fingertip key points; the method further includes: determining the number of input gestures in the target key frame according to the gesture type corresponding to each first fingertip key point;
[0015] Create touch hotspots, including:
[0016] When the number of input gestures is 1, a single-hand touch hot zone is established;
[0017] When the number of input gestures is 2, a two-hand touch hot zone is established, and the two-hand touch hot zone includes touch sub-hot zones corresponding to each input gesture.
[0018] In an optional implementation, the area of the single-hand touch hot zone is smaller than the area of the double-hand touch hot zone.
[0019] In an optional implementation, performing a touch operation corresponding to the first gesture data in the touch hot zone includes:
[0020] In the target key frame, when the first distance corresponding to the target fingertip key point is less than or equal to the preset second distance threshold, the touch position is determined in the touch hot zone based on the spatial coordinates of the target fingertip key point, and the target fingertip key point is at least one of the target key points;
[0021] Perform a touch operation at the touch location.
[0022] In an optional implementation, the touch operation includes a tap operation or a long press operation; and the method further includes:
[0023] Obtain second gesture data corresponding to each key frame in N key frames before the target key frame, where N is an integer greater than or equal to 1;
[0024] Calculating, according to the second gesture data, a second distance between a target fingertip key point and a corresponding first fingertip key point in each key frame;
[0025] Perform touch operations at the touch location, including:
[0026] When the second distances corresponding to the first N key frames are all greater than the preset second distance threshold, performing a tap operation at the touch position;
[0027] When the second distance corresponding to the first N key frames is less than or equal to a preset second distance threshold, a long press operation is performed at the touch position.
[0028] In an optional implementation, the first distance between each second fingertip key point and the corresponding first fingertip key point may be calculated by the following formula 1:
[0029]
[0030] In formula 1, i represents the serial number of the second fingertip key point, d i represents the first distance between the second fingertip key point with serial number i and its corresponding first fingertip key point, x i ,y i 、z i represents the horizontal coordinate, vertical coordinate and vertical coordinate of the second fingertip key point with serial number i, respectively. f ,y f 、z f Respectively represent the horizontal coordinate, vertical coordinate and vertical coordinate of the first fingertip key point.
[0031] In a second aspect, an embodiment of the present application provides a gesture recognition device, including:
[0032] an acquisition module, configured to acquire first gesture data corresponding to a target key frame, wherein the first gesture data includes the spatial coordinates of a first fingertip key point in the target key frame, and the spatial coordinates of a plurality of second fingertip key points corresponding to the first fingertip key point, wherein the second fingertip key points are fingertip key points belonging to the same palm as the first fingertip key points;
[0033] A calculation module, configured to calculate a first distance between each second fingertip key point and a corresponding first fingertip key point based on the spatial coordinates of the first fingertip key point and the spatial coordinates of a plurality of second fingertip key points;
[0034] A determination module, configured to determine, when the first distance is greater than or equal to a preset first distance threshold, the corresponding second fingertip key point as a target key point;
[0035] The determination module is further used to determine that the gesture type corresponding to the first fingertip key point is an input gesture when the number of target key points corresponding to the first fingertip key point is within a preset number range.
[0036] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising: a processor and a memory storing computer program instructions;
[0037] When the processor executes the computer program instructions, the gesture recognition method of any optional implementation manner of the first aspect of the present application is implemented.
[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, a gesture recognition method as in any optional implementation of the first aspect of the present application is implemented.
[0039] In a fifth aspect, an embodiment of the present application provides a vehicle, which includes: a gesture recognition device as in the second aspect or an electronic device as in the third aspect.
[0040] The gesture recognition method, device, equipment, computer storage medium and vehicle of the embodiment of the present application can obtain the first gesture data corresponding to the target key frame, and use the first gesture data to calculate the first distance between each second fingertip key point and the corresponding first fingertip key point. Then, when the first distance is greater than or equal to the preset first distance threshold, the corresponding second fingertip key point is determined to be the target key point. In this way, the first distance between the remaining fingertip key points and the first fingertip key point is calculated based on the first fingertip key point, and the state of the remaining fingertip key points can be accurately determined by comparing the first distance corresponding to the remaining fingertip key points with the first distance threshold. In the embodiment of the present application, when the number of target key points corresponding to the first fingertip key point is within the preset number range, the gesture type corresponding to the first fingertip key point is determined to be an input gesture. In this way, the shape of the palm can be identified according to the state of the remaining fingertip key points, so as to accurately determine whether the gesture type corresponding to the first fingertip key point is an input gesture. In this way, the accuracy of input gesture recognition can be improved, which is conducive to improving the accuracy of input operation judgment in touch interaction scenarios and improving the interactive experience between users and electronic devices. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0042] Figure 1 is a flowchart of a gesture recognition method provided by an embodiment of the present application;
[0043] Figure 2 is a schematic diagram of a hand image provided by another embodiment of the present application;
[0044] Figure 3 is a schematic diagram of a gesture type provided by yet another embodiment of the present application;
[0045] Figure 4 is a schematic diagram of a single-hand touch hot zone provided in yet another embodiment of the present application;
[0046] Figure 5is a schematic diagram of a two-hand touch hot zone provided in yet another embodiment of the present application;
[0047] Figure 6 is a schematic diagram of determining a touch position in a gesture recognition method provided in yet another embodiment of the present application;
[0048] Figure 7 is a structural diagram of a gesture recognition device provided in yet another embodiment of the present application;
[0049] Figure 8 It is a structural diagram of an electronic device provided in yet another embodiment of the present application. DETAILED DESCRIPTION
[0050] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0051] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0052] As described in the background technology, touch interaction brings great convenience to people's work and life, and people's demand for the interactivity of electronic devices is also increasing. The most common way to provide input for electronic devices such as computer systems is through a keyboard, but keyboards are not very portable devices. For this reason, many devices are equipped with virtual keyboards, and users input through virtual keyboards. Therefore, how to accurately recognize input gestures based on virtual keyboards to perform input operations has become a problem that needs to be solved urgently.
[0053] Based on this, in order to improve the accuracy of input gesture recognition and thus enhance the user's interactive experience with the electronic device, the inventor, after in-depth thinking, cleverly designed a gesture recognition method, device, equipment, computer storage medium and vehicle.
[0054] The gesture recognition method provided in the embodiment of the present application can be executed by a device that can be a gesture recognition device, or a partial module in the gesture recognition device that is used to execute the gesture recognition method. The gesture recognition method provided in the embodiment of the present application is described in detail by taking the gesture recognition device executing the gesture recognition method as an example.
[0055] The following first introduces the gesture recognition method provided in the embodiment of the present application in conjunction with the accompanying drawings.
[0056] Figure 1 FIG. 1 is a flow chart of a gesture recognition method provided by an embodiment of the present application. Figure 1 As shown, the gesture recognition method may specifically include the following steps S110 to S140.
[0057] S110, obtaining first gesture data corresponding to the target key frame, the first gesture data including the spatial coordinates of a first fingertip key point in the target key frame, and the spatial coordinates of multiple second fingertip key points corresponding to the first fingertip key point, the second fingertip key point being the fingertip key point belonging to the same palm as the first fingertip key point.
[0058] In step S110, the target key frame may be a key frame acquired by an image acquisition device, and the target key frame may include hand information. As an example, a video stream may be acquired by a camera, and a key frame may be captured from the video stream. The target key frame may be a key frame corresponding to a preset time period. In one embodiment, the target key frame may be a key frame acquired at the current moment.
[0059] In step S110, the first gesture data may include gesture data obtained by identifying hand information of a target key frame. The first fingertip key point may be a pre-selected fingertip key point, for example, it may be a thumb key point. The multiple second fingertip key points corresponding to the first fingertip key point may include the remaining fingertip key points belonging to the same palm as the first fingertip key point, for example, the index finger, middle finger, ring finger, and little finger. The first gesture data may be obtained by processing the target key frame using a pre-trained neural network model. The training process of the neural network model may be implemented by methods known in the art and will not be elaborated herein. The training sample of the neural network model may be a hand image with manually annotated skeletal point information. The skeletal point information may include the spatial coordinates of the skeletal points. As an example, the hand image with manually annotated skeletal point information may be as follows. Figure 2As shown. The above-mentioned spatial coordinates can be spatial coordinates determined based on a preset coordinate system, for example, spatial coordinates determined with the hand joint bone points as the origin. The spatial coordinates can be three-dimensional coordinates, which can be represented by real parameters of the physical world. As an example, an RGB image with a resolution of 640*480 can be generated based on the target key frame, and the hand bone points in the image can be identified by a neural network model to obtain a hand bone point spatial coordinate data set n:[x n ,y n ,z n ], n represents the number of the hand skeleton point. The spatial coordinates of the first fingertip key point and the spatial coordinates of multiple second fingertip key points corresponding to the first fingertip key point are determined from the hand skeleton point spatial coordinate data set to obtain the first gesture data.
[0060] S120, based on the spatial coordinates of the first fingertip key point and the spatial coordinates of the plurality of second fingertip key points, respectively calculate a first distance between each second fingertip key point and the corresponding first fingertip key point.
[0061] In step S120, the first distance between each second fingertip key point and the corresponding first fingertip key point can be calculated by the following formula 1.
[0062]
[0063] In formula 1, i represents the serial number of the second fingertip key point, d i Indicates the first distance between the second fingertip key point with sequence number i and its corresponding first fingertip key point. i ,y i 、z i represents the horizontal coordinate, vertical coordinate and vertical coordinate of the second fingertip key point with serial number i, respectively. f ,y f 、z f Respectively represent the horizontal coordinate, vertical coordinate and vertical coordinate of the first fingertip key point.
[0064] S130: When the first distance is greater than or equal to a preset first distance threshold, determine the corresponding second fingertip key point as the target key point.
[0065] In step S130, the preset first distance threshold may be a distance threshold pre-set according to the finger position corresponding to the second fingertip key point. When the first distance between the second fingertip key point and the corresponding first fingertip key point is less than the corresponding first distance threshold, the finger corresponding to the second fingertip key point may be considered to be in a closed state; when the first distance between the second fingertip key point and the corresponding first fingertip key point is greater than or equal to the corresponding first distance threshold, the finger corresponding to the second fingertip key point may be considered to be in an open state. In one embodiment, the second fingertip key point corresponding to the finger in the open state may be used as the above-mentioned target key point.
[0066] S140: When the number of target key points corresponding to the first fingertip key point is within a preset number range, determine that the gesture type corresponding to the first fingertip key point is an input gesture.
[0067] In step S140, the gesture type corresponding to the first fingertip key point may be determined based on the number of fingers corresponding to the first fingertip key point that are in an open state. In one embodiment, the gesture type may be divided into input gestures and non-input gestures according to preset rules based on the number of fingers corresponding to the first fingertip key point that are in an open state. The preset rules may include: when the number of fingers corresponding to the first fingertip key point that are in an open state is within a preset number interval, determining that the hand corresponding to the first fingertip key point is an input gesture; when the number of fingers corresponding to the first fingertip key point that are in an open state is not within a preset number interval, determining that the hand corresponding to the first fingertip key point is a non-input gesture. The preset number interval can be set according to actual needs, and this application does not limit this. Figure 3 A schematic diagram of gesture types provided by an embodiment of the present application is shown. Figure 3 As shown, taking the thumb key point as the first fingertip key point as an example, when the fingers other than the thumb in the same palm are all in an open state or in a closed state, the user generally does not need touch input, so the corresponding gesture type is a non-input gesture. When some of the fingers other than the thumb (such as the index finger) are in an open state, the user may need touch input, so the corresponding gesture type is an input gesture.
[0068] The gesture recognition method of the embodiment of the present application can obtain the first gesture data corresponding to the target key frame, and use the first gesture data to calculate the first distance between each second fingertip key point and the corresponding first fingertip key point. Then, when the first distance is greater than or equal to the preset first distance threshold, the corresponding second fingertip key point is determined to be the target key point. In this way, taking the first fingertip key point as a reference, the first distance between the remaining fingertip key points and the first fingertip key point is calculated, and by comparing the first distance corresponding to the remaining fingertip key points with the first distance threshold, the state of the remaining fingertip key points can be accurately determined. In the embodiment of the present application, when the number of target key points corresponding to the first fingertip key point is within a preset number interval, the gesture type corresponding to the first fingertip key point is determined to be an input gesture. Based on the number of target key points corresponding to the first fingertip key point, the gesture information contained in the target key frame is generated, and the gesture information includes the gesture type. In this way, the shape of the palm can be identified according to the state of the remaining fingertip key points, so as to accurately determine whether the target key frame contains the gesture type corresponding to the first fingertip key point as an input gesture. Then, when the gesture type includes an input gesture, a touch hot zone is established, and the touch operation corresponding to the first gesture data is performed in the touch hot zone. In this way, the accuracy of touch processing input gesture recognition can be improved, which is beneficial to improving the accuracy of input operation judgment in touch interaction scenarios and improving the interaction experience between users and electronic devices.
[0069] In one embodiment, after determining that the gesture type corresponding to the first fingertip key point is an input gesture, the method may further include:
[0070] Create touch hotspots.
[0071] An operation corresponding to the first gesture data is performed in the touch hot zone.
[0072] In the above implementation, establishing the touch hotspot can be implemented according to methods known in the art, which are not limited here. As an example, the touch hotspot can be established according to the spatial coordinates of the preset reference points in the target key frame and the preset touch hotspot size. The preset reference points may include but are not limited to wrist joint key points, user head key points, and user shoulder key points.
[0073] The above implementation can establish a touch hot zone after determining that the first fingertip key point is an input gesture, and perform an operation corresponding to the first gesture data in the touch hot zone. In this way, the touch hot zone can be established when it is determined that the user needs to perform an input operation, thereby reducing the situation where the touch hot zone is established when the user does not need to perform an input operation, thereby improving the user's interactive experience with the electronic device.
[0074] In one embodiment, the target key frame may include 1 or 2 first fingertip key points. The method may further include: determining the number of input gestures in the target key frame according to the gesture type corresponding to each first fingertip key point.
[0075] Establishing touch hotspots may include:
[0076] When the number of input gestures is 1, a single-hand touch hot zone is established.
[0077] When the number of input gestures is 2, a two-hand touch hot zone is established, and the two-hand touch hot zone includes touch sub-hot zones corresponding to each input gesture.
[0078] In the above implementation, the target key frame may include one or two palms, and correspondingly may include one or two first fingertip key points, such as two thumb key points. The above implementation may determine the gesture type corresponding to each first fingertip key point based on the number of target key points corresponding to each first fingertip key point, thereby determining the number of input gestures in the target key frame.
[0079] When establishing a touch hotspot, the size, shape, etc. of the touch hotspot can be adjusted according to the number of input gestures. As an example, a single-hand touch hotspot can be as follows: Figure 4 As shown, the two-hand touch hot zone can be Figure 5 As shown. Figure 5 As shown, in the two-hand touch hot zone, the touch sub-hot zones corresponding to the two hands respectively can be separated from each other.
[0080] Since keyboard operation habits require cooperation of both hands, when the left and right hands are used in the same hot zone at the same time, due to the accuracy of the computer vision reasoning algorithm and the angle of the human body, the hand that is clicking will be misjudged, resulting in operation errors.
[0081] The above-mentioned embodiment establishes a corresponding one-hand touch hot zone or a two-hand touch hot zone according to the number of input gestures, so that the type of touch hot zone can be flexibly adjusted according to the number of hands that the user actually performs touch operations. In this way, a complete keyboard hot zone can be established for a single input gesture during single-hand input (only one hand appears in the target key frame, or both hands appear in the target key frame but only one hand is in the input gesture); during two-hand input (both hands appear in the target key frame and are both input gestures), a touch sub-hot zone is established based on each input gesture in both hands to achieve simultaneous operation of both hands. In this way, when the spatial gesture keyboard is manipulated by both hands, the risk of misjudgment can be reduced by controlling different areas with both hands, thereby improving the accuracy of touch processing; at the same time, because the touch hot zones of both hands are separated, there is no need to consider the distance between the two hands, and the user can choose a comfortable posture for operation, such as when sitting, placing both elbows on the armrests of the chair for operation, thus improving the flexibility of touch processing and further improving the user's experience of interacting with electronic devices.
[0082] In one embodiment, the area of the single-hand touch hot zone may be smaller than the area of the double-hand touch hot zone.
[0083] Since keyboard operation is usually done with two hands, and a 26-key keyboard has many keys, it is difficult to take into account the operation requirements of one or both hands if a touch hotspot with a fixed size or shape is used. For example, for a virtual keyboard, the touch hotspot suitable for two-handed operation is larger, and there will be a dead zone when operating with one hand; the touch hotspot suitable for one-handed operation is smaller, and the keys are densely distributed, which is prone to misoperation. In addition, it is very difficult to use combination keys in the one-handed touch hotspot. It is difficult to complete the operation of switching between uppercase and lowercase by pressing the shift key and the letter key at the same time during the input process.
[0084] In the above embodiment, when inputting with one hand, a smaller, complete keyboard hot zone is established for a single input gesture; when inputting with two hands, a touch sub-hot zone is established based on each input gesture in both hands, and two larger half-keyboard hot zones are constructed to achieve simultaneous operation of both hands, continuous input, and combination key input. In this way, the accuracy of touch processing in both single-handed input scenarios and double-handed input scenarios can be improved.
[0085] In one embodiment, performing a touch operation corresponding to the first gesture data in the touch hot zone may specifically include:
[0086] In the target key frame, when the first distance corresponding to the target fingertip key point is less than or equal to the preset second distance threshold, the touch position is determined in the touch hot zone based on the spatial coordinates of the target fingertip key point, and the target fingertip key point is at least one of the target key points.
[0087] Perform a touch operation at the touch location.
[0088] The preset second distance threshold may be a preset threshold for characterizing the bending state of the finger corresponding to the target fingertip key point. When the distance between the target fingertip key point and the first fingertip key point is less than or equal to the corresponding second distance threshold, the finger corresponding to the target fingertip key point may be considered to be in a bending state. At this time, the finger may be considered to be performing a touch operation. Determining the touch position in the touch hot zone based on the spatial coordinates of the target fingertip key point may include determining the touch position in the touch hot zone based on a mapping relationship between the spatial coordinates and the touch hot zone. As an example, Figure 6 As shown, when touch input is performed with one hand, the touch position (e.g., the k key) can be determined in the one-hand touch hot zone based on the spatial coordinates of the target fingertip key point (e.g., the index finger); when touch input is performed with both hands, the touch position (e.g., the d key, the p key) can be determined in the two-hand touch hot zone based on the spatial coordinates of the target fingertip key point (e.g., the index fingers of both hands).
[0089] This embodiment can determine the touch position in the touch hot zone based on the spatial coordinates of the target fingertip key point when the finger corresponding to the target finger key point performs a touch operation, so as to perform the corresponding touch operation at the touch position, thereby further improving the accuracy of touch processing.
[0090] In one embodiment, the touch operation may include a tap operation or a long press operation.
[0091] The method may also include:
[0092] The second gesture data corresponding to each key frame in the first N key frames of the target key frame is obtained, where N is an integer greater than or equal to 1.
[0093] According to the second gesture data, a second distance between the target fingertip key point and the corresponding first fingertip key point in each key frame is calculated respectively.
[0094] Perform touch operations at the touch location, which may include:
[0095] When the second distances corresponding to the first N key frames are all greater than the preset second distance threshold, a tap operation is performed at the touch position.
[0096] When the second distance corresponding to the first N key frames is less than or equal to a preset second distance threshold, a long press operation is performed at the touch position.
[0097] In the above embodiment, the state information of the target fingertip key point in the target key frame can be determined based on the previous N frames of the historical key frame of the target key frame. For example, if the second distances corresponding to the previous N frames of the historical key frame are all greater than the preset second distance threshold, it can be considered that the finger corresponding to the target fingertip key point did not perform a touch operation before the target key frame. In the target key frame, the state information of the target fingertip key point can include that the finger corresponding to the target fingertip key point changes from an extended state to a bent state, and at this time, it can be considered that the finger corresponding to the target fingertip key point performs a tapping operation. If the second distances corresponding to the previous N frames of the historical key frame are all less than or equal to the preset second distance threshold, it can be considered that the finger corresponding to the target fingertip key point performs a touch operation before the target key frame. In the target key frame, the state information of the target fingertip key point can include that the finger corresponding to the target fingertip key point remains in a bent state, and at this time, it can be considered that the finger corresponding to the target fingertip key point performs a long press operation. In this way, the dynamic changes of the finger corresponding to the target fingertip key point can be determined based on the historical key frame and the current key frame, so as to perform the corresponding touch operation, thereby improving the accuracy of touch processing.
[0098] like Figure 7 As shown, the gesture recognition device 200 may include an acquisition module 201 , a calculation module 202 , and a determination module 203 .
[0099] The acquisition module 201 is used to acquire the first gesture data corresponding to the target key frame, the first gesture data including the spatial coordinates of the first fingertip key point in the target key frame, and the spatial coordinates of multiple second fingertip key points corresponding to the first fingertip key point, the second fingertip key point being the fingertip key point belonging to the same palm as the first fingertip key point.
[0100] The calculation module 202 is used to calculate the first distance between each second fingertip key point and the corresponding first fingertip key point based on the spatial coordinates of the first fingertip key point and the spatial coordinates of multiple second fingertip key points.
[0101] The determination module 203 is used to determine the corresponding second fingertip key point as the target key point when the first distance is greater than or equal to a preset first distance threshold.
[0102] The determination module 203 is further configured to determine that the gesture type corresponding to the first fingertip key point is an input gesture when the number of target key points corresponding to the first fingertip key point is within a preset number range.
[0103] The gesture recognition device of the embodiment of the present application can obtain the first gesture data corresponding to the target key frame, and use the first gesture data to calculate the first distance between each second fingertip key point and the corresponding first fingertip key point. Then, when the first distance is greater than or equal to the preset first distance threshold, the corresponding second fingertip key point is determined to be the target key point. In this way, taking the first fingertip key point as a reference, the first distance between the remaining fingertip key points and the first fingertip key point is calculated, and by comparing the first distance corresponding to the remaining fingertip key points with the first distance threshold, the state of the remaining fingertip key points can be accurately determined. In the embodiment of the present application, when the number of target key points corresponding to the first fingertip key point is within a preset number interval, the gesture type corresponding to the first fingertip key point is determined to be an input gesture. Based on the number of target key points corresponding to the first fingertip key point, the gesture information contained in the target key frame is generated, and the gesture information includes the gesture type. In this way, the shape of the palm can be identified according to the state of the remaining fingertip key points, so as to accurately determine whether the target key frame contains the gesture type corresponding to the first fingertip key point as an input gesture. Then, when the gesture type includes an input gesture, a touch hot zone is established, and the touch operation corresponding to the first gesture data is performed in the touch hot zone. In this way, the accuracy of touch processing input gesture recognition can be improved, which is beneficial to improving the accuracy of input operation judgment in touch interaction scenarios and improving the interaction experience between users and electronic devices.
[0104] In one embodiment, the gesture recognition device may further include:
[0105] Create a module for creating touch hotspots.
[0106] The execution module is used to execute an operation corresponding to the first gesture data in the touch hot zone.
[0107] In one embodiment, the target key frame may include one or two first fingertip key points. The gesture recognition device may further include: a determination module for determining the number of input gestures in the target key frame according to the gesture type corresponding to each first fingertip key point.
[0108] The establishment module is used to establish the touch hot zone, which may specifically include:
[0109] A module is established to establish a single-hand touch hot zone when the number of input gestures is 1.
[0110] The establishing module is used to establish a two-hand touch hot zone when the number of input gestures is 2, and the two-hand touch hot zone includes a touch sub-hot zone corresponding to each input gesture.
[0111] In one embodiment, the area of the single-hand touch hot zone may be smaller than the area of the double-hand touch hot zone.
[0112] In one embodiment, the execution module is used to execute the touch operation corresponding to the first gesture data in the touch hot zone, which may specifically include:
[0113] A determination module is used to determine the touch position in the touch hot zone based on the spatial coordinates of the target fingertip key point when the first distance corresponding to the target fingertip key point in the target key frame is less than or equal to a preset second distance threshold, and the target fingertip key point is at least one of the target key points.
[0114] The execution module is used to execute the touch operation at the touch position.
[0115] In one embodiment, the touch operation may include a tap operation or a long press operation.
[0116] The gesture recognition device may further include:
[0117] The acquisition module is used to acquire the second gesture data corresponding to each key frame in the first N key frames of the target key frame, where N is an integer greater than or equal to 1.
[0118] The calculation module is used to calculate the second distance between the target fingertip key point and the corresponding first fingertip key point in each key frame according to the second gesture data.
[0119] The execution module is used to execute the touch operation at the touch position, which may specifically include:
[0120] The execution module is used to perform a tap operation at the touch position when the second distances corresponding to the first N key frames are all greater than a preset second distance threshold.
[0121] The execution module is used to perform a long press operation at the touch position when the second distance corresponding to the first N key frames is less than or equal to a preset second distance threshold.
[0122] The gesture recognition device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.
[0123] Figure 8 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.
[0124] The electronic device may include a processor 301 and a memory 302 storing computer program instructions.
[0125] Specifically, the processor 301 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0126] The memory 302 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 302 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 302 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 302 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 302 is a non-volatile solid-state memory.
[0127] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present application.
[0128] The processor 301 implements any one of the gesture recognition methods in the above embodiments by reading and executing computer program instructions stored in the memory 302 .
[0129] As an example, the electronic device may further include a communication interface 303 and a bus 310. Figure 8 As shown, the processor 301, the memory 302, and the communication interface 303 are connected via a bus 310 and communicate with each other.
[0130] The communication interface 303 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.
[0131] Bus 310 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 310 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.
[0132] The electronic device can execute the gesture recognition method in the embodiment of the present application, thereby realizing the combination Figure 1 and Figure 7 The present invention describes a method and apparatus for gesture recognition.
[0133] In addition, in combination with the gesture recognition method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the gesture recognition methods in the above embodiment is implemented.
[0134] In addition, in combination with the gesture recognition method in the above embodiment, the embodiment of the present invention can provide a vehicle for implementation. The vehicle includes the gesture recognition device or electronic device in the above embodiment.
[0135] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0136] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
[0137] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0138] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.
[0139] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.
Claims
1. A gesture recognition method, characterized in that: include: Acquire first gesture data corresponding to a target key frame, wherein the first gesture data includes spatial coordinates of a first fingertip key point in the target key frame, and spatial coordinates of a plurality of second fingertip key points corresponding to the first fingertip key point, wherein the second fingertip key points are fingertip key points belonging to the same palm as the first fingertip key point; Based on the spatial coordinates of the first fingertip key point and the spatial coordinates of the plurality of second fingertip key points, respectively calculating a first distance between each of the second fingertip key points and the corresponding first fingertip key point; When the first distance is greater than or equal to a preset first distance threshold, determining the corresponding second fingertip key point as a target key point; When the number of target key points corresponding to the first fingertip key point is within a preset number range, it is determined that the gesture type corresponding to the first fingertip key point is an input gesture.
2. The method according to claim 1, characterized in that After determining that the gesture type corresponding to the first fingertip key point is an input gesture, the method further includes: Establish touch hotspots; An operation corresponding to the first gesture data is performed in the touch hot zone.
3. The method according to claim 2, characterized in that The target key frame includes one or two first fingertip key points; the method further includes: determining the number of input gestures in the target key frame according to the gesture type corresponding to each first fingertip key point; The step of establishing a touch hot zone includes: When the number of the input gestures is 1, a single-hand touch hot zone is established; When the number of the input gestures is 2, a two-hand touch hot zone is established, and the two-hand touch hot zone includes a touch sub-hot zone corresponding to each input gesture.
4. The method according to claim 3, characterized in that: The area of the single-hand touch hot zone is smaller than the area of the double-hand touch hot zone.
5. The method according to claim 2, characterized in that: The performing the touch operation corresponding to the first gesture data in the touch hot zone includes: In the target key frame, when the first distance corresponding to the target fingertip key point is less than or equal to a preset second distance threshold, determining the touch position in the touch hot zone based on the spatial coordinates of the target fingertip key point, the target fingertip key point being at least one of the target key points; A touch operation is performed at the touch position.
6. The method according to claim 5, characterized in that The touch operation includes a tap operation or a long press operation; the method further includes: Obtaining second gesture data corresponding to each key frame in N key frames before the target key frame, where N is an integer greater than or equal to 1; Calculating, according to the second gesture data, a second distance between the target fingertip key point and the corresponding first fingertip key point in each key frame; The performing a touch operation at the touch position includes: When the second distances corresponding to the first N key frames are all greater than a preset second distance threshold, performing a tap operation at the touch position; When the second distance corresponding to the first N key frames is less than or equal to a preset second distance threshold, a long press operation is performed at the touch position.
7. The method according to any one of claims 1 to 6, characterized in that: The first distance between each second fingertip key point and the corresponding first fingertip key point may be calculated by the following formula 1: In formula 1, i represents the serial number of the second fingertip key point, d i represents the first distance between the second fingertip key point with serial number i and its corresponding first fingertip key point, x i ,y i 、z i They represent the horizontal coordinate, vertical coordinate and vertical coordinate of the second fingertip key point with serial number i, respectively. f ,y f 、z f Respectively represent the horizontal coordinate, vertical coordinate and vertical coordinate of the first fingertip key point.
8. A gesture recognition device, characterized in that: include: an acquisition module, configured to acquire first gesture data corresponding to a target key frame, wherein the first gesture data includes the spatial coordinates of a first fingertip key point in the target key frame, and the spatial coordinates of a plurality of second fingertip key points corresponding to the first fingertip key point, wherein the second fingertip key points are fingertip key points belonging to the same palm as the first fingertip key point; a calculation module, configured to calculate, based on the spatial coordinates of the first fingertip key point and the spatial coordinates of the plurality of second fingertip key points, a first distance between each of the second fingertip key points and the corresponding first fingertip key point; A determination module, configured to determine, when the first distance is greater than or equal to a preset first distance threshold, the corresponding second fingertip key point as a target key point; The determination module is further configured to determine that the gesture type corresponding to the first fingertip key point is an input gesture when the number of target key points corresponding to the first fingertip key point is within a preset number range.
9. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the gesture recognition method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the gesture recognition method according to any one of claims 1 to 7 is implemented.
11. A vehicle, characterized in that: The vehicle includes the gesture recognition device according to claim 8 or the electronic device according to claim 9.