Gesture recognition method, apparatus and device, computer storage medium and vehicle

Abstract

Embodiments of the present application disclose a gesture recognition method, apparatus and device, a computer storage medium and a vehicle. The method comprises: acquiring first gesture data; on the basis of spatial coordinates of a first fingertip key point and spatial coordinates of a plurality of second fingertip key points, respectively calculating a first distance between each second fingertip key point and the corresponding first fingertip key point (S120); when the first distance is greater than or equal to a preset first distance threshold, determining that the corresponding second fingertip key point is a target key point (S130); and when the number of target key points corresponding to the first fingertip key point is within a preset number interval, determining that the gesture type corresponding to the first fingertip key point is an input gesture (S140). The embodiments of the present application can improve the accuracy of input gesture recognition, thereby improving the experience of a user interacting with an electronic device.

Description

Gesture recognition method, device, equipment, computer storage medium and vehicle

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on August 28, 2023, with application number 202311089050.7 and application name “Gesture Recognition Method, Device, Equipment, Computer Storage Medium and Vehicle”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of computer technology, and relates to, but is not limited to, a gesture recognition method, apparatus, device, computer storage medium, and vehicle. Background Art

[0004] Touch interaction brings enormous convenience to people's work and daily lives. To further facilitate user interaction with electronic devices, air touch has emerged. Air touch involves capturing hand movements through a camera, creating a virtual touch hotspot in space for touch operations. The user's actions within the hotspot control the corresponding electronic device.

[0005] Related technologies involve presetting a touch plane based on the user's hand position. When the distance between a finger joint and the preset touch plane is less than a preset distance threshold, the user is deemed to have performed an input operation. However, in some cases, the user may accidentally touch the touch plane, resulting in a misjudgment of the input operation.

[0006] With the increasing demand for interactivity, how to accurately judge user input operations has become a problem that needs to be solved urgently.

[0007] Summary of the Invention

[0008] Embodiments of the present application provide a gesture recognition method, apparatus, device, computer storage medium, and vehicle, which can improve the accuracy of input gesture recognition, thereby enhancing the user's interaction experience with the electronic device.

[0009] In a first aspect, an embodiment of the present application provides a gesture recognition method, comprising:

[0010] Acquire first gesture data, the first gesture data including spatial coordinates of a first fingertip key point and spatial coordinates of a plurality of second fingertip key points corresponding to the first fingertip key point, where the second fingertip key points are fingertip key points belonging to the same palm as the first fingertip key point;

[0011] Based on the spatial coordinates of the first fingertip key point and the spatial coordinates of the plurality of second fingertip key points, respectively calculating a first distance between each second fingertip key point and the corresponding first fingertip key point;

[0012] When the first distance is greater than or equal to a preset first distance threshold, determining the corresponding second fingertip key point as the target key point;

[0013] When the number of target key points corresponding to the first fingertip key point is within a preset number range, it is determined that the gesture type corresponding to the first fingertip key point is an input gesture.

[0014] In an optional implementation, after determining that the gesture type corresponding to the first fingertip key point is an input gesture, the method further includes:

[0015] Establish touch hot zones;

[0016] An operation corresponding to the first gesture data is performed in the touch hot zone.

[0017] In an optional embodiment, the first gesture data includes one or two first fingertip key points; the method further includes: determining the number of input gestures according to the gesture type corresponding to each first fingertip key point;

[0018] Create touch hotspots, including:

[0019] When the number of input gestures is 1, a single-hand touch hot zone is established;

[0020] When the number of input gestures is 2, a two-hand touch hot zone is established, and the two-hand touch hot zone includes touch sub-hot zones corresponding to each input gesture.

[0021] In an optional embodiment, the area of ​​the single-hand touch hot zone is smaller than the area of ​​the double-hand touch hot zone.

[0022] In an optional implementation, performing a touch operation corresponding to the first gesture data in the touch hot zone includes:

[0023] In a target key frame, when a first distance corresponding to a target fingertip key point is less than or equal to a preset second distance threshold, determining a touch position in the touch hot zone based on the spatial coordinates of the target fingertip key point, the target fingertip key point being at least one of the target key points;

[0024] Perform a touch action at the touch location.

[0025] In an optional implementation, the touch operation includes a tap operation or a long press operation; and the method further includes:

[0026] Obtain the second gesture data corresponding to each key frame in the N key frames preceding the target key frame, where N is an integer greater than or equal to 1;

[0027] Calculating, based on the second gesture data, a second distance between the target fingertip key point and the corresponding first fingertip key point in each key frame;

[0028] Perform touch actions at the touch location, including:

[0029] When the second distances corresponding to the first N key frames are all greater than a preset second distance threshold, performing a tap operation at the touch position;

[0030] When the second distance corresponding to the first N key frames is less than or equal to a preset second distance threshold, a long press operation is performed at the touch position.

[0031] In an optional embodiment, the first distance between each second fingertip key point and the corresponding first fingertip key point can be calculated using Formula 1;

[0032] In formula 1, i represents the serial number of the second fingertip key point, d i represents the first distance between the second fingertip key point with sequence number i and its corresponding first fingertip key point, x i 、y i 、z i They represent the horizontal coordinate, vertical coordinate and vertical coordinate of the second fingertip key point with serial number i, respectively. f 、y f 、z f Respectively represent the horizontal coordinate, vertical coordinate and vertical coordinate of the first fingertip key point.

[0033] In a second aspect, an embodiment of the present application provides a gesture recognition device, comprising:

[0034] an acquiring unit configured to acquire first gesture data, the first gesture data including spatial coordinates of a first fingertip key point and spatial coordinates of a plurality of second fingertip key points corresponding to the first fingertip key point, the second fingertip key points being fingertip key points belonging to the same palm as the first fingertip key point;

[0035] a calculation part configured to calculate a first distance between each second fingertip key point and the corresponding first fingertip key point based on the spatial coordinates of the first fingertip key point and the spatial coordinates of the plurality of second fingertip key points;

[0036] A determining part configured to determine the corresponding second fingertip key point as the target key point when the first distance is greater than or equal to a preset first distance threshold;

[0037] The determination part is further configured to determine that the gesture type corresponding to the first fingertip key point is an input gesture when the number of target key points corresponding to the first fingertip key point is within a preset number range.

[0038] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising: a processor and a memory storing computer program instructions;

[0039] When the processor executes the computer program instructions, the gesture recognition method of any optional implementation manner of the first aspect of the present application is implemented.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, a gesture recognition method according to any optional embodiment of the first aspect of the present application is implemented.

[0041] In a fifth aspect, an embodiment of the present application provides a vehicle, comprising: a gesture recognition device as in the second aspect or an electronic device as in the third aspect.

[0042] The gesture recognition method, apparatus, device, computer storage medium, and vehicle of the embodiments of the present application can obtain first gesture data and use the first gesture data to calculate a first distance between each second fingertip key point and the corresponding first fingertip key point. Then, if the first distance is greater than or equal to a preset first distance threshold, the corresponding second fingertip key point is determined to be a target key point. Thus, using the first fingertip key point as a reference, the first distances between the remaining fingertip key points and the first fingertip key point are calculated. By comparing the first distances corresponding to the remaining fingertip key points with the first distance threshold, the status of the remaining fingertip key points can be accurately determined. Furthermore, if the number of target key points corresponding to the first fingertip key point is within a preset range, the embodiment of the present application determines that the gesture type corresponding to the first fingertip key point is an input gesture. In this way, the palm shape can be identified based on the status of the remaining fingertip key points, thereby accurately determining whether the gesture type corresponding to the first fingertip key point is an input gesture. This improves the accuracy of input gesture recognition, thereby improving the accuracy of input operation judgment in touch interaction scenarios and enhancing the user's interactive experience with the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0044] FIG1 is a flow chart of a gesture recognition method provided by one embodiment of the present application;

[0045] FIG2 is a schematic diagram of a hand image provided by another embodiment of the present application;

[0046] FIG3 is a schematic diagram of gesture types provided by yet another embodiment of the present application;

[0047] FIG4 is a schematic diagram of a single-hand touch hot zone provided by yet another embodiment of the present application;

[0048] FIG5 is a schematic diagram of two-hand touch hot zones provided by yet another embodiment of the present application;

[0049] FIG6 is a schematic diagram of determining a touch position in a gesture recognition method provided in yet another embodiment of the present application;

[0050] FIG7 is a structural diagram of a gesture recognition device provided by yet another embodiment of the present application;

[0051] FIG8 is a schematic structural diagram of an electronic device provided in yet another embodiment of the present application. DETAILED DESCRIPTION

[0052] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0053] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0054] As described in the background, touch interaction has brought tremendous convenience to people's work and daily lives, and people's demand for interactive electronic devices is also increasing. The most common way to provide input to electronic devices such as computer systems is through a keyboard. However, keyboards are not very portable. To this end, many devices are equipped with virtual keyboards, which users use for input. Therefore, how to accurately recognize input gestures based on virtual keyboards to perform input operations has become a pressing issue.

[0055] Based on this, in order to improve the accuracy of input gesture recognition and thus enhance the user's interactive experience with electronic devices, the inventors, after in-depth thinking, cleverly designed a gesture recognition method, apparatus, device, computer storage medium and vehicle.

[0056] The gesture recognition method provided in the embodiments of the present application may be executed by a device that can be a gesture recognition device, or by a module within the gesture recognition device that is used to execute the gesture recognition method. The gesture recognition method provided in the embodiments of the present application is described in detail using the gesture recognition device executing the gesture recognition method as an example.

[0057] The following first introduces the gesture recognition method provided in the embodiment of the present application with reference to the accompanying drawings.

[0058] Figure 1 shows a flow chart of a gesture recognition method provided by an embodiment of the present application. As shown in Figure 1 , the gesture recognition method may specifically include the following steps S110 to S140.

[0059] S110, obtaining first gesture data, the first gesture data including the spatial coordinates of a first fingertip key point and the spatial coordinates of a plurality of second fingertip key points corresponding to the first fingertip key point, the second fingertip key points being fingertip key points belonging to the same palm as the first fingertip key point.

[0060] In step S110, as an example, a video stream may be acquired through a camera, a key frame may be captured from the video stream, and the first gesture data may be acquired from the key frame. The key frame may be a key frame corresponding to a preset time period. In one embodiment, the key frame may be a key frame captured at the current moment.

[0061] In step S110, the first gesture data may include gesture data obtained by recognizing the hand information of the key frame. The first fingertip key point may be a pre-selected fingertip key point, for example, it may be a thumb key point. The multiple second fingertip key points corresponding to the first fingertip key point may include the remaining fingertip key points belonging to the same palm as the first fingertip key point, for example, the index finger, middle finger, ring finger, and little finger. The first gesture data may be obtained by processing the key frame using a pre-trained neural network model. The training process of the neural network model may be implemented by methods well known in the art and will not be described in detail here. The training sample of the neural network model may be a hand image with manually annotated skeletal point information, and the skeletal point information may include the spatial coordinates of the skeletal points. As an example, the hand image with manually annotated skeletal point information may be as shown in FIG2. The above-mentioned spatial coordinates may be spatial coordinates determined based on a preset coordinate system, for example, spatial coordinates determined with the hand joint skeletal points as the origin. The spatial coordinates may be three-dimensional coordinates, which may be represented by real parameters of the physical world. As an example, an RGB image with a resolution of 640*480 can be generated based on the key frame, and the hand skeleton points in the image can be identified by the neural network model to obtain the hand skeleton point spatial coordinate dataset n:[x n ,y n ,z n ], n represents the number of the hand skeleton point. The spatial coordinates of the first fingertip key point and the spatial coordinates of multiple second fingertip key points corresponding to the first fingertip key point are determined from the hand skeleton point spatial coordinate data set to obtain the first gesture data.

[0062] S120 , based on the spatial coordinates of the first fingertip key point and the spatial coordinates of the plurality of second fingertip key points, respectively calculate a first distance between each second fingertip key point and the corresponding first fingertip key point.

[0063] In step S120 , the first distance between each second fingertip key point and the corresponding first fingertip key point can be calculated using Formula 1.

[0064] In formula 1, i represents the serial number of the second fingertip key point, d i Indicates the first distance between the second fingertip key point with sequence number i and its corresponding first fingertip key point. i 、y i 、z i They represent the horizontal coordinate, vertical coordinate and vertical coordinate of the second fingertip key point with serial number i, respectively. f 、y f 、z f Respectively represent the horizontal coordinate, vertical coordinate and vertical coordinate of the first fingertip key point.

[0065] S130: When the first distance is greater than or equal to a preset first distance threshold, determine the corresponding second fingertip key point as the target key point.

[0066] In step S130, the preset first distance threshold may be a distance threshold pre-set based on the finger position corresponding to the second fingertip key point. If the first distance between the second fingertip key point and the corresponding first fingertip key point is less than the corresponding first distance threshold, the finger corresponding to the second fingertip key point may be considered to be in a closed state; if the first distance between the second fingertip key point and the corresponding first fingertip key point is greater than or equal to the corresponding first distance threshold, the finger corresponding to the second fingertip key point may be considered to be in an open state. In one embodiment, the second fingertip key point corresponding to the finger in the open state may be used as the target key point.

[0067] S140: When the number of target key points corresponding to the first fingertip key point is within a preset number range, determine that the gesture type corresponding to the first fingertip key point is an input gesture.

[0068] In step S140, the gesture type corresponding to the first fingertip key point can be determined based on the number of fingers in an open state corresponding to the first fingertip key point. In one embodiment, based on the number of fingers in an open state corresponding to the first fingertip key point, the gesture type can be divided into input gestures and non-input gestures according to preset rules. The preset rules may include: when the number of fingers in an open state corresponding to the first fingertip key point is within a preset number interval, determining that the hand corresponding to the first fingertip key point is an input gesture; when the number of fingers in an open state corresponding to the first fingertip key point is not within the preset number interval, determining that the hand corresponding to the first fingertip key point is a non-input gesture. The preset number interval can be set according to actual needs and is not limited in this application. Figure 3 shows a schematic diagram of gesture types provided by an embodiment of the present application. As shown in Figure 3, taking the thumb key point as the first fingertip key point as an example, in the same palm, when the remaining fingers except the thumb are all in an open state or all in a closed state, the user generally does not need touch input, so the corresponding gesture type is a non-input gesture. When some fingers (such as the index finger) among the fingers other than the thumb are in an open state, the user may need touch input, so the corresponding gesture type is an input gesture.

[0069] The gesture recognition method of an embodiment of the present application can obtain first gesture data corresponding to a keyframe and use the first gesture data to calculate a first distance between each second fingertip key point and the corresponding first fingertip key point. Then, if the first distance is greater than or equal to a preset first distance threshold, the corresponding second fingertip key point is determined to be a target key point. Thus, using the first fingertip key point as a reference, the first distances between the remaining fingertip key points and the first fingertip key point are calculated. By comparing the first distances corresponding to the remaining fingertip key points with the first distance threshold, the status of the remaining fingertip key points can be accurately determined. Furthermore, if the number of target key points corresponding to the first fingertip key point is within a preset range, the embodiment of the present application determines that the gesture type corresponding to the first fingertip key point is an input gesture. Based on the number of target key points corresponding to the first fingertip key point, gesture information is generated for the keyframe, including the gesture type. In this way, the palm shape can be identified based on the status of the remaining fingertip key points, thereby accurately determining whether the keyframe contains an input gesture. If the gesture type includes an input gesture, a touch hotspot is established, and a touch operation corresponding to the first gesture data is performed within the touch hotspot. In this way, the accuracy of touch processing input gesture recognition can be improved, which is conducive to improving the accuracy of input operation judgment in touch interaction scenarios and enhancing the interaction experience between users and electronic devices.

[0070] In one embodiment, after determining that the gesture type corresponding to the first fingertip key point is an input gesture, the method may further include:

[0071] Create touch hot zones.

[0072] An operation corresponding to the first gesture data is performed in the touch hot zone.

[0073] In the above-described embodiment, establishing a touch hotspot can be achieved using methods known in the art and are not limited herein. For example, a touch hotspot can be established based on the spatial coordinates of preset reference points in a keyframe and a preset touch hotspot size. Preset reference points may include, but are not limited to, wrist joint key points, user head key points, and user shoulder key points.

[0074] In the above embodiment, after determining that the first fingertip key point is an input gesture, a touch hotspot is established, and the operation corresponding to the first gesture data is performed in the touch hotspot. In this way, the touch hotspot can be established when it is determined that the user needs to perform an input operation, thereby reducing the situation where the touch hotspot is established when the user does not need to perform an input operation, thereby improving the user's interactive experience with the electronic device.

[0075] In one embodiment, the first gesture data may include one or two first fingertip key points. The method may further include: determining the number of input gestures according to the gesture type corresponding to each first fingertip key point.

[0076] Establishing touch hotspots can include:

[0077] When the number of input gestures is 1, a single-hand touch hot zone is established.

[0078] When the number of input gestures is 2, a two-hand touch hot zone is established, and the two-hand touch hot zone includes touch sub-hot zones corresponding to each input gesture.

[0079] In the above embodiment, the first gesture data may include one or two palms, and accordingly may include one or two first fingertip key points, such as two thumb key points. The above embodiment may determine the gesture type corresponding to each first fingertip key point based on the number of target key points corresponding to each first fingertip key point, thereby determining the number of input gestures.

[0080] When establishing a touch hotspot, the size and shape of the touch hotspot can be adjusted based on the number of input gestures. As an example, a single-hand touch hotspot can be shown in Figure 4, and a two-hand touch hotspot can be shown in Figure 5. As shown in Figure 5, within the two-hand touch hotspot, the touch sub-hotspots corresponding to the two hands can be separated from each other.

[0081] Since keyboard operation is usually done with two hands, when both hands are used in the same hot zone at the same time, the accuracy of the computer vision reasoning algorithm and the angle of the human body may misjudge the clicking hand, resulting in operation errors.

[0082] The above embodiment establishes a corresponding one-hand touch hot zone or a two-hand touch hot zone according to the number of input gestures, so that the type of touch hot zone can be flexibly adjusted according to the number of hands that the user actually performs touch operations. In this way, in the case of one-hand input (only one hand appears in the key frame, or both hands appear in the key frame but only one hand is in the input gesture), a complete keyboard hot zone can be established for a single input gesture; in the case of two-hand input (both hands appear in the key frame and both are input gestures), a touch sub-hot zone is established based on each input gesture in both hands to achieve simultaneous operation of both hands. In this way, when the spatial gesture keyboard is manipulated by both hands, different areas can be controlled by both hands to reduce the risk of misjudgment and improve the accuracy of touch processing; at the same time, because the touch hot zones of both hands are separated, there is no need to consider the distance between the hands. The user can choose a comfortable posture to operate, such as when sitting, placing both elbows on the armrests of the chair to operate. In this way, the flexibility of touch processing is improved, and the user's experience of interacting with the electronic device is further improved.

[0083] In one embodiment, the area of ​​the single-hand touch hot zone may be smaller than the area of ​​the double-hand touch hot zone.

[0084] Because keyboard operation is often performed with two hands, and 26-key keyboards have many keys, using fixed-size or fixed-shape touch hotspots makes it difficult to accommodate both one-handed and two-handed operation. For example, a virtual keyboard with a larger touch hotspot suitable for two-handed operation creates a dead zone when operated with one hand; a smaller touch hotspot suitable for one-handed operation results in densely distributed keys, making it easy to misoperate. Furthermore, using key combinations with a single-handed touch hotspot is extremely difficult, making it difficult to simultaneously press the shift key and a letter key to switch between uppercase and lowercase letters.

[0085] In the above embodiment, in the case of one-handed input, a smaller, complete keyboard hot zone is established for a single input gesture; in the case of two-handed input, a touch sub-hot zone is established for each input gesture in each hand, forming two larger half-keyboard hot zones, enabling simultaneous two-handed operation, continuous input, and combination key input. This can improve the accuracy of touch processing in both one-handed and two-handed input scenarios.

[0086] In one embodiment, performing a touch operation corresponding to the first gesture data in the touch hot zone may specifically include:

[0087] In the target key frame, when the first distance corresponding to the target fingertip key point is less than or equal to the preset second distance threshold, the touch position is determined in the touch hot zone based on the spatial coordinates of the target fingertip key point, and the target fingertip key point is at least one of the target key points.

[0088] Perform a touch action at the touch location.

[0089] The preset second distance threshold can be a pre-set threshold for characterizing the bending state of the finger corresponding to the target fingertip key point. When the distance between the target fingertip key point and the first fingertip key point is less than or equal to the corresponding second distance threshold, the finger corresponding to the target fingertip key point can be considered to be in a bent state. In this case, it can be considered that the finger is performing a touch operation. Determining the touch position in the touch hot zone based on the spatial coordinates of the target fingertip key point can include determining the touch position in the touch hot zone based on a mapping relationship between the spatial coordinates and the touch hot zone. As an example, as shown in Figure 6, in the case of single-handed touch input, the touch position (e.g., the k key) can be determined in the single-handed touch hot zone based on the spatial coordinates of the target fingertip key point (e.g., the index finger); in the case of double-handed touch input, the touch position (e.g., the d key, the p key) can be determined in the double-handed touch hot zone based on the spatial coordinates of the target fingertip key point (e.g., the index fingers of both hands).

[0090] This embodiment can determine the touch position in the touch hotspot based on the spatial coordinates of the target fingertip key point when the finger corresponding to the target finger key point performs a touch operation, so as to perform the corresponding touch operation at the touch position. This helps to further improve the accuracy of touch processing.

[0091] In one embodiment, the touch operation may include a tap operation or a long press operation.

[0092] The method may further include:

[0093] The second gesture data corresponding to each key frame in N key frames before the target key frame is obtained, where N is an integer greater than or equal to 1.

[0094] According to the second gesture data, a second distance between the target fingertip key point and the corresponding first fingertip key point in each key frame is calculated respectively.

[0095] Perform touch operations at the touch location, which may include:

[0096] When the second distances corresponding to the first N key frames are all greater than the preset second distance threshold, a tap operation is performed at the touch position.

[0097] When the second distance corresponding to the first N key frames is less than or equal to a preset second distance threshold, a long press operation is performed at the touch position.

[0098] As an example, after the touch hot zone is established, a video stream can be obtained through a camera, and a target key frame can be captured from the video stream.

[0099] In the above embodiment, the state information of the target fingertip key point in the target key frame can be determined based on the N historical key frames preceding the target key frame. For example, if the second distances corresponding to the N historical key frames preceding the target key frame are all greater than a preset second distance threshold, it can be assumed that the finger corresponding to the target fingertip key point did not perform a touch operation before the target key frame. In the target key frame, the state information of the target fingertip key point can include that the finger corresponding to the target fingertip key point changes from an extended state to a bent state, in which case it can be assumed that the finger corresponding to the target fingertip key point performed a tap operation. If the second distances corresponding to the N historical key frames preceding the target key frame are all less than or equal to the preset second distance threshold, it can be assumed that the finger corresponding to the target fingertip key point performed a touch operation before the target key frame. In the target key frame, the state information of the target fingertip key point can include that the finger corresponding to the target fingertip key point remains in a bent state, in which case it can be assumed that the finger corresponding to the target fingertip key point performed a long press operation. In this way, the dynamic changes of the finger corresponding to the target fingertip key point can be determined based on the historical key frames and the current key frame, so that the corresponding touch operation can be performed, thereby improving the accuracy of touch processing.

[0100] As shown in FIG. 7 , the gesture recognition device 200 may include an acquisition part 201 , a calculation part 202 , and a determination part 203 .

[0101] The acquisition part 201 is configured to acquire first gesture data, which includes the spatial coordinates of a first fingertip key point and the spatial coordinates of multiple second fingertip key points corresponding to the first fingertip key point, where the second fingertip key point is a fingertip key point belonging to the same palm as the first fingertip key point.

[0102] The calculation part 202 is configured to calculate the first distance between each second fingertip key point and the corresponding first fingertip key point based on the spatial coordinates of the first fingertip key point and the spatial coordinates of the plurality of second fingertip key points.

[0103] The determining part 203 is configured to determine the corresponding second fingertip key point as the target key point when the first distance is greater than or equal to a preset first distance threshold.

[0104] The determining part 203 is further configured to determine that the gesture type corresponding to the first fingertip key point is an input gesture when the number of target key points corresponding to the first fingertip key point is within a preset number range.

[0105] The gesture recognition device of an embodiment of the present application can obtain first gesture data corresponding to a keyframe and use the first gesture data to calculate a first distance between each second fingertip key point and the corresponding first fingertip key point. Then, if the first distance is greater than or equal to a preset first distance threshold, the corresponding second fingertip key point is determined to be a target key point. Thus, using the first fingertip key point as a reference, the first distances between the remaining fingertip key points and the first fingertip key point are calculated. By comparing the first distances corresponding to the remaining fingertip key points with the first distance threshold, the status of the remaining fingertip key points can be accurately determined. Furthermore, if the number of target key points corresponding to the first fingertip key point is within a preset range, the embodiment of the present application determines that the gesture type corresponding to the first fingertip key point is an input gesture. Based on the number of target key points corresponding to the first fingertip key point, gesture information is generated for the keyframe, including the gesture type. In this way, the palm shape can be identified based on the status of the remaining fingertip key points, thereby accurately determining whether the keyframe contains an input gesture. If the gesture type is an input gesture, a touch hotspot is established, and a touch operation corresponding to the first gesture data is performed within the touch hotspot. In this way, the accuracy of touch processing input gesture recognition can be improved, which is conducive to improving the accuracy of input operation judgment in touch interaction scenarios and enhancing the interaction experience between users and electronic devices.

[0106] In one embodiment, the gesture recognition device may further include:

[0107] The establishment part is configured to establish the touch hot zone.

[0108] The execution part is configured to execute an operation corresponding to the first gesture data in the touch hot zone.

[0109] In one embodiment, the first gesture data may include one or two first fingertip key points. The gesture recognition device may further include: a determination unit configured to determine the number of input gestures according to the gesture type corresponding to each first fingertip key point.

[0110] The setup section is to create touch hotspots, which may include:

[0111] The establishing part is configured to establish a one-hand touch hot zone when the number of input gestures is 1.

[0112] The establishing part is configured to establish a two-hand touch hot zone when the number of input gestures is 2, and the two-hand touch hot zone includes a touch sub-hot zone corresponding to each input gesture.

[0113] In one embodiment, the area of ​​the single-hand touch hot zone may be smaller than the area of ​​the double-hand touch hot zone.

[0114] In one embodiment, the execution part is configured to execute the touch operation corresponding to the first gesture data in the touch hot zone, which may specifically include:

[0115] The determination part is configured to determine the touch position in the touch hot zone based on the spatial coordinates of the target fingertip key point when the first distance corresponding to the target fingertip key point in the target key frame is less than or equal to a preset second distance threshold, and the target fingertip key point is at least one of the target key points.

[0116] The execution part is configured to execute the touch operation at the touch position.

[0117] In one embodiment, the touch operation may include a tap operation or a long press operation.

[0118] The gesture recognition device may further include:

[0119] The acquisition part is configured to acquire the second gesture data corresponding to each key frame in the first N key frames of the target key frame, where N is an integer greater than or equal to 1.

[0120] The calculation part is configured to calculate the second distance between the target fingertip key point and the corresponding first fingertip key point in each key frame according to the second gesture data.

[0121] The execution part is configured to perform the touch operation at the touch position, which may specifically include:

[0122] The execution part is configured to perform a tap operation at the touch position when the second distances corresponding to the first N key frames are all greater than a preset second distance threshold.

[0123] The execution part is configured to perform a long press operation at the touch position when the second distance corresponding to the first N key frames is less than or equal to a preset second distance threshold.

[0124] The gesture recognition device provided in the embodiment of the present application can implement each process implemented in the method embodiment of FIG1 , and will not be described again here to avoid repetition.

[0125] FIG8 shows a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0126] The electronic device may include a processor 301 and a memory 302 storing computer program instructions.

[0127] Specifically, the processor 301 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0128] The memory 302 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 302 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 302 may include removable or non-removable (or fixed) media. Where appropriate, the memory 302 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 302 is a non-volatile solid-state memory.

[0129] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present application.

[0130] The processor 301 reads and executes computer program instructions stored in the memory 302 to implement any one of the gesture recognition methods in the above embodiments.

[0131] As an example, the electronic device may further include a communication interface 303 and a bus 310. As shown in FIG8, the processor 301, the memory 302, and the communication interface 303 are connected via the bus 310 and communicate with each other.

[0132] The communication interface 303 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0133] Bus 310 includes hardware, software or both, and the components of online data flow metering equipment are coupled to each other. For example, but not limitation, bus can include accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 310 can include one or more buses. Although the present application embodiment describes and shows specific bus, the application considers any suitable bus or interconnection.

[0134] The electronic device can execute the gesture recognition method in the embodiment of the present application, thereby realizing the gesture recognition method and device described in conjunction with FIG. 1 and FIG. 7 .

[0135] In addition, in conjunction with the gesture recognition method in the above embodiments, embodiments of the present application may provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the gesture recognition methods in the above embodiments is implemented.

[0136] In addition, in combination with the gesture recognition method in the above embodiment, the present invention can provide a vehicle to implement the gesture recognition method, wherein the vehicle includes the gesture recognition device or electronic device in the above embodiment.

[0137] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0138] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0139] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0140] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or flowchart and the combination of the boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0141] The above description is only a specific embodiment of the present application. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the scope of protection of the present application. Industrial Applicability

[0142] Embodiments of the present application provide a gesture recognition method, apparatus, device, computer storage medium, and vehicle. The gesture recognition method includes: obtaining first gesture data; calculating a first distance between each second fingertip key point and its corresponding first fingertip key point based on the spatial coordinates of a first fingertip key point and the spatial coordinates of multiple second fingertip key points; determining the corresponding second fingertip key point as a target key point when the first distance is greater than or equal to a preset first distance threshold; and determining the gesture type corresponding to the first fingertip key point as an input gesture when the number of target key points corresponding to the first fingertip key point is within a preset number range. Implementing the above method can improve the accuracy of input gesture recognition, thereby enhancing the user's experience interacting with the electronic device.

Claims

1. A gesture recognition method, the method comprising: Acquire first gesture data, the first gesture data including spatial coordinates of a first fingertip key point and spatial coordinates of a plurality of second fingertip key points corresponding to the first fingertip key point, the second fingertip key points being fingertip key points belonging to the same palm as the first fingertip key point; Based on the spatial coordinates of the first fingertip key point and the spatial coordinates of the plurality of second fingertip key points, respectively calculating a first distance between each of the second fingertip key points and the corresponding first fingertip key point; When the first distance is greater than or equal to a preset first distance threshold, determining the corresponding second fingertip key point as a target key point; When the number of target key points corresponding to the first fingertip key point is within a preset number range, it is determined that the gesture type corresponding to the first fingertip key point is an input gesture.

2. The method according to claim 1, wherein: After determining that the gesture type corresponding to the first fingertip key point is an input gesture, the method further includes: Establish touch hotspots; An operation corresponding to the first gesture data is performed in the touch hot zone.

3. The method according to claim 2, wherein: The first gesture data includes one or two first fingertip key points; The method further includes: determining the number of input gestures according to the gesture type corresponding to each first fingertip key point; The step of establishing a touch hot zone includes: When the number of the input gestures is 1, a single-hand touch hot zone is established; When the number of the input gestures is 2, a two-hand touch hot zone is established, and the two-hand touch hot zone includes a touch sub-hot zone corresponding to each input gesture.

4. The method according to claim 3, wherein: The area of ​​the single-hand touch hot zone is smaller than the area of ​​the double-hand touch hot zone.

5. The method according to claim 2, wherein: The performing the touch operation corresponding to the first gesture data in the touch hot zone includes: In a target key frame, when the first distance corresponding to a target fingertip key point is less than or equal to a preset second distance threshold, determining the touch position in the touch hot zone based on the spatial coordinates of the target fingertip key point, the target fingertip key point being at least one of the target key points; A touch operation is performed at the touch position.

6. The method according to claim 5, wherein: The touch operation includes a tap operation or a long press operation; the method further includes: Obtaining second gesture data corresponding to each key frame in N key frames before the target key frame, where N is an integer greater than or equal to 1; Calculating, according to the second gesture data, a second distance between the target fingertip key point and the corresponding first fingertip key point in each key frame; The performing a touch operation at the touch position includes: When the second distances corresponding to the first N key frames are all greater than a preset second distance threshold, performing a tap operation at the touch position; When the second distance corresponding to the first N key frames is less than or equal to a preset second distance threshold, a long press operation is performed at the touch position.

7. The method according to any one of claims 1 to 6, wherein: Calculate the first distance between each second fingertip key point and the corresponding first fingertip key point according to Formula 1; In formula 1, i represents the serial number of the second fingertip key point, d i represents the first distance between the second fingertip key point with serial number i and its corresponding first fingertip key point, x i ,y i 、z i They represent the horizontal coordinate, vertical coordinate and vertical coordinate of the second fingertip key point with serial number i, respectively. f ,y f 、z f Respectively represent the horizontal coordinate, vertical coordinate and vertical coordinate of the first fingertip key point.

8. A gesture recognition device, comprising: an acquisition part, configured to acquire first gesture data, the first gesture data including spatial coordinates of a first fingertip key point and spatial coordinates of a plurality of second fingertip key points corresponding to the first fingertip key point, the second fingertip key points being fingertip key points belonging to the same palm as the first fingertip key point; a calculation part, configured to calculate a first distance between each of the second fingertip key points and the corresponding first fingertip key point based on the spatial coordinates of the first fingertip key point and the spatial coordinates of the plurality of second fingertip key points; A determination part, configured to determine the corresponding second fingertip key point as a target key point when the first distance is greater than or equal to a preset first distance threshold; The determining part is further configured to determine that the gesture type corresponding to the first fingertip key point is an input gesture when the number of target key points corresponding to the first fingertip key point is within a preset number range.

9. An electronic device, comprising: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the gesture recognition method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the gesture recognition method according to any one of claims 1 to 7.

11. A vehicle, comprising the gesture recognition device according to claim 8 or the electronic device according to claim 9.

Images