Gesture recognition method, and electronic device

Abstract

The present application relates to the technical field of human-computer interaction. Provided are a gesture recognition method, and an electronic device. The gesture recognition method comprises: in response to a user operation, acquiring gesture sensing information, wherein the user operation is used for triggering gesture control; on the basis of the gesture sensing information, acquiring a first gesture; and reporting to a first application information for describing the first gesture, such that the first application executes a first operation corresponding to the first gesture, wherein the first application is an application subscribing to the first gesture. Compared with the method for performing gesture recognition by an application itself, the method in the present application involves performing gesture recognition in a unified manner by a first system service in an electronic device, which helps to prevent collisions among a plurality of applications during gesture recognition, thereby reducing the number of instances of repeated gesture recognition in the electronic device, and helping to reduce the waste of computing power of the electronic device.

Description

Gesture recognition method and electronic device

[0001] The present application claims priority to the Chinese patent application No. 202411394419.X, filed on September 30, 2024, and entitled "Gesture recognition method and electronic device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of human-computer interaction, and in particular to a gesture recognition method and an electronic device. BACKGROUND

[0003] The functions supported by applications (such as system native applications and third-party applications) can be triggered by gestures. The gesture recognition process is used to identify the gestures corresponding to user operations.

[0004] Taking a third-party application as an example, in the gesture recognition process, the electronic device obtains gesture sensing information and sends the gesture sensing information to the third-party application that is running. According to the gesture sensing information, the gesture recognition logic inside the third-party application predicts the corresponding gesture and executes the control event corresponding to the gesture.

[0005] However, the gesture recognition processes of applications can conflict with each other, wasting the computing power of the electronic device. SUMMARY

[0006] To solve the above technical problems, the present application provides a gesture recognition method and an electronic device. The technical solution provided by the present application completes gesture recognition through a first system service in the electronic device. By processing gesture sensing information, a first gesture is obtained, and an application subscribing to the first gesture is notified to execute an operation corresponding to the first gesture. By taking over the gesture recognition process in the electronic device by the system service, redundancy in gesture recognition of multiple applications caused by independent gesture recognition of applications can be avoided as much as possible, which helps to save the computing power of the electronic device.

[0007] In a first aspect, the gesture recognition method can be applied to an electronic device. Here, the gesture recognition method executed by a first system service in the electronic device is taken as an example for description, which includes: in response to a user operation, obtaining gesture sensing information, the user operation being used to trigger gesture control; according to the gesture sensing information, obtaining a first gesture; reporting information describing the first gesture to a first application, so that the first application executes a first operation corresponding to the first gesture, the first application being an application subscribing to the first gesture.

[0008] In the embodiments of the present application, any gesture recognition is taken over by a unified system service. Compared with the manner that an application completes gesture recognition by itself, on the one hand, it helps to avoid multiple applications simultaneously performing gesture recognition on the same gesture to be recognized, leading to conflicts in gesture recognition among multiple applications. For example, the related art may have a case where multiple applications perform gesture recognition based on the same gesture sensing information, leading to multiple times of gesture recognition of the same user operation by the electronic device. However, according to the usual use habit, one user operation is performed for one application. Therefore, there is an unnecessary gesture recognition process in the multiple times of gesture recognition, that is, there is redundancy among gesture recognitions performed by multiple applications respectively.

[0009] In the embodiments of the present application, gesture recognition is completed by a fixed system service, and gesture recognition for the same gesture will not be redundant, which solves the problem that independent gesture recognition by an application is prone to conflict, helps to reduce repeated gesture recognition processes, and thus reduces the algorithmic power occupation of gesture recognition on the electronic device and reduces the occurrence of freezing of the electronic device.

[0010] On the other hand, after recognizing the gesture according to the gesture sensing information, the electronic device performs a corresponding operation through an application subscribing to the gesture. By separating the subject of gesture recognition and the subject of performing the corresponding operation of gesture control, the user operation for triggering gesture control has little impact on other applications in the electronic device that do not subscribe to the gesture, which helps to reduce the interference of gesture recognition on other irrelevant applications in the electronic device.

[0011] In a possible implementation manner, after obtaining the first gesture according to the at least one type of gesture factor, the method further includes: in response to a user operation, obtaining updated gesture sensing information; obtaining the motion parameter of the first gesture according to the updated gesture sensing information; and reporting the motion parameter of the first gesture to the first application, so that the first application continues to perform the first operation.

[0012] In this way, after the corresponding gesture is recognized, the motion parameter of the first gesture is determined through the gesture sensing information, so that the application adjusts the implementation effect of the first operation according to the motion parameter. In this way, it helps to achieve the implementation effect of the first operation, which changes flexibly with the change of the user operation, and is easy to achieve more precise gesture control.

[0013] In a possible implementation, the motion parameter of the first gesture is related to a gesture control type to which the first gesture belongs; in a case where the first gesture belongs to a sliding control type, the motion parameter of the first gesture is used to indicate a change in a sliding distance; or, in a case where the first gesture belongs to a zooming control type, the motion parameter of the first gesture is used to indicate a change in a zooming ratio; or, in a case where the first gesture belongs to a rotating control type, the motion parameter of the first gesture is used to indicate a change in a rotating angle. The motion parameter is used to control an implementation effect of the first operation in a process of the user operation.

[0014] In a possible implementation, before the gesture sensing information is acquired in response to the user operation, the method further includes: acquiring subscription information sent by the first application, the subscription information being used to indicate at least one gesture supported by the first application, the at least one gesture including the first gesture.

[0015] In this way, the application notifies the first system service of at least one gesture supported by the application through the subscription information in advance, so that the electronic device can normally respond to gesture control after the function module takes over gesture recognition. In a case where the gesture subscribed by the application is recognized, the function module notifies the application to complete a control event corresponding to the gesture, thereby achieving the effect of isolating gesture recognition and gesture control from each other through the function module, and helping to reduce interference caused by the gesture recognition process on the application.

[0016] In a possible implementation, the first gesture is acquired according to the gesture sensing information, including: acquiring at least one type of gesture factor according to the gesture sensing information, each type of gesture factor being used to describe a gesture parameter in a corresponding dimension; and acquiring the first gesture according to the at least one type of gesture factor.

[0017] In this way, different gesture control types can be distinguished through the gesture factor. In an embodiment corresponding to the implementation, gesture factors used to distinguish various gesture control types overlap, thereby enabling gesture recognition logic to adapt to gesture recognition requirements of various gesture control types by integrating gesture recognition processes of different gesture control types into one gesture recognition logic through the gesture factor.

[0018] In addition, in the gesture recognition process, at least one type of gesture factor is acquired according to the gesture sensing information, and the first gesture is determined through the at least one type of gesture factor. The acquired gesture factor includes a gesture factor shared by multiple gesture control types. That is, at least one type of gesture factor is shared by multiple gesture control types in the gesture recognition method. Compared with separately designing gesture recognition logic for each type of gesture or each gesture, the embodiment reuses the gesture factor in the gesture recognition process, which helps to reduce the calculation amount of the electronic device in the gesture recognition process, thereby helping to improve the efficiency of gesture recognition.

[0019] Moreover, this way has better compatibility for identifying different gestures. After introducing a new gesture, this way supports determining the gesture control type to which the new gesture belongs, and adding the new gesture to the gesture control type, thereby completing the expansion of gesture recognition logic, and thus helping to improve the adaptability of gesture recognition logic to different use scenarios.

[0020] In a possible implementation, the first gesture is acquired according to the at least one type of gesture factor, including: acquiring a first gesture attribute according to the at least one type of gesture factor, the first gesture attribute being used to represent a first gesture control type to which the first gesture belongs; acquiring at least one second gesture attribute according to the at least one type of gesture factor, the second gesture attribute being used to represent an attribute parameter associated with the first gesture control type; and acquiring the first gesture according to the first gesture attribute and the at least one second gesture attribute.

[0021] In this way, the first gesture is determined through the at least one gesture factor respectively determining the gesture control type and the specific gesture parameter. In the embodiment corresponding to the present implementation, the factors determining the first gesture are split, and the determination processes of each gesture attribute obtained by splitting are independent of each other. After a new gesture is applied, the gesture recognition method only needs to adjust and expand the branch of the method for determining part of the gesture attributes in the gesture recognition method, so as to be compatible with recognizing the new gesture, thereby helping to improve the gesture recognition method to be compatible with more gestures and improve the universality of the gesture recognition method.

[0022] In a possible implementation, the at least one type of gesture factor includes multiple types of gesture factors with competitive priorities, the competitive priorities being used to represent the order in which the multiple types of gesture factors participate in gesture recognition; and the first type is acquired according to the at least one type of gesture factor, including: acquiring a first gesture factor from the multiple types of gesture factors, the first gesture factor being a gesture factor with a higher competitive priority in the multiple types of gesture factors; and acquiring a first gesture attribute according to a gesture control type corresponding to the first gesture factor, in a case where the first gesture factor satisfies a corresponding determination condition.

[0023] In this way, the order in which the gesture factors are used in the process of determining the gesture control type is indicated through the competitive priorities, so as to ensure that the process of determining the gesture control type is performed in an orderly manner. In the present embodiment, the branch of determining the gesture control type is determined through the satisfaction of the corresponding determination condition by the gesture factor, and the gesture control type to which the first gesture belongs is acquired by preferentially using the gesture factor with a higher competitive priority. In a case where the gesture factor with a higher competitive priority satisfies the corresponding determination condition, the gesture factors with lower competitive priorities do not need to be used, and thus it is helpful to simplify the determination logic of the gesture control type.

[0024] In addition, the gesture factors are selected according to the competition priority, so that the gesture recognition process supports the gesture factors to be acquired in stages. If the gesture factor with a higher competition priority does not meet the corresponding determination condition, the gesture factor with a lower competition priority is determined again, so as to reduce the total number of gesture factors to be acquired, and the efficiency of gesture recognition is improved.

[0025] In a possible implementation, the at least one type of gesture factor includes a plurality of types of gesture factors with competition priorities, the competition priorities being used to represent the order in which the plurality of types of gesture factors participate in gesture recognition; and the first gesture attribute is acquired according to the at least one type of gesture factor, including: acquiring a first gesture factor from the plurality of types of gesture factors, the first gesture factor being a gesture factor with a highest competition priority in the plurality of types of gesture factors; and in a case where the first gesture factor does not meet a corresponding determination condition, acquiring a second gesture factor from the plurality of types of gesture factors, the second gesture factor being a gesture factor with the highest competition priority in gesture factors that have not participated in gesture recognition among the plurality of types of gesture factors; and in a case where the second gesture factor meets the corresponding determination condition, determining the first gesture attribute according to a gesture control type corresponding to the second gesture factor.

[0026] In this way, it is ensured that each gesture factor in the plurality of types of gesture factors participates in gesture recognition only once, and the same gesture factor is not repeatedly judged, so that the gesture control type is not determined due to repeated use of the gesture factor, and the stability of the gesture recognition logic is ensured.

[0027] In a possible implementation, the at least one type of gesture factor includes a plurality of types of gesture factors with competition priorities; and the first gesture attribute is acquired according to the at least one type of gesture factor, including: acquiring a target gesture factor, the target gesture factor being a gesture factor with a highest competition priority in the plurality of types of gesture factors; determining whether the target gesture factor meets a corresponding determination condition; in a case where the target gesture factor meets the corresponding determination condition, determining the first gesture attribute according to a gesture control type corresponding to the target gesture factor, where the first gesture attribute is used to represent the gesture control type corresponding to the target gesture factor; and in a case where the target gesture factor does not meet the corresponding determination condition, updating the target gesture factor to be a gesture factor with the highest competition priority in gesture factors that have not participated in gesture recognition among the plurality of types of gesture factors, and repeatedly performing the operation of determining whether the target gesture factor meets the corresponding determination condition until the first gesture control type is determined, or all gesture factors in the plurality of types of gesture factors have participated in gesture recognition.

[0028] In this way, according to the competitive priorities among the plurality of gesture factors, whether each gesture factor meets the corresponding determination condition is verified one by one. In the case that the target gesture factor meets the determination condition, the first gesture attribute can be determined. In this way, the number of times of determining whether the gesture factor meets the determination condition is reduced, and thus the efficiency of determining the first gesture attribute is improved.

[0029] In a possible implementation, the plurality of gesture factors includes a direction factor and a barycenter factor; the competitive priority of the direction factor is higher than the competitive priority of the barycenter factor; the gesture control type corresponding to the direction factor is a sliding control type, and the gesture control type corresponding to the barycenter factor is a rotating control type; or the gesture control type corresponding to the direction factor is a sliding control type, and the gesture control type corresponding to the barycenter factor is a zooming control type; wherein the direction factor is used to represent the moving direction of at least one gesture recognition point indicated by the gesture sensing information, and the barycenter factor is used to represent the distance between at least one gesture recognition point and a geometric barycenter in the case that the total number of gesture recognition points is greater than 2, the geometric barycenter being the barycenter of a geometric figure with at least one gesture recognition point as a vertex. Optionally, the determination condition corresponding to the direction factor includes that there are more than or equal to a first number of gesture recognition points, each of which corresponds to the same direction factor; and / or the determination condition corresponding to the barycenter factor includes that there are a second number of gesture recognition points, and for each gesture recognition point in the second number of gesture recognition points, the change amount of the barycenter factor corresponding to the gesture recognition point is greater than or equal to a first distance threshold.

[0030] In this way, since the direction factor is simpler to obtain than the barycenter factor, by setting the competitive priority of the direction factor to be higher than the competitive priority of the barycenter factor, the usage rate of the direction factor in determining the gesture control type is improved, and the usage rate of the barycenter factor is reduced (for example, for a gesture belonging to the sliding control type, the electronic device does not need to additionally obtain the barycenter factor, since the gesture can be determined to belong to the sliding control type by using the direction factor), and the computing pressure of the electronic device is reduced.

[0031] In a possible implementation, the first gesture attribute is determined according to at least one gesture factor, and the method further includes: obtaining a third gesture factor from the at least one gesture factor, the third gesture factor being used to verify the validity of the user operation; and in the case that the third gesture factor meets the validity condition, obtaining the first gesture attribute used to represent the gesture control type corresponding to the selected gesture factor, the selected gesture factor being the first gesture factor or the second gesture factor.

[0032] The third gesture factor is used to verify validity of the user operation, so that gesture recognition is performed when the user operation has a certain displacement or has a certain speed. The third gesture factor can filter out invalid user operations with small movement amplitudes, thereby helping to improve the accuracy of gesture recognition. In addition, in this way, it is also helpful to avoid the electronic device from identifying a user's accidental touch as a user operation, thereby helping to avoid the first system service from notifying the first application to perform the first operation when the user's accidental touch causes the first operation to be performed unnecessarily, and causing the electronic device to operate abnormally.

[0033] In a possible implementation, the third gesture factor includes at least one of the following: a distance factor used to represent a movement distance of the at least one gesture recognition point; and a speed factor used to represent a movement speed of the at least one gesture recognition point.

[0034] Optionally, the validity condition corresponding to the speed factor includes that the speed factor corresponding to each of the at least one gesture factor is greater than or equal to a speed threshold, and / or the validity condition corresponding to the distance factor includes that the distance factor corresponding to each of the at least one gesture factor is greater than or equal to a second distance threshold.

[0035] The speed factor and the distance factor can accurately reflect the movement of a certain period of user operation. Therefore, setting the speed factor and / or the distance factor as the third gesture factor helps to accurately reflect whether an effective user operation occurs in time.

[0036] In a possible implementation, the first gesture is obtained according to the first gesture attribute and the at least one second gesture attribute, including: continuously obtaining n predicted gestures according to the corresponding first gesture attribute and the at least one second gesture attribute during the duration of the user operation, n being a positive integer; and obtaining the first gesture according to the n predicted gestures when the n predicted gestures satisfy an accuracy condition.

[0037] In a case where the to-be-recognized gesture is continuously determined as the same gesture for multiple times, the gesture is determined as the first gesture, which helps to improve the accuracy of gesture recognition.

[0038] In a possible implementation, the at least one second gesture attribute corresponding to the first gesture control type includes at least one of: a total number of gesture recognition points indicated by the gesture sensing information, a motion attribute; the first gesture attribute includes at least one of: a sliding control type, a zooming control type, and a rotating control type; the motion attribute corresponding to the sliding control type includes a sliding direction attribute, the motion attribute corresponding to the zooming control type includes a zooming trend attribute, and the motion attribute corresponding to the rotating control type includes a rotating direction attribute; and the total number of gesture recognition points is at least one of: one finger, two fingers, three fingers, four fingers, and five fingers, the sliding direction attribute is at least one of: up, down, left, right, left up, right up, left down, and right down, the zooming trend attribute is at least one of: opening and pinching, and the rotating direction attribute is at least one of: clockwise and counterclockwise.

[0039] In this way, the first gesture attribute and the at least one second gesture attribute can be used to describe a variety of gestures, which helps to improve the recognition ability of the gesture recognition method for different gestures.

[0040] In a possible implementation, the information used to describe the first gesture includes first information and second information, the first information is determined according to the first gesture attribute and the motion attribute, and the second information is the total number of gesture recognition points.

[0041] In this way, the first information is obtained by combining the first gesture attribute and the motion attribute in the second gesture attribute, which helps to reduce the types of parameters that need to be transmitted to the first application.

[0042] In a possible implementation, the competitive priority of the gesture factor is in a positive correlation with the total number of gesture control types associated with the gesture factor, and the competitive priority of the direction factor is higher than that of the gravity center factor; the gesture control type corresponding to the direction factor is the sliding control type, and the gesture control type corresponding to the gravity center factor is the rotating control type.

[0043] In this way, the number of gesture control types associated with the gesture control type is used to set the direction of each branch in the control judgment process, which helps to improve the orderliness in the process of obtaining the gesture control type. The direction factor is simpler to obtain than the gravity center factor, and by setting the competitive priority of the direction factor to be higher than that of the gravity center factor, the use rate of the direction factor in the process of determining the gesture control type is improved, and the use rate of the gravity center factor is reduced (for example, for a gesture belonging to the sliding control type, the direction factor can be used to determine that the gesture belongs to the sliding control type, and the electronic device does not need to additionally obtain the gravity center factor), which helps to reduce the computing pressure of the electronic device.

[0044] In a possible implementation, the at least one type of gesture factor includes at least one of: a quantity factor, used to represent a total number of at least one gesture recognition point indicated by the gesture sensing information; a direction factor, used to represent a moving direction of the at least one gesture recognition point; a distance factor, used to represent a moving distance of the at least one gesture recognition point; a barycenter factor, used to represent a distance between the at least one gesture recognition point and a geometric barycenter in a case where the total number of gesture recognition points is greater than 2, the geometric barycenter being a barycenter of a geometric figure with the at least one gesture recognition point as a vertex; and a speed factor, used to represent a moving speed of the at least one gesture recognition point.

[0045] In this way, the motion trend of the gesture can be described from different aspects by using the above-mentioned several types of gesture factors, each gesture of the same gesture control type is consistent in the gesture factors, and gestures of different gesture control types are different in the same gesture factor, so that the correct gesture control type can be determined according to the gesture factors, and then the correct gesture can be recognized, thereby improving the accuracy of gesture recognition.

[0046] In a possible implementation, the gesture sensing information includes position information of the at least one gesture recognition point; and the at least one type of gesture factor is obtained according to the gesture sensing information, including: in a case where the at least one type of gesture factor includes the barycenter factor, position information of the geometric barycenter is determined according to the position information of the at least one gesture recognition point; and the barycenter factor corresponding to each gesture recognition point in the at least one gesture recognition point is determined according to the position information of the gesture recognition point and the position information of the geometric barycenter.

[0047] In this way, the distance between the gesture recognition point and the geometric barycenter is reflected by using the barycenter factor, so that the relative scaling trend of the gesture can be determined according to the barycenter factor in a subsequent step. In this embodiment, the geometric barycenter is taken as a reference point to calculate the distance between each gesture recognition point and the geometric barycenter. Compared with determining the distance between any two gesture recognition points at a time, the barycenter position is taken as a reference point in this embodiment, which helps to reduce the calculation amount in the process of obtaining the barycenter factor by the electronic device, and simplify the gesture recognition logic corresponding to the scaling control type gesture. Meanwhile, it is also helpful to accurately represent the scaling trend of the gesture.

[0048] In a possible implementation, the gesture sensing information includes position information of the at least one gesture recognition point; and the at least one type of gesture factor is obtained according to the gesture sensing information, including: in a case where the at least one type of gesture factor includes the relative factor, for each relative factor, the relative factor corresponding to each gesture recognition point in the at least one gesture recognition point is determined according to the position information of the at least one gesture recognition point and the position reference information, the relative factor being at least one of the direction factor, the speed factor and the distance factor.

[0049] Therefore, by using the position information of the gesture recognition point and the position reference information, the direction factor, the speed factor and the distance factor are determined, which helps to determine the direction factor, the speed factor and the distance factor of the electronic device in a short time, thereby improving the timeliness of the motion trend indicated by the direction factor, the speed factor and the distance factor.

[0050] In a possible implementation, the first system service is located at a system framework layer in the electronic device.

[0051] Therefore, the gesture recognition function is located at the system framework layer, so that the recognition result of the gesture recognition is not affected by the types of the applications installed in the electronic device.

[0052] In a second aspect, the present application provides an electronic device, which comprises a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory, so that the electronic device executes the method in the first aspect and any possible implementation of the first aspect.

[0053] In a possible implementation, when the processor reads the computer instructions from the memory, the electronic device further executes: in response to a user operation, obtaining updated gesture sensing information; obtaining the motion parameter of the first gesture according to the updated gesture sensing information; and reporting the motion parameter of the first gesture to the first application, so that the first application continues to execute the first operation.

[0054] In a possible implementation, the motion parameter of the first gesture is related to a gesture control type to which the first gesture belongs; in a case where the first gesture belongs to a sliding control type, the motion parameter of the first gesture is used to indicate a change in sliding distance; in a case where the first gesture belongs to a zooming control type, the motion parameter of the first gesture is used to indicate a change in zooming scale; and in a case where the first gesture belongs to a rotating control type, the motion parameter of the first gesture is used to indicate a change in rotating angle.

[0055] In a possible implementation, when the processor reads the computer instructions from the memory, the electronic device further executes: obtaining subscription information of the first application, the subscription information being used to indicate at least one gesture supported by the first application, and the at least one gesture including the first gesture.

[0056] In a possible implementation, the electronic device obtains the first gesture according to the gesture sensing information, including: obtaining at least one type of gesture factor according to the gesture sensing information, the at least one type of gesture factor being used to describe a gesture parameter in one dimension respectively; and obtaining the first gesture according to the at least one type of gesture factor.

[0057] In a possible implementation, the electronic device acquires the first gesture according to the at least one type of gesture factor, including: acquiring a first gesture attribute according to the at least one type of gesture factor, the first gesture attribute being used to represent a first gesture control type to which the first gesture belongs; acquiring at least one second gesture attribute according to the at least one type of gesture factor, the at least one second gesture attribute being used to represent at least one attribute parameter corresponding to the first gesture control type; and acquiring the first gesture according to the first gesture attribute and the at least one second gesture attribute.

[0058] In a possible implementation, the at least one type of gesture factor includes a plurality of types of gesture factors with competitive priorities, the competitive priorities being used to represent an order in which the plurality of types of gesture factors participate in gesture recognition; and the electronic device acquires the first gesture attribute according to the at least one type of gesture factor, including: acquiring a first gesture factor from the plurality of types of gesture factors, the first gesture factor being a gesture factor with a highest competitive priority in the plurality of types of gesture factors; and determining the first gesture attribute according to a gesture control type corresponding to the first gesture factor, in a case where the first gesture factor satisfies a corresponding determination condition.

[0059] In a possible implementation, the at least one type of gesture factor includes a plurality of types of gesture factors with competitive priorities, the competitive priorities being used to represent an order in which the plurality of types of gesture factors participate in gesture recognition; and the electronic device acquires the first gesture attribute according to the at least one type of gesture factor, including: acquiring a first gesture factor from the plurality of types of gesture factors, the first gesture factor being a gesture factor with a highest competitive priority in the plurality of types of gesture factors; acquiring a second gesture factor from the plurality of types of gesture factors, the second gesture factor being a gesture factor with a highest competitive priority in gesture factors that do not participate in gesture recognition in the plurality of types of gesture factors, in a case where the first gesture factor does not satisfy a corresponding determination condition; and determining the first gesture attribute according to a gesture control type corresponding to the second gesture factor, in a case where the second gesture factor satisfies a corresponding determination condition.

[0060] In a possible implementation, the at least one type of gesture factor includes multiple types of gesture factors with competing priorities; and the obtaining the first gesture attribute according to the at least one type of gesture factor includes: obtaining a target gesture factor, the target gesture factor being a gesture factor with the highest competing priority among the multiple types of gesture factors; determining whether the target gesture factor meets a corresponding determination condition; in a case where the target gesture factor meets the corresponding determination condition, determining the first gesture attribute according to a gesture control type corresponding to the target gesture factor, the first gesture attribute being used to represent the gesture control type corresponding to the target gesture factor; and in a case where the target gesture factor does not meet the corresponding determination condition, updating the target gesture factor to a gesture factor with the highest competing priority among gesture factors that have not participated in gesture recognition, and repeatedly performing the operation of determining whether the target gesture factor meets the corresponding determination condition until the first gesture control type is determined, or all gesture factors in the multiple types of gesture factors have participated in gesture recognition.

[0061] In a possible implementation, the multiple types of gesture factors include a direction factor and a barycenter factor; the competing priority of the direction factor is higher than the competing priority of the barycenter factor; the gesture control type corresponding to the direction factor is a sliding control type, and the gesture control type corresponding to the barycenter factor is a rotating control type; or the gesture control type corresponding to the direction factor is a sliding control type, and the gesture control type corresponding to the barycenter factor is a zooming control type; the direction factor is used to represent a moving direction of at least one gesture recognition point indicated by gesture sensing information, and the barycenter factor is used to represent a distance between the at least one gesture recognition point and a geometric barycenter in a case where a total number of gesture recognition points is greater than two, the geometric barycenter being a barycenter of a geometric figure with the at least one gesture recognition point as a vertex. Optionally, the determination condition corresponding to the direction factor includes: there are a plurality of gesture recognition points each corresponding to the same direction factor, and the plurality of gesture recognition points are greater than or equal to a first quantity; and / or the determination condition corresponding to the barycenter factor includes: there are a second quantity of gesture recognition points, and for each gesture recognition point in the second quantity of gesture recognition points, a change amount of the barycenter factor corresponding to the gesture recognition point is greater than or equal to a first distance threshold.

[0062] In some possible implementations, the electronic device obtains the first gesture attribute according to the at least one type of gesture factor, and further includes: obtaining a third gesture factor from the at least one type of gesture factor, the third gesture factor being used to verify validity of the user operation; and in a case where the third gesture factor meets a validity condition, obtaining the first gesture attribute used to represent the gesture control type corresponding to the selected gesture factor, the selected gesture factor being the first gesture factor or the second gesture factor.

[0063] In a possible implementation, the third gesture factor includes at least one of the following: a distance factor used to represent a moving distance of the at least one gesture recognition point; and a speed factor used to represent a moving speed of the at least one gesture recognition point. Optionally, the validity condition corresponding to the speed factor includes that the speed factor corresponding to each of the at least one gesture factor is greater than or equal to a speed threshold, and / or the validity condition corresponding to the distance factor includes that the distance factor corresponding to each of the at least one gesture factor is greater than or equal to a second distance threshold.

[0064] In a possible implementation, the first gesture is acquired according to the first gesture attribute and the at least one second gesture attribute, including: continuously acquiring n predicted gestures according to the corresponding first gesture attribute and the at least one second gesture attribute during a user operation process, n being a positive integer; and acquiring the first gesture according to the n predicted gestures in a case where the n predicted gestures satisfy an accuracy condition.

[0065] In a possible implementation, the at least one second gesture attribute corresponding to the first gesture control type includes at least one of the following: a total number of gesture recognition points indicated by the gesture sensing information, and a motion attribute; the first gesture attribute includes at least one of the following: a sliding control type, a zooming control type, and a rotating control type; the motion attribute corresponding to the sliding control type includes a sliding direction attribute, the motion attribute corresponding to the zooming control type includes a zooming trend attribute, and the motion attribute corresponding to the rotating control type includes a rotating direction attribute; and the total number of gesture recognition points is at least one of the following: one finger, two fingers, three fingers, four fingers, and five fingers, the sliding direction attribute is at least one of the following: up, down, left, right, left up, right up, left down, and right down, the zooming trend attribute is at least one of the following: opening and pinching, and the rotating direction attribute is at least one of the following: clockwise and counterclockwise.

[0066] In a possible implementation, the information used to describe the first gesture includes first information and second information, the first information is determined according to the first gesture attribute and the total number of gesture recognition points, and the second information is a motion attribute.

[0067] In a possible implementation, the at least one type of gesture factor includes at least one of the following: a quantity factor used to represent a total number of at least one gesture recognition point indicated by the gesture sensing information; a direction factor used to represent a moving direction of the at least one gesture recognition point; a distance factor used to represent a moving distance of the at least one gesture recognition point; a barycenter factor used to represent a distance between the at least one gesture recognition point and a geometric barycenter in a case where the total number of gesture recognition points is greater than two, the geometric barycenter being a barycenter of a geometric figure with the at least one gesture recognition point as a vertex; and a speed factor used to represent a moving speed of the at least one gesture recognition point.

[0068] In a possible implementation, the gesture sensing information comprises position information of at least one gesture recognition point; and the electronic device acquires at least one gesture factor according to the gesture sensing information, comprising: in a case where the at least one gesture factor comprises a gravity center factor, determining position information of a geometric gravity center according to the position information of the at least one gesture recognition point; and determining, according to the position information of the at least one gesture recognition point and the position information of the geometric gravity center, a gravity center factor corresponding to each gesture recognition point in the at least one gesture recognition point.

[0069] In a possible implementation, the gesture sensing information comprises position information of at least one gesture recognition point; and the electronic device acquires at least one gesture factor according to the gesture sensing information, comprising: in a case where the at least one gesture factor comprises a relative factor, for each relative factor, determining, according to the position information of the at least one gesture recognition point and position reference information, a relative factor corresponding to each gesture recognition point in the at least one gesture recognition point, the relative factor being at least one of a direction factor, a speed factor and a distance factor.

[0070] In a possible implementation, the first system service is located at a system framework layer in the electronic device.

[0071] In a third aspect, a computer readable storage medium is provided, and the computer readable storage medium stores computer instructions. The computer instructions, when executed by a processing circuit, implement the method in any possible implementation of the first aspect.

[0072] In a fourth aspect, a chip system is provided, and the chip system comprises a processing circuit and a storage medium. The storage medium stores computer instructions. The computer instructions, when executed by the processing circuit, implement the method in any possible implementation of the first aspect.

[0073] In a fifth aspect, a computer program product containing instructions is provided. When the computer program product is run on a computer, the computer is caused to perform the method in any possible implementation of the first aspect.

[0074] The technical effects of any possible implementation of the second aspect to the fifth aspect can refer to the technical effects of the first aspect and any possible implementation of the first aspect, which will not be repeated here. The implementation of the above aspects can be further combined to provide more implementation. BRIEF DESCRIPTION OF DRAWINGS

[0075] FIG. 1 is a structural schematic diagram of an electronic device according to an embodiment of the present application;

[0076] FIG. 2A is a schematic diagram of a functional module according to an embodiment of the present application;

[0077] FIG. 2B is a software structure block diagram of an electronic device according to an embodiment of the present application;

[0078] FIG. 3 is a flow diagram of a gesture recognition method according to an embodiment of the present application;

[0079] FIG. 4 is a diagram of a function gesture subscription process according to an embodiment of the present application;

[0080] FIG. 5 is a cycle diagram of a gesture recognition method according to an embodiment of the present application;

[0081] FIG. 6 is a diagram of a direction interval distribution according to an embodiment of the present application;

[0082] FIG. 7 is a diagram of a geometric figure barycenter distribution according to an embodiment of the present application;

[0083] FIG. 8 is a diagram of a pinch operation according to an embodiment of the present application;

[0084] FIG. 9 is a global diagram of a gesture recognition method according to an embodiment of the present application;

[0085] FIG. 10 is a diagram of a sliding gesture trajectory according to an embodiment of the present application;

[0086] FIG. 11 is a diagram of a spread gesture trajectory according to an embodiment of the present application;

[0087] FIG. 12 is a diagram of a rotation gesture trajectory according to an embodiment of the present application;

[0088] FIG. 13 is an interaction flow diagram of a gesture recognition method according to an embodiment of the present application;

[0089] FIG. 14 is a hand following effect diagram of a sliding gesture according to an embodiment of the present application;

[0090] FIG. 15 is a hand following effect diagram of a pinch gesture according to an embodiment of the present application;

[0091] FIG. 16 is a change effect diagram of a pinch gesture according to an embodiment of the present application;

[0092] FIG. 17 is an interaction flow diagram of another gesture recognition method according to an embodiment of the present application;

[0093] FIG. 18 is a structure diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0094] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0095] The terms "comprise", "comprising", "include", "including", and "has", "having", and any variations thereof, used in the description and in the claims of the application, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that comprises a list of steps or elements is not necessarily limited to the listed steps or elements, but can optionally include additional steps or elements not expressly listed or inherent to such process, method, system, product, or apparatus.

[0096] Hereinafter, the terms "first", "second", and the like are used only for the purpose of description, and should not be construed to indicate or imply relative importance or imply a specified number of the technical features indicated. Therefore, the features defined with "first", "second", and the like can explicitly or implicitly include one or more of the features.

[0097] In the embodiments of the present application, the words "exemplarily" or "for example" and the like are used to represent an example, illustration or description. Any embodiment or design scheme described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the words "exemplarily" or "for example" and the like are intended to present the relevant concept in a specific manner.

[0098] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more. "And / or" in this document is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the existence of A alone, the existence of A and B together, and the existence of B alone.

[0099] As described in the background, the functions supported by an application (such as a system tool, a third-party application) can be triggered by a gesture. The gesture can be understood as being completed by a user's hand action for controlling an electronic device to implement a preset function control instruction.

[0100] After detecting the user operation, the sensor in the electronic device determines the position information of at least one gesture recognition point based on the sensing of the user operation, to obtain gesture sensing information. The user operation corresponds to a to-be-recognized gesture, and the electronic device identifies the to-be-recognized gesture according to the gesture sensing information. Taking a contact gesture as an example, the gesture recognition point refers to a contact point generated after the user's hand contacts the touch screen. The contact point does not disappear for a certain duration during the gesture. The electronic device can obtain multiple gesture sensing information by periodically detecting the position information of the contact point; and the movement trajectory of the gesture recognition point can be analyzed through the gesture sensing information, so as to identify the gesture corresponding to the user operation.

[0101] In a possible implementation, taking the application completing the gesture recognition process as an example, after the electronic device obtains the gesture sensing information, the electronic device sends the gesture sensing information to a third-party application that is running. The application internally has gesture recognition logic, and the application delivers the gesture sensing information to the gesture recognition logic; the gesture recognition logic analyzes the movement track of the gesture recognition point through the gesture sensing information, and outputs a gesture recognition result, so as to determine a function gesture that matches the gesture to be recognized. Subsequently, the application executes the function gesture for triggering a function.

[0102] With the continuous expansion of application functions, more and more applications support gesture control. The gesture control types supported by the application at the present stage include but are not limited to at least one of the following: a sliding control type, a zooming control type, and a rotating control type. Because the total number of single gestures is limited, with the continuous increase of the types of application functions, multi-finger gestures, combined gestures (such as sliding first and then rotating, pinching first and then rotating, sliding first and then pinching) and the like are gradually added to the application. The gesture recognition logic in the application is designed based on the gestures supported by the application, so as to unfold gesture recognition in response to user operations.

[0103] Because the gesture recognition logic manners of different gestures are different, in the case of adding a gesture to the application, the gesture recognition logic in the application needs to be updated, so as to ensure that the updated gesture recognition logic can accurately recognize the added gesture. This leads to the fact that the developer of the application needs to dynamically adjust and add gesture recognition logic following the expansion of functions. It can be seen that the expandability of the gesture recognition method is poor, and more human costs are wasted.

[0104] Moreover, conflicts may exist in the gesture recognition processes of various applications in the embodiment. For example, at a moment, the electronic device obtains gesture sensing information in response to user operations, and reports the gesture recognition information to application 1 and application 2; application 1 and application 2 simultaneously perform gesture recognition according to the gesture sensing information, and the situation of repeated gesture recognition by multiple applications occurs. The repeated gesture recognition causes waste of computing power of the electronic device, and is likely to cause the electronic device to appear to be stuck.

[0105] In addition, because the gesture recognition logics in different applications are different, the gestures recognized by various applications based on the same gesture sensing information may be inconsistent, leading to low accuracy of gesture control. Therefore, the above method is likely to cause gesture control abnormalities, and affects the accuracy and fluency of the gesture interaction process.

[0106] Based on the above problems, the embodiment of the present application provides a gesture recognition method, which is executed by a first system service in an electronic device. After obtaining gesture sensing information, the first system service obtains a first gesture through the gesture sensing information; then, the first system service reports information describing the first gesture to a first application subscribing to the first gesture, so that the first application executes a first operation corresponding to the first gesture. Compared with the gesture recognition executed independently by the application, the gesture recognition is uniformly taken over by the first system service, and after the first gesture is recognized, the first system service notifies the application subscribing to the first gesture to execute the corresponding operation. The gesture recognition is uniformly performed by the system service, which helps to avoid the conflict of gesture recognition between multiple applications and helps to save the computing power of the electronic device.

[0107] Moreover, the recognition logic of different types of gestures is unified in this way, so as to ensure that the same gesture can be recognized according to the same gesture sensing information, and the reliability of the gesture recognition process is ensured.

[0108] In addition, in the embodiment, the gesture recognition logic can be extended in the case of adding a gesture in the application (for example, the process of obtaining the gesture factor through the gesture sensing information is not affected), which helps to improve the expansibility of the gesture recognition method.

[0109] In some embodiments, the gesture recognition method can be applied to an electronic device. The electronic device is also referred to as an access terminal, a user unit, a user station, a mobile station, a mobile station, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communication device, a user agent or a user device.

[0110] The electronic device in the embodiment of the present application includes but is not limited to a mobile phone, a personal computer, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) electronic device, an augmented reality (AR) electronic device, a wireless terminal in industrial control, a wireless terminal in self driving, a wireless terminal in remote medical treatment, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, a vehicle-mounted terminal, a road side unit (RSU) with terminal function, etc.

[0111] Optionally, the operating system installed in the electronic device includes but is not limited to or other operating systems.

[0112] For ease of understanding, please refer to FIG. 1, which shows a structural schematic diagram of an electronic device. As shown in FIG. 1, the electronic device 100 at least includes a processor 110, a memory 120, a communication module 130, a display screen 140 and a sensor 150.

[0113] The processor 110 can include one or more processing units. For example, the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated in one or more processors.

[0114] The memory 120 includes an external memory and an internal memory. The internal memory can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various function applications and data processing of the electronic device 100 by running the instructions stored in the internal memory. The internal memory can include a program storage area and a data storage area. The memory 120 can be disposed in the processor 110 to store instructions and data. The external memory refers to a memory card connected to the electronic device 100 through an external memory interface.

[0115] The communication module 130 is used to realize the communication function of the electronic device 100. Optionally, the communication module 130 includes an antenna, a mobile communication module, a wireless communication module, a modem processor and a baseband processor, etc. Optionally, at least part of the function modules of the mobile communication module can be disposed in the processor. The wireless communication module is used to support the electronic device to perform wireless communication.

[0116] The electronic device 100 realizes the display function through a GPU, a display screen 140, etc. The display screen 140 is used to display images, videos, etc. In some embodiments, the electronic device 100 can include 1 or N display screens 140, and N is a positive integer greater than 1. The GPU and the display screen 140 can be used to support the electronic device 100 to display an interactive interface of a system tool or a third-party application, and the interactive interface displays content for interacting with a user. The user can provide a gesture operation to change or adjust the display content in the interactive interface.

[0117] The sensor 150 can include a pressure sensor 150A, a touch sensor 150B, etc. The pressure sensor 150A is configured to sense a pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 150A can be disposed under the display screen 140. The electronic device 100 can also calculate a touch position according to a detection signal of the pressure sensor 150A. The touch sensor 150B, also referred to as a "touch panel". The touch sensor 150B can be disposed on the display screen 140, and the touch sensor 150B and the display screen 140 together form a touch screen, also referred to as a "touch screen". The touch sensor 150B is configured to detect a touch operation applied thereto or in the vicinity thereof. The touch sensor 150B can transmit a detected touch operation to an application processor to determine a touch event type and execute an instruction corresponding to the touch event type, thereby enabling interaction with a user.

[0118] The sensor module 150 can further include an optical sensor. The optical sensor includes an optical camera, etc. Optionally, the optical camera can be disposed under the display screen 140 and configured to capture a gesture posture when a user operation is sensed.

[0119] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than those illustrated, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0120] As shown in FIG. 2A, the electronic device 100 provided by the embodiments of the present application can include the following software functional modules: a gesture sensing module 210 and a competition and recognition module 220. Optionally, the competition and recognition module 220 is included in a first system service. Illustratively, the first system service can also include the gesture sensing module 210.

[0121] The gesture sensing module 210 is configured to acquire gesture sensing information of a to-be-recognized gesture when a to-be-recognized gesture is detected. The gesture sensing module 210 sends the gesture sensing information to the competition and recognition module 220. Optionally, a gesture recognition point of the to-be-recognized gesture is recognized by a sensor of the electronic device; the gesture sensing module 210 determines position information of the gesture recognition point to generate the gesture sensing information.

[0122] The gesture sensing information is also referred to as a gesture bare event. Illustratively, the gesture sensing module 210 sends the gesture bare event to the competition and recognition module 220, so as to transmit the gesture sensing information to the competition and recognition module 220.

[0123] The competition and recognition module 220 is configured to obtain the first gesture according to the gesture sensing information. Optionally, the competition and recognition module 220 is further configured to report information describing the execution of the first gesture to the first application, so that the first application executes the first operation corresponding to the first gesture, and the first application is an application subscribing to the first gesture.

[0124] In some embodiments, the competition and recognition module 220 is provided with a fusion recognition algorithm and a multi-finger competition mechanism. The fusion recognition algorithm fuses the recognition logic of multiple types of gestures, and the multi-finger competition mechanism adapts gestures completed by different numbers of fingers. For details, please refer to the following embodiments.

[0125] In some other embodiments, the gesture sensing module 210 and the competition and recognition module 220 are deployed in the operating system of the electronic device 100. In the field of software development, the operating system can be divided into a system side and an application side. The system side is configured to provide basic services, and the business side is configured to interface with user needs. As shown in FIG. 2A, the gesture sensing module 210 and the competition and recognition module 220 are self-provided by the operating system. In this case, the gesture sensing module 210 and the competition and recognition module 220 are located in the system side of the operating system. Optionally, the system native tool belongs to the system side, and the third-party application belongs to the application side.

[0126] In one example, the application on the business side notifies the competition and recognition module 220 of at least one gesture supported by the application through business subscription. After the gesture sensing module 210 obtains the gesture sensing information, the competition and recognition module 220 obtains the first gesture based on the gesture sensing information, and the first gesture is the gesture notified to the competition and recognition module 220 through the business subscription.

[0127] Optionally, the competition and recognition module 220 determines the first application from the at least one application based on the subscription information in the above business subscription process, and notifies the first application to execute the first operation corresponding to the first gesture, so that the application side starts the business processing process to realize gesture control.

[0128] In some embodiments, the software system of the electronic device 100 adopts a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application take the Android system with a layered architecture as an example to exemplarily illustrate the software structure of the electronic device 100. FIG. 2B is a software structure block diagram of an electronic device provided by an embodiment of the present application. The electronic device is the electronic device 100 in FIG. 1. The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Optionally, the layers communicate with each other through a software interface. In some embodiments, the Android system is divided into four layers, and from top to bottom, they are an application layer, an application framework layer, a TS (arkts runtime) runtime and a system library, and a kernel layer.

[0129] The application layer can include a series of application packages.

[0130] As shown in FIG. 2B, the application packages can include applications such as a camera, a gallery, a calendar, a call, a map, navigation, wireless fidelity (WLAN), Bluetooth, music, video, short message, etc.

[0131] The application framework layer provides an application programming interface (API) and a programming framework for the applications of the application layer. The application framework layer includes some pre-defined functions. Optionally, the applications in the application layer support gesture control.

[0132] As shown in FIG. 2B, the application framework layer can include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, a first system service, etc. The application framework layer is also referred to as a system framework layer.

[0133] The window manager is used to manage window programs. The window manager can be used to acquire a display screen size, determine whether there is a status bar, lock a screen, take a screenshot, etc.

[0134] The content provider is used to store and acquire data, and make the data accessible to the applications. The data includes video, image, audio, dialed and received calls, browsing history and bookmarks, a phonebook, etc.

[0135] The view system includes visual controls. For example, a control for displaying text, a control for displaying pictures, etc. The view system can be used to build an application. A display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0136] The telephony manager is used to provide a communication function of the electronic device 100. For example, management of a call state (including call connection, call hang-up, etc.).

[0137] The resource manager provides various resources for the applications, such as localized strings, icons, pictures, layout files, video files, etc.

[0138] The notification manager enables applications to display notification information in the status bar, which can be used to convey alert-type messages that can automatically disappear after a brief stay without user interaction. The notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the form of a figure or a scroll bar text in the top status bar of the system, such as a notification of an application running in the background, and can also be a notification that appears in the form of a dialog window on the screen. For example, the status bar prompts text information, emits a prompt sound, the electronic device vibrates, the indicator light flashes, etc.

[0139] The first system service is used to provide a unified gesture recognition service. Optionally, the first system service includes a competition and recognition module. Optionally, the first system service also includes a gesture sensing module.

[0140] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0141] The core library contains two parts: one part is the function function required to be called by the java language, and the other part is the core library of Android.

[0142] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform the management of the object life cycle, the management of the stack, the management of the thread, the management of the security and the exception, and the garbage collection, etc. The system library includes a plurality of function modules, such as a surface manager, media libraries, a three-dimensional graphics processing library (such as OpenGL ES), a two-dimensional graphics engine (such as SGL), etc.

[0143] The surface manager is used to manage the display subsystem, and provides a fusion of two-dimensional (2D) and three-dimensional (3D) layers for a plurality of applications.

[0144] The media library supports multiple commonly used audio, video format playback and recording, and static image files, etc. The media library can support multiple audio and video encoding formats, such as: moving picture experts group 4 (MPEG4), high-efficiency video coding (H.264), moving picture experts group audio layer III (MP3), advanced audio coding (AAC), adaptive multi rate (AMR), joint photographic experts group (JPG), portable network graphics (PNG), etc.

[0145] The three-dimensional graphics processing library is used to realize three-dimensional graphics drawing, image rendering, synthesis and layer processing, etc. The two-dimensional graphics engine is a drawing engine for 2D drawing.

[0146] The kernel layer is a layer between hardware and software. The kernel layer at least contains display driver, camera driver, audio driver, sensor driver.

[0147] The gesture recognition method provided by the present application is described below with the electronic device as the execution subject. Optionally, the embodiment is implemented by the first system service in the electronic device 100 in FIG. 1. The embodiment is described by taking an example of that the electronic device comprises a gesture sensing module and a competition and recognition module. Please refer to FIG. 3, which is a flowchart of the gesture recognition method provided by the embodiment of the present application. As shown in FIG. 3, the gesture recognition method provided by the embodiment of the present application at least comprises steps S310 to S340.

[0148] S310, the first system service acquires gesture sensing information in response to a user operation.

[0149] The user operation is used to trigger gesture control. In some embodiments, the user operation is a to-be-recognized gesture sensed by the electronic device, which is used to instruct the electronic device to implement a preset function. The preset function is implemented by an application installed in the electronic device performing corresponding operation. Exemplarily, the preset function comprises at least one of the following: refreshing display content, adjusting display effect of display content, opening / closing an application, calling an application running in the background, moving an application to run in the background, etc.

[0150] Optionally, the preset function is implemented by an application in the electronic device. Exemplarily, the application comprises at least one of the following: a system native application, a third-party application.

[0151] The system native application includes system tools and application programs in the operating system that cannot be uninstalled. The system tools are used for services and maintenance of the operating system. The functions of the system tools include, but are not limited to, system management, performance optimization, hardware management, data cleaning, security maintenance, and the like. For example, the command prompt, the task manager, the hard disk cleaner, and the like belong to the system native application. The application programs in the operating system that cannot be uninstalled include the application store, the notepad, the photo album, the call application, and the like. Exemplarily, the system native application is developed by the development team of the operating system.

[0152] The third-party application is provided by the operating system developer or other developers. Optionally, the electronic device obtains the installation package of the third-party application from a content publishing platform, and installs the third-party application to the electronic device by decompressing the installation package.

[0153] In some embodiments, according to the different types of the application, the gesture includes at least one of the following: a gesture for triggering a function provided by the system native application, and a gesture for triggering a function provided by the third-party application.

[0154] Optionally, the gesture to be recognized is made by a user using the electronic device. The user controls the electronic device to perform a corresponding operation through the gesture, so as to complete a preset function. Exemplarily, after detecting the user operation, a first system service in the electronic device identifies a first gesture corresponding to the user operation; then, the first system service notifies the application to perform an operation corresponding to the gesture. The process of "determining the gesture corresponding to the user operation" mentioned herein is the process of gesture recognition.

[0155] In some embodiments, the gesture supported by the electronic device is a gesture subscribed by the application. Optionally, before step S310 is performed, the at least one gesture supported by the application is obtained through the subscription information from the application.

[0156] Optionally, the gesture recognition process is completed by a competition and recognition module. Exemplarily, the first system service includes the competition and recognition module. Exemplarily, the competition and recognition module is provided with a subscription interface, and the application notifies the at least one gesture supported by the application to the competition and recognition module through the subscription interface. For example, the application reports the subscription information to the first system service through the subscription interface.

[0157] As shown in FIG. 4, the competition and recognition module obtains the subscription information from the application; and the competition and recognition module notifies the application of the subscription result. The subscription information is used to represent the at least one gesture supported by the application, and the subscription result is used to notify the application of the success of the gesture subscription.

[0158] Exemplarily, after the application acquires the subscription result, at least one gesture supported by the application is called a subscription gesture of the application, and a gesture recognition process about the subscription gesture is completed by the competition and recognition module. Exemplarily, for each of the at least one gesture, the subscription information includes a type identifier and an attribute parameter. The type identifier is used to represent a gesture control type to which the gesture belongs, and the attribute parameter is used to represent a specific implementation manner of the gesture in the gesture control type to which the gesture belongs. The attribute parameter includes at least one of a moving direction, a rotating direction, a moving distance, a total number of gesture recognition points, and the like.

[0159] Optionally, after a new application is installed in the electronic device or a gesture is added to a certain application, the application notifies, through a subscription interface, subscription information about the added gesture.

[0160] Exemplarily, a gesture mapping table is stored and maintained in the electronic device, and the gesture mapping table is used to record a mapping relationship between a gesture and an application that subscribes to the gesture. For example, for a certain gesture, the gesture mapping table stores a gesture identifier of the gesture, and each of the applications that subscribe to the gesture has an application identifier. By setting the gesture mapping table, the applications that can subscribe to a first gesture can be quickly determined after the first gesture is recognized by the first system service.

[0161] In some embodiments, the gesture control type to which the gesture belongs includes at least one of a sliding control type, a zooming control type, a rotating control type, and the like. In the embodiments of the present application, the sliding control type can also be abbreviated as a sliding type, the zooming control type can also be abbreviated as a zooming type, and the rotating control type can also be abbreviated as a rotating type.

[0162] The sliding control type includes at least one gesture that is realized by stretching a hand, and moving the hand through an elbow joint and / or an arm. The gestures included in the sliding control type can realize operations including but not limited to refreshing display content, switching a display page, adjusting brightness, adjusting volume, adjusting a video playing progress, calling an application running in the background, moving an application to run in the background, and the like.

[0163] The zooming control type includes at least one gesture that is realized by fixing a wrist, and folding or unfolding a finger joint. The gestures included in the zooming control type can realize operations including but not limited to adjusting a display size of display content (such as enlarging or reducing a picture or text), adjusting a shooting parameter of a camera, intercepting display content, calling a system tool (such as entering a desktop arrangement tool through a pinching operation), and the like.

[0164] The rotating control type includes at least one gesture that is realized by keeping a relative position between fingers unchanged, and rotating a hand through a wrist rotation. The gestures included in the rotating control type can realize operations of adjusting a display angle of display content.

[0165] In some embodiments, a plurality of gestures are included in one gesture control type, and the overall motion trend of the plurality of gestures is similar, and there are differences in specific implementations. In the embodiments of the present application, each of the plurality of gestures has an attribute parameter, and each gesture included in the same gesture control type can be distinguished by the attribute parameter. Exemplarily, the attribute parameter includes a gesture attribute dimension of a total number of gesture recognition points and a motion attribute dimension. For example, the plurality of gestures belonging to the same gesture control type respectively have different total numbers of gesture recognition points.

[0166] Optionally, the gesture attribute includes at least one of the following: a total number of gesture recognition points, a motion attribute. The total number of gesture recognition points takes at least one of the following values: single finger, double finger, three fingers, four fingers, five fingers. The motion attribute is used to describe the overall motion trajectory of each finger corresponding to at least one gesture recognition point.

[0167] Exemplarily, the motion attribute corresponding to the sliding control type includes a sliding direction attribute. In one example, the sliding direction attribute is at least one of the following: up, down, left, right, left up, right up, left down, right down.

[0168] Exemplarily, the motion attribute corresponding to the scaling control type includes a scaling trend attribute. In one example, the scaling trend attribute is at least one of the following: opening, pinching. That is, the scaling control type can be subdivided into an opening control type and a pinching control type.

[0169] Exemplarily, the motion attribute corresponding to the rotating control type includes a rotating direction attribute. In one example, the rotating direction attribute is at least one of the following: clockwise, counterclockwise.

[0170] For example, a certain gesture belongs to the sliding control type, the sliding direction attribute is right, and the total number of gesture recognition points is 2, then the gesture is a two-finger sliding operation. For another example, a certain gesture belongs to the scaling control type, the scaling trend attribute is pinching, and the total number of gesture recognition points is 5, then the gesture is a five-finger pinching operation. For another example, a certain gesture belongs to the rotating control type, the rotating direction attribute is clockwise, and the total number of gesture recognition points is 4, then the gesture is a four-finger clockwise rotating operation.

[0171] In some embodiments, the gesture is a single gesture. For example, the gesture to be recognized is a two-finger sliding gesture.

[0172] In other embodiments, the gesture is a composite gesture. The composite gesture is obtained by combining at least two single gestures. For example, a certain gesture is first a three-finger scaling and then a three-finger pinching. For example, a certain gesture is first a three-finger sliding and then a two-finger rotating. In this case, there is at least one common recognition point between the at least two single gestures included in the composite gesture.

[0173] The common recognition point refers to a same gesture recognition point included in at least two single gestures in the composite gesture. That is, in a case where the first single gesture in the composite gesture is switched to the second single gesture, at least one common recognition point in at least one gesture recognition point corresponding to the first electronic gesture does not disappear. In this case, the first system service in the electronic device performs gesture recognition once every detection interval during continuous execution of the user operation, which helps to timely perceive the gesture change.

[0174] In some embodiments, the gestures are classified into at least two types according to acquisition manners, i.e., a contact type gesture and a non-contact type gesture. The contact type gesture refers to an action gesture made by the user after the hand of the user contacts the electronic device. In this case, the gesture to be recognized can be sensed by a pressure sensor in the electronic device. The gesture to be recognized refers to a first gesture to be recognized, which can be referred to as the gesture to be recognized before the first gesture is determined by the gesture recognition method.

[0175] The non-contact gesture refers to an action gesture made by the user without contacting the electronic device. For the non-contact type gesture, the gesture to be recognized can be photographed by an optical camera in the electronic device. The electronic device performs image recognition on the photographed image of the gesture to be recognized to obtain at least one gesture recognition point. The electronic device determines gesture sensing information according to a display position of the at least one gesture recognition point in the image.

[0176] In some embodiments, the gesture sensing information is used to describe a posture of the gesture to be recognized. Optionally, the gesture sensing information includes position information of at least one gesture recognition point. The gesture recognition point is a key point of the gesture to be recognized. For example, during continuous execution of the gesture to be recognized, the position information of the gesture recognition point changes with movement of the gesture to be recognized.

[0177] Optionally, for the contact type gesture, the gesture recognition point includes at least one contact point between the hand of the user and the display screen of the electronic device. For example, the gesture recognition point is a contact point between the finger pulp of a finger and the display screen of the electronic device. For example, assuming that there are four contact points between the gesture to be recognized and the display screen of the electronic device, the four contact points are all gesture recognition points, and the gesture sensing information is used to describe the position information of the four contact points.

[0178] Optionally, for the non-contact type gesture, the gesture recognition point includes a point on the edge of the gesture to be recognized. Illustratively, after detecting the gesture to be recognized, an optical camera in the electronic device captures a gesture image, and the gesture recognition point is located on the contour line of the gesture to be recognized displayed in the gesture image. The gesture recognition point includes the end point of a finger. In this case, the gesture recognition point can be obtained by feature point recognition on the gesture image. The feature point recognition method includes but is not limited to a real-time target detection algorithm (you only look once, YOLO).

[0179] Illustratively, the position information of the gesture recognition point is represented by coordinates. For example, the position information of the gesture recognition point p1 is represented as (x1, y1), the position information of the gesture recognition point p2 is represented as (x2, y2), and so on. S is a positive integer. For example, s is a positive integer less than or equal to 5.

[0180] Illustratively, for the contact type gesture, the position information of the gesture recognition point is determined by the display coordinate system, and the coordinate information is two-dimensional information. The display coordinate system is a two-dimensional coordinate system used to locate the display content. For example, for a gesture to be recognized belonging to the non-contact type gesture, the position of the gesture recognition point can be determined by the image coordinate system, and the coordinate information is two-dimensional information. The image coordinate system is a two-dimensional coordinate system used to locate the pixels in the image.

[0181] In some embodiments, in the case where the gesture to be recognized corresponds to multiple gesture recognition points, the gesture sensing information further includes a recognition point index. The recognition point index is used to identify a gesture recognition point, and different gesture recognition points have different gesture indices. Here, "multiple" means two or more.

[0182] Optionally, the recognition point index is represented by a number, for example, the recognition point indices of multiple gesture recognition points are natural numbers that are incremented step by step (for example, incremented step by step with an increment step of 1). Illustratively, for a gesture to be recognized belonging to the contact type gesture, the recognition point index is referred to as a finger index or index, which is used to indicate the finger number corresponding to the gesture recognition point.

[0183] FIG. 5 is a cycle diagram of a gesture recognition method provided by an embodiment of the present application. As shown in FIG. 5, the first system service obtains gesture sensing information of a gesture to be recognized; in the case where the gesture to be recognized satisfies the competitive recognition condition, the first system service performs subsequent steps of gesture recognition (for details, please refer to the introduction and description of steps S320, S330 and S340, etc.); in the case where the gesture to be recognized does not satisfy the competitive recognition condition, the first system service re-obtains the gesture sensing information.

[0184] Optionally, the competition recognition condition is used to determine whether the gesture recognition process is completed. Exemplarily, the competition recognition condition may, for example, include that the function gesture matched with the to-be-recognized gesture is not determined, and the to-be-recognized gesture moves. Wherein, whether the to-be-recognized gesture moves can be determined by the plurality of gesture sensing information. For example, the electronic device obtains two gesture sensing information successively, and according to the two gesture sensing information, whether the to-be-recognized gesture moves can be obtained. For example, the two gesture sensing information are the same, and the electronic device can obtain that the to-be-recognized gesture does not move. For another example, the two gesture sensing information are different, and the electronic device obtains that the to-be-recognized gesture moves.

[0185] In one example, the competition recognition condition is that the function gesture matched with the to-be-recognized gesture is not determined, and the to-be-recognized gesture moves. That is, the to-be-recognized gesture is not currently recognized, and the user operation is not ended. Exemplarily, in the case that the function gesture matched with the to-be-recognized gesture is not determined, and the to-be-recognized gesture moves, the first system service in the electronic device executes the subsequent step of gesture recognition; in the case that the function gesture matched with the to-be-recognized gesture is determined, or the to-be-recognized gesture does not move, the electronic device re-executes step S310.

[0186] In the case that the to-be-recognized gesture does not move, the movement trend of the gesture cannot be determined according to the gesture sensing information, and in this case, the subsequent gesture recognition step is not executed, which helps to avoid invalid calculation of the electronic device.

[0187] In some embodiments, step S310 is executed by the gesture sensing module. Optionally, the gesture sensing module executes step 310 multiple times to obtain gesture sensing information multiple times, from the time when the to-be-recognized gesture is detected to the time when the gesture recognition is completed.

[0188] Exemplarily, the gesture sensing module obtains gesture sensing information every first time interval. Wherein, the first time interval is pre-set, such as 0.011 seconds, 0.017 seconds, 0.034 seconds, etc. The specific value of the first time interval is related to the hardware device of the electronic device, the power consumption mode and other factors, which are not limited here. For example, the to-be-recognized gesture is a contact gesture, and the electronic device detects the touch operation of the user's finger on the touch screen, triggering the gesture sensing module to obtain gesture sensing information every first time interval. Then, in the case that the user's finger is detected to leave the display screen, the electronic device stops obtaining gesture sensing information. Then in this process, the electronic device can obtain a plurality of gesture sensing information.

[0189] In one example, after obtaining the gesture sensing information, the first system service in the electronic device determines, according to the gesture sensing information, a first gesture matching the to-be-recognized gesture. If the gesture matching the to-be-recognized gesture is recognized in a subsequent step, the gesture recognition ends; if the gesture matching the to-be-recognized gesture cannot be recognized, the first system service in the electronic device re-executes step S310 until a first gesture matching the to-be-recognized gesture is determined, or the number of times of obtaining the gesture sensing information exceeds a threshold number of times of obtaining, and it is determined that the gesture recognition fails. The threshold number of times of obtaining is pre-set, such as 3 times, 5 times, etc.

[0190] Compared with waiting to obtain gesture sensing information at multiple moments or starting to perform a subsequent gesture recognition step only after detecting that the to-be-recognized gesture ends in the related art, the embodiment starts to perform a subsequent gesture recognition step immediately after obtaining gesture sensing information each time, which helps to shorten the duration of the gesture recognition process, so as to determine a gesture matching the to-be-recognized gesture before the to-be-recognized gesture ends, thereby helping to improve the response speed of the electronic device to user operations, reduce response delay, and help to improve the use fluency of the electronic device.

[0191] S320, the first system service obtains at least one type of gesture factor according to the gesture sensing information.

[0192] Each type of gesture factor in the at least one type of gesture factor is used to describe a gesture parameter in a corresponding dimension, and different types of gesture factors are used to respectively describe the motion trend of the to-be-recognized gesture from different dimensions. Optionally, the gesture parameter refers to a physical quantity that can be sensed by the electronic device during the duration of the user operation. The gesture parameter can also be calculated from the sensed physical quantity. Gesture parameters in different dimensions are used to represent different physical quantities. For example, the dimensions of the gesture parameters are not limited to at least one of the following: quantity dimension, direction dimension, gravity center dimension, speed dimension, distance dimension.

[0193] Optionally, the gesture factors are classified according to a determination manner, including at least the following two: relative gesture factor and absolute gesture factor. For example, the absolute gesture factor is determined by the electronic device according to the gesture sensing information, and the relative gesture factor is determined by the electronic device according to the gesture sensing information and other reference information. The other reference information includes but is not limited to gesture sensing information obtained at other moments. The relative gesture factor is also called a relative factor, and the absolute gesture factor is also called an absolute factor.

[0194] In some embodiments, the types of gesture factors include, but are not limited to, at least one of: a quantity factor, a direction factor, a barycenter factor, a distance factor, and a speed factor. Optionally, the quantity factor is used to describe a gesture parameter in a quantity dimension, the direction factor is used to describe a gesture parameter in a direction dimension, the barycenter factor is used to describe a gesture parameter in a barycenter dimension, the speed factor is used to describe a gesture parameter in a speed dimension, and the distance factor is used to describe a gesture parameter in a distance dimension.

[0195] The quantity factor is used to represent a total number of gesture recognition points included in the gesture to be recognized. Optionally, the quantity factor is used to determine a gesture matching the gesture to be recognized.

[0196] In some embodiments, the quantity factor belongs to an absolute gesture factor. Optionally, the first system service in the electronic device determines the quantity factor according to the gesture sensing information, including: the first system service counts the number of gesture recognition points included in the gesture sensing information, and obtains the quantity factor. For example, in the case that the gesture sensing information includes a recognition point serial number, the electronic device subtracts the smallest recognition point serial number from the largest recognition point serial number, and adds one to the obtained numerical result to obtain the quantity factor.

[0197] The direction factor is used to represent a moving direction of at least one gesture recognition point. Optionally, the direction factor is used to determine a gesture control type to which the gesture to be recognized belongs. In addition, the direction factor can also be used to determine an attribute parameter, which will be described in the following embodiments.

[0198] Optionally, the direction factor includes a direction factor corresponding to each gesture recognition point. The direction factor corresponding to a certain gesture recognition point is used to represent the moving direction of the gesture recognition point. By comparing the direction factors corresponding to each gesture recognition point, the similarities and differences of the moving directions of each gesture recognition point can be determined.

[0199] It can be understood that, during the duration of the gesture to be recognized, the moving track of the gesture to be recognized is unstable. For the same gesture recognition point, the direction factors determined according to gesture recognition information at different time points can also be different. Therefore, the direction factor has timeliness. For example, the direction factor obtained through the gesture sensing information at the first time point is used to represent the moving direction of the gesture to be recognized within the first time period, and the direction factors corresponding to each gesture recognition point are respectively used to represent the moving directions of each gesture recognition point within the first time period.

[0200] The first time period is a time period including the first time. For example, the first time period is a time period with the second time as a starting time and the first time as an ending time. For example, the electronic device obtains gesture sensing information 2 at the second time, obtains gesture sensing information 1 at the first time, and obtains the position reference information of each gesture recognition point based on the gesture sensing information 1 during the process of determining the gesture factor of at least one gesture. The second time is earlier than the first time.

[0201] The direction factor belongs to the relative gesture factor. Optionally, the first system service obtains the direction factor based on the gesture sensing information, including: for each gesture recognition point in the at least one gesture recognition point, the first system service obtains the direction factor corresponding to the gesture recognition point based on the position information of the gesture recognition point and the position reference information of the gesture recognition point.

[0202] The first time refers to the time when the gesture sensing information is obtained when the step 310 is executed last time before the step 320 is executed this time.

[0203] Optionally, the position reference information is used to indicate the historical position of the gesture recognition point during the process of the user operation. For example, the position information is obtained based on the gesture sensing information at the first time, and the position reference information is obtained based on the gesture sensing information at the second time. The gesture sensing information at the second time belongs to the other reference information mentioned above, and the second time is earlier than the first time.

[0204] Optionally, the position reference information corresponds to the starting position of the gesture recognition point during the movement process of the gesture to be recognized, and the position information corresponds to the ending position of the gesture recognition point during the movement process of the gesture to be recognized. In the embodiment of the present application, the position reference information is also referred to as reference information, and both have the same meaning. The position information is also referred to as first position information, and both have the same meaning.

[0205] For example, time 1 and time 2 are both first times, and the second times corresponding to time 1 and time 2 are both time 3. Time 3 can be the starting time of the gesture recognition process. In one example, after detecting the gesture to be recognized, the electronic device obtains gesture sensing information at an initial time, and stores the gesture sensing information at the initial time, so as to obtain the position reference information of each gesture recognition point based on the gesture sensing information at the initial time during the execution of the step S330.

[0206] Exemplarily, each first time point corresponds to a different second time point. For example, the first time point and the second time point are separated by k first time intervals, the gesture sensing information of the first time point is gesture sensing information of time point i, the second time point is time point i-k*t, k is a positive number, i is a positive number greater than k, and t represents the duration of the first time interval. For example, if k=1, the second time point is the collection time point of the last obtained gesture sensing information.

[0207] In some embodiments, the first system service obtains the direction factor corresponding to the gesture recognition point according to the position information of the gesture recognition point and the position reference information of the gesture recognition point, including: the first system service obtains a direction vector according to the position reference information and the position information; and the first system service obtains the direction factor corresponding to the gesture recognition point according to the direction vector.

[0208] Optionally, the first system service obtains the direction factor corresponding to the gesture recognition point according to the direction vector, including: the first system service determines the direction angle of the direction vector as the direction factor corresponding to the gesture recognition point. The direction angle refers to the deflection angle of the direction vector relative to the x-axis in the display coordinate system or the image coordinate system in the counterclockwise direction.

[0209] Exemplarily, the display coordinate system can be a two-dimensional coordinate system with the left bottom corner as the origin, the horizontal right direction as the positive direction of the x-axis, and the horizontal upward direction as the positive direction of the y-axis in the vertical screen form of the electronic device.

[0210] In one example, the first position information of the gesture recognition point 1 is (endX, endY), the reference position information of the gesture recognition point 1 is (startX, startY), and the direction angle θ of the direction vector is calculated according to the following formula:

[0211] Wherein, Math.atan2 is used to calculate the plane angle between the direction vector and the positive direction of the x-axis.

[0212] Optionally, the first system service obtains the direction factor corresponding to the gesture recognition point according to the direction vector, including: the electronic device determines the direction factor corresponding to the gesture recognition point according to the direction interval to which the direction vector belongs. Exemplarily, if the direction angle of the direction vector belongs to a first direction interval in N direction intervals, the first system service represents the direction factor corresponding to the gesture recognition point by the first direction interval, and N is a positive integer greater than 1 and less than 360.

[0213] Wherein, the [0, 360°] range on the two-dimensional plane is divided into N direction intervals, and each direction interval corresponds to an angle range [a°, b°), and a and b are real numbers in the [0, 360] range.

[0214] In one example, as shown in FIG. 6, the two-dimensional plane is divided into 8 direction intervals, each direction interval has an angle span of 45°, and the 8 direction intervals are: right interval [-22.5°, 22.5°), right lower interval [22.5°, 67.5°), lower interval [67.5°, 112.5°), left lower interval [112.5°, 157.5°), left interval [157.5°, -157.5°), left upper interval [-157.5°, -112.5°), upper interval [-112.5°, -67.5°), and right upper interval [-67.5°, -22.5°). If the direction angle of a certain direction vector is 10°, then the direction vector belongs to the right direction interval, and the direction factor corresponding to the gesture recognition point is: right.

[0215] In this embodiment, on one hand, the direction factor corresponding to the gesture recognition point is determined by the direction interval to which the direction vector belongs, the selectable values of the direction factor are limited, and thus it is helpful to quickly compare whether the respective direction factors of the at least one gesture recognition point are the same. In this way, it is helpful to speed up the efficiency of gesture recognition.

[0216] On the other hand, the movement direction of each gesture recognition point in the movement process is deviated, and in this way, it is helpful to offset the influence of the movement deviation of each gesture recognition point on the accuracy of gesture recognition, and thus it is helpful to improve the accuracy of the gesture recognition process.

[0217] The distance factor is used to represent the movement distance of the at least one gesture recognition point. Optionally, the distance factor is used to verify whether the gesture sensing information is generated by the gesture control process. That is, the first system service can determine whether the gesture control occurs or whether the user operation is valid by the distance factor, and specific reference can be made to the following embodiments.

[0218] The distance factor belongs to the relative gesture factor. The distance factor obtained by the gesture sensing information at the first time is used to represent the movement distance of the at least one gesture recognition point in the first time period. Optionally, each gesture recognition point of the at least one gesture recognition point corresponds to a distance factor. The distance factor corresponding to a certain gesture recognition point is used to represent the movement distance of the gesture recognition point in the first time period.

[0219] Optionally, the first system service in the electronic device acquires the distance factor according to the gesture sensing information, including: for each gesture recognition point of the at least one gesture recognition point, the first system service acquires the distance factor corresponding to the gesture recognition point according to the position information of the gesture recognition point and the position reference information of the gesture recognition point. The position information and the position reference information are described above and will not be repeated here.

[0220] In one example, the position information of the gesture recognition point is (endX, endY), the position reference information is (startX, startY), and the calculation formula of the distance factor L corresponding to the gesture recognition point is:

[0221] In the case that the total number of gesture recognition points is greater than 2, the barycenter factor is used to represent the distances respectively existing between the geometric barycenter of the gesture to be recognized and at least one gesture recognition point.

[0222] The geometric barycenter of the gesture to be recognized is the barycenter of a geometric figure having at least one gesture recognition point as a vertex. As shown in FIG. 7, the five gesture recognition points P1, P2, P3, P4 and P5 corresponding to the gesture to be recognized are P1, P2, P3, P4 and P5, and the geometric barycenter is the barycenter G of the geometric figure 700 having P1, P2, P3, P4 and P5 as vertices.

[0223] Optionally, the barycenter factor is used to determine the gesture control type to which the gesture to be recognized belongs. In addition, the barycenter factor can also be used to determine the attribute parameter, which will be described in the following embodiments.

[0224] The barycenter factor belongs to the relative gesture factor. Optionally, the barycenter factor includes at least one barycenter factor corresponding to each gesture recognition point. The barycenter factor corresponding to a certain gesture recognition point is used to represent the distance between the gesture recognition point and the geometric barycenter.

[0225] Optionally, the first system service in the electronic device acquires the barycenter factor according to the gesture sensing information, including: the first system service obtains the position information of at least one gesture recognition point according to the gesture sensing information; the first system service acquires the position information of the geometric barycenter according to the position information of the at least one gesture recognition point; and for each gesture recognition point in the at least one gesture recognition point, the first system service acquires the barycenter factor corresponding to the gesture recognition point according to the position information of the gesture recognition point and the position information of the geometric barycenter.

[0226] Illustratively, assuming that the total number of the at least one gesture recognition point is p, and p is a positive integer greater than 2; the first system service acquires the position information of the geometric barycenter according to the position information of the at least one gesture recognition point, including: the first system service divides the p gesture recognition points into p-2 recognition point sets according to the position information of the p gesture recognition points; the first system service acquires the horizontal coordinate of the geometric barycenter according to the areas of the p-2 triangles and the horizontal coordinates of the gesture recognition points; and the first system service acquires the vertical coordinate of the geometric barycenter according to the areas of the p-2 triangles and the vertical coordinates of the gesture recognition points in the recognition point sets.

[0227] Each of the p-2 sets of recognition points includes 3 gesture recognition points, which can form a triangle, and the p-2 sets of recognition points are used to divide the geometric figure into p-2 mutually disjoint triangles. For example, there are at least two sets of recognition points in the p-2 sets of recognition points, and at least one same gesture recognition point is included in the at least two sets of recognition points.

[0228] As shown in FIG. 8, 5 gesture recognition points include A (x1, y1), B (x2, y2), C (x3, y3), D (x4, y4) and E (x5, y5), and the 5 gesture recognition points form a geometric figure 800, which is divided into 3 sets of recognition points. The 3 sets of recognition points are: set of recognition points 1 (including gesture recognition points A, B and C), set of recognition points 2 (including gesture recognition points A, D and C) and set of recognition points 3 (including gesture recognition points A, E and D). The area of the triangle 810 formed by the set of recognition points 1 is s1.

[0229] In one example, the abscissa x of the geometric barycenter is obtained by the formula:

[0230] wherein s i represents the area of the triangle formed by the i-th set of recognition points, x i0 , x i1 , x i2 represent the respective abscissas of the 3 gesture recognition points included in the i-th set of recognition points, and i is a positive integer less than or equal to p-2.

[0231] In one example, the ordinate y of the geometric barycenter is obtained by the formula:

[0232] wherein y i0 , y i1 , y i2 represent the respective ordinates of the 3 gesture recognition points included in the i-th set of recognition points, and the explanations of other parameters refer to the above embodiments.

[0233] For example, the first system service obtains the barycentric factor corresponding to the gesture recognition point according to the position information of the gesture recognition point and the position information of the geometric barycenter, including: the first system service processes the position information of the gesture recognition point and the position information of the geometric barycenter by a distance formula to obtain a distance result; and the first system service determines the distance result as the barycentric factor corresponding to the gesture recognition point. The formula used to calculate the distance result is the same as the formula used to calculate the distance factor in the above embodiments, and the details are described above and will not be repeated here.

[0234] ​​A speed factor is used to represent the moving speed of each of the at least one gesture recognition point. Optionally, the speed factor is used to determine the gesture control type to which the gesture to be recognized belongs.

[0235] The speed factor belongs to the relative gesture factors. The speed factor has a time effect, and the speed factor obtained by the gesture sensing information at the first time point is used to represent the moving speed of the at least one gesture recognition point in a time period including the first time point.

[0236] Optionally, the speed factor includes speed factors corresponding to the at least one gesture recognition point respectively. The speed factor corresponding to a gesture recognition point is used to represent the moving speed of the gesture recognition point. For example, for a gesture recognition point, the speed factor obtained according to the gesture sensing information at the first time point is used to represent the moving speed of the gesture recognition point in the first time period.

[0237] For example, the first system service in the electronic device obtains the speed factor according to the gesture sensing information, including: the electronic device obtains the distance factor; for each of the at least one gesture recognition point, the electronic device determines the speed factor corresponding to the gesture recognition point according to the distance factor corresponding to the gesture recognition point and the length of the first time period.

[0238] In one example, the speed factor corresponding to a gesture recognition point is positively correlated with the distance factor corresponding to the gesture recognition point and is negatively correlated with the length of the first time period. For example, the speed factor corresponding to a gesture recognition point is equal to the distance factor divided by the length of the first time period.

[0239] In some embodiments, the electronic device obtains the at least one type of gesture factor according to the gesture recognition information, which is completed by a competition and recognition module in the electronic device. Optionally, the competition and recognition module is a functional module of the system layer of the operating system of the electronic device.

[0240] For example, after the gesture sensing module obtains the gesture sensing information at the first time point, the gesture sensing module sends the gesture sensing information at the first time point to the competition and recognition module, so that the competition and recognition module determines the at least one type of gesture factor at the first time point according to the gesture sensing information at the first time point.

[0241] In some embodiments, the at least one type of gesture factor is a necessary gesture factor in the gesture recognition process. Optionally, a necessary factor group is stored and maintained in the electronic device, and the at least one type of gesture factor is included in the necessary factor group.

[0242] For example, if the gesture control types supported by the electronic device include a sliding control type, the set of mandatory factors includes a gesture factor for judging the sliding control type. For example, if the gesture control types supported by the electronic device include a sliding control type and a zoom control type, the gesture factors determined according to the gesture sensing information include a gesture factor for judging the sliding control type and a gesture factor for judging the zoom control type.

[0243] In some embodiments, the mandatory gesture factors are related to gestures supported by the electronic device, and the set of mandatory factors is stored and maintained by the competition and identification module. Optionally, after receiving the subscription information from the application, the competition and identification module determines the mandatory gesture factors corresponding to the new gesture, and if the set of mandatory factors includes the mandatory gesture factors corresponding to the new gesture, the set of mandatory factors does not need to be updated; if the set of mandatory factors does not include the mandatory gesture factors corresponding to the new gesture, the competition and identification module adds the mandatory gesture factors corresponding to the new gesture to the set of mandatory factors to obtain an updated set of mandatory factors.

[0244] Optionally, in a case where all applications supporting a certain gesture control type have been uninstalled, the competition and identification module determines at least one mandatory gesture factor corresponding to the gesture control type, and deletes the at least one mandatory gesture factor from the set of mandatory factors to obtain an updated set of mandatory factors.

[0245] The embodiment helps to reduce the total number of gesture factors required to be obtained in the gesture recognition process. After the electronic device determines the mandatory gesture factors according to the gesture sensing information, the subsequent gesture recognition steps can be performed. Compared with calculating gesture factors of all types, the calculation amount consumed by calculating the mandatory gesture factors is less. Therefore, the embodiment helps to reduce the calculation amount required in the gesture recognition process, helps to shorten the time consumption of the gesture factor obtaining process, and thus improves the efficiency of gesture recognition.

[0246] In some embodiments, the gesture factors have a competition priority, which is used to indicate the order in which each gesture factor in at least one type of gesture factor participates in the gesture recognition process. Optionally, the gesture factors with a higher competition priority are given priority consideration in the recognition process of the gesture to be recognized. Optionally, if a gesture control type to which the gesture to be recognized belongs is determined according to a certain gesture factor, other gesture factors in the at least one type of gesture factor that are lower than the competition priority no longer participate in the step of determining the gesture control type to which the gesture to be recognized belongs.

[0247] Optionally, each type of gesture factor is arranged in descending order of competition priority, for example, the number factor, the direction factor, and the gravity center factor. For example, the speed factor and the distance factor can be used if the direction factor satisfies the factor determination condition corresponding to the direction factor.

[0248] Exemplarily, in a case where the direction factor satisfies a determination condition corresponding to the direction factor, the first system service in the electronic device verifies, according to the speed factor and the distance factor, whether the to-be-identified gesture belongs to the sliding control type.

[0249] In one example, in a case where the speed factor is greater than or equal to a speed threshold value and the distance factor is greater than or equal to a distance threshold value, the to-be-identified gesture belongs to the sliding control type; in a case where the speed factor is less than the speed threshold value or the distance factor is less than the distance threshold value, the to-be-identified gesture does not belong to the sliding control type. The speed threshold value and the distance threshold value are pre-set.

[0250] Exemplarily, the electronic device determines the calculation order of at least one type of gesture factor according to the competition priority, and determines at least one type of gesture factor according to the gesture sensing information in the calculation order. For a type of gesture factor, the higher the competition priority of the gesture factor, the earlier the calculation order of the type of gesture factor can be.

[0251] For example, the at least one type of gesture factor includes gesture factor 1, gesture factor 2 and gesture factor 3, and the three gesture factors are arranged in the order from high to low of the competition priority as follows: gesture factor 1, gesture factor 3, gesture factor 2, and the three gesture factors are arranged in the order from early to late of the calculation order as follows: gesture factor 1, gesture factor 3, gesture factor 2. In this example, in the process of performing step S320, the electronic device first acquires gesture factor 1 according to the gesture sensing information, and finally acquires gesture factor 2 according to the gesture sensing information.

[0252] The above embodiment introduces the process of performing step S320 by the electronic device to obtain at least one type of gesture factor. After determining at least one type of gesture factor, the electronic device performs step S330 to determine the gesture control type to which the to-be-identified gesture belongs according to at least one type of gesture factor, and obtains the gesture recognition result of the to-be-identified gesture. The following several embodiments introduce this step.

[0253] S330, the first system service acquires a first gesture corresponding to the gesture sensing information according to at least one type of gesture factor.

[0254] The first gesture refers to a gesture matched with the to-be-identified gesture. That is, the first gesture is a gesture corresponding to a user operation recognized by the electronic device. Optionally, in a case where gesture recognition is successful, the first system service in the electronic device acquires information for describing the first gesture. Exemplarily, the information for describing the first gesture is also referred to as a gesture recognition result, and both have the same meaning.

[0255] In some embodiments, the information for describing the first gesture is used to trigger execution of a first operation corresponding to the first gesture. Optionally, the information for describing the first gesture includes a gesture identifier of the first gesture. For example, the gesture identifier can be a name, a serial number, or the like of the first gesture.

[0256] Optionally, the information for describing the first gesture includes a type identifier and gesture attributes. For the type identifier and gesture attributes, please refer to the foregoing description. For example, the information for describing the first gesture includes a type identifier of a sliding control type, a motion direction attribute of rightward, and a total number of gesture recognition points of 3, when the first gesture is a three-finger rightward sliding gesture.

[0257] Optionally, the information for describing the first gesture includes first information and second information, the first information is determined according to the first gesture attribute and the motion attribute, and the second information is a total number of gesture recognition points.

[0258] The first gesture attribute is used to represent a first gesture control type to which the first gesture belongs. For example, the first gesture attribute is a type identifier of the first gesture control type. For a method of determining the first gesture attribute, please refer to the following description.

[0259] For example, the first information is used to indicate a motion type of the first gesture. For example, the first information can be composed of the first gesture attribute and the motion attribute. For example, the first information is a motion type of rightward sliding, when the first gesture is a three-finger rightward sliding gesture.

[0260] Optionally, in a case of a failed gesture recognition, a first system service of the electronic device generates a gesture recognition result for prompting that a to-be-recognized gesture is invalid. For example, the gesture recognition result includes a failure prompt information. The failure prompt information is used to represent a failure of the current gesture recognition. The failure prompt information can be represented as “error” or “fail”, or the like.

[0261] In one example, in a case of a failed gesture recognition, the first system service in the electronic device does not respond to the to-be-recognized gesture. For example, if the gesture recognition fails, the competition and recognition module does not send a gesture recognition result to any application, so as to reduce interference of invalid gesture recognition results on the application.

[0262] In another example, in a case of a failed gesture recognition, the competition and recognition module sends a sensing information set to at least one application, the sensing information set including gesture sensing information respectively acquired at at least one time instant in response to a user operation; for a second application of the at least one application, the second application starts a gesture recognition process of application content according to the sensing information set after receiving the sensing information set.

[0263] Any application in the at least one application is provided with gesture recognition logic. For example, the electronic device determines whether an application has independent gesture recognition logic. If the application has gesture recognition logic, the electronic device sends the set of sensing information to the application; if the application does not have gesture recognition logic, the set of sensing information does not need to be sent to the application.

[0264] Optionally, the at least one application includes an application in a running state in the electronic device. For example, for an application, if there is a running process of the application in the CPU of the electronic device, the electronic device sends the set of sensing information to the application. The second application can be any one of the at least one application. For example, the second application is a third-party application in the at least one application. Gesture recognition logic on which the second application depends for gesture recognition can be designed by a developer of the second application.

[0265] The process of determining the first gesture according to the gesture factor is described below through several embodiments.

[0266] In some embodiments, a first system service in the electronic device obtains a first gesture according to at least one type of gesture factor, including: obtaining a first gesture attribute according to the at least one type of gesture factor; obtaining at least one second gesture attribute corresponding to a first gesture control type of the first gesture according to the at least one type of gesture factor; and obtaining the first gesture according to the first gesture attribute and the at least one second gesture attribute.

[0267] The first gesture control type is a gesture control type to which the first gesture belongs, and the first gesture attribute is used to represent the first gesture control type to which the first gesture belongs. For the types of gesture control types, refer to the embodiments above. The second gesture attribute is used to represent the specific implementation of the first gesture. The second gesture attributes corresponding to different types of gesture control types are not completely the same.

[0268] Optionally, the second gesture attribute includes at least one of the following: the total number of gesture recognition points, motion attribute. The motion attribute corresponding to different gesture control types is different. For example, the motion attribute corresponding to the sliding control type includes a sliding direction attribute, the motion attribute corresponding to the scaling control type includes a scaling trend attribute, and the motion attribute corresponding to the rotation control type includes a rotation direction attribute. For optional values of the motion attribute, refer to the embodiments above.

[0269] In some embodiments, the at least one type of gesture factor includes multiple types of gesture factors with competitive priorities, and the competitive priority is used to represent the order in which each type of gesture factor participates in gesture recognition in the multiple types of gesture factors.

[0270] In some embodiments, the first system service acquires the first gesture attribute according to the at least one gesture factor includes: the first system service acquires a target gesture factor, the target gesture factor being a gesture factor with the highest competitive priority among the plurality of gesture factors. Then, the first system service acquires a judgment condition corresponding to the target gesture factor, and judges whether the target gesture factor satisfies the corresponding judgment condition. In the case that the target gesture factor satisfies the corresponding judgment condition, the first system service determines the first gesture attribute according to a gesture control type corresponding to the target gesture factor. In the case that the target gesture factor does not satisfy the corresponding judgment condition, the first system service updates the target gesture factor to a gesture factor with a higher competitive priority among gesture factors in the plurality of gesture factors that have not participated in gesture recognition, and repeatedly performs the operation of acquiring the judgment condition corresponding to the target gesture factor to judge whether the target gesture factor satisfies the corresponding judgment condition until the first gesture attribute is determined or all gesture factors in the plurality of gesture factors have participated in gesture recognition.

[0271] In the case that the plurality of gesture factors exist, the target gesture factor is a gesture factor with a higher competitive priority among the plurality of gesture factors. For example, the competitive priority of the target gesture factor is higher than or equal to the competitive priority of other gesture factors in the plurality of gesture factors. For example, the target gesture factor is a gesture factor with the highest competitive priority among the optional gesture factors.

[0272] For example, the plurality of gesture factors include gesture factor 1, gesture factor 2 and gesture factor 3, and the three gesture factors are sorted according to the competitive priority from high to low as follows: gesture factor 1, gesture factor 2, gesture factor 3. Gesture factor 1 has been selected once in step S330, and gesture factor 2 has not been selected in step S330, so gesture factor 2 is determined as the target gesture factor.

[0273] In this way, it is helpful to ensure that each gesture factor participates in the process of determining the gesture recognition result in step S330 once, so as to avoid repeated judgment of the same gesture factor, and it is helpful to improve the efficiency of the gesture recognition process.

[0274] Optionally, the selection times of the gesture factors at different time points are independent of each other and do not affect each other. For gesture factors of the same type, the gesture factors corresponding to the same gesture recognition point obtained according to gesture sensing information at different time points have selection times that are not cumulative.

[0275] For example, the selection times of the direction factor 1 do not affect the selection times of the direction factor 2, wherein the direction factor 1 is acquired according to the gesture sensing information at time 1, and the direction factor 2 is determined according to the gesture sensing information at time 2, and the time 2 is later than the time 1.

[0276] The determination condition corresponding to the target gesture factor is used to determine whether the gesture parameter represented by the target gesture factor meets a condition. In the embodiments of the present application, the determination condition is also referred to as a factor determination condition. Optionally, when the total number of types of the plurality of gesture factors is greater than or equal to 2, there are at least two gesture factors corresponding to different determination conditions.

[0277] The determination condition in the embodiments of the present application distinguishes between various gesture control types by using the similarities and differences of various gesture factors. For example, a certain gesture control type has the same characteristics on the first gesture factor as other gesture control types, and the characteristics of the first gesture factor are different from those of other gesture control types. Through the determination condition corresponding to the first gesture factor, the gesture recognition process can recognize the gesture control type.

[0278] In some embodiments, the first system service in the electronic device acquires the first gesture attribute according to the gesture control type corresponding to the target gesture factor, including: the first system service acquires the first gesture attribute used to represent the gesture control type corresponding to the target gesture factor.

[0279] The gesture control type corresponding to the target gesture factor is related to the type of the target gesture factor. Optionally, different types of gesture factors correspond to different gesture control types. For example, the gesture control type corresponding to the target gesture factor is pre-set in the gesture recognition algorithm.

[0280] In some embodiments, the plurality of gesture factors includes a direction factor and a gravity center factor. Optionally, the competitive priority of the direction factor is higher than that of the gravity center factor. For example, the gesture control type corresponding to the direction factor is a sliding control type, and the gesture control type corresponding to the gravity center factor is a rotating control type. For example, the gesture control type corresponding to the direction factor is a sliding control type, and the gesture control type corresponding to the gravity center factor is a zooming control type.

[0281] For example, the target gesture factor is a direction factor, and the gesture parameter described by the direction factor meets the determination condition corresponding to the direction factor. Then, the first gesture attribute is acquired according to the gesture control type corresponding to the direction factor. In this case, the first gesture attribute is used to represent the sliding control type.

[0282] For example, the competitive priority of the direction factor is higher than that of the gravity center factor, the gesture control type corresponding to the direction factor is a sliding control type, and the gesture control type corresponding to the gravity center factor is a zooming control type.

[0283] Optionally, the competition priority of the gravity factor is higher than the competition priority of the direction factor, the gesture control type corresponding to the direction factor is a sliding control type, and the gesture control type corresponding to the gravity factor is a rotating control type.

[0284] In some embodiments, the plurality of gesture factors includes a direction factor and a gravity factor, and the competition priority of the direction factor is higher than the competition priority of the gravity factor.

[0285] Optionally, the competition priority of a gesture factor is in a positive correlation with the category of the gesture control type associated with the gesture factor in the gesture recognition process. For example, in the case where the competition priority of the direction factor is set to be higher than the competition priority of the gravity factor, the determination condition corresponding to the direction factor can be used to determine whether the first gesture control type belongs to the sliding control type or the non-sliding control type. That is, whether the gesture is of the sliding control type or of the non-sliding control type, the direction factor needs to be used in the gesture recognition process. Therefore, the gesture control types associated with the direction factor include the sliding control type, the non-sliding control type (such as the zooming control type and the rotating control type).

[0286] The determination condition corresponding to the gravity factor is used to determine which one of the non-sliding control types the first gesture control type belongs to. Therefore, the gesture control types associated with the gravity factor include the zooming control type and the rotating control type. It can be seen that the category of the gesture control types associated with the direction factor is three, and the category of the gesture control types associated with the gravity factor is two. Therefore, the competition priority of the direction factor is higher than the competition priority of the gravity factor.

[0287] As can be seen from the above embodiment, the calculation amount in the process of obtaining the direction factor is smaller than the calculation amount in the process of obtaining the gravity factor. By setting the competition priority of the direction factor to be higher than the competition priority of the gravity factor, for the first gesture of the sliding control type, after the direction factor is determined, the determination condition corresponding to the direction factor can be used to identify that the first gesture control type is of the sliding control type. Therefore, in this setting mode of the competition priority, for the first gesture of the sliding control type, the gravity factor does not need to be calculated according to the gesture sensing information, thereby helping to reduce the calculation amount of gesture recognition and helping to improve the efficiency of the gesture device.

[0288] The determination conditions corresponding to the gesture factors of various types are introduced below.

[0289] Optionally, the determination condition corresponding to the direction factor is that there are a plurality of gesture recognition points greater than or equal to the first quantity, each of the plurality of gesture recognition points corresponding to the same direction factor; and the gesture control type corresponding to the direction factor is a sliding control type. Illustratively, in the case that the first quantity of gesture recognition points each correspond to the same direction factor, a first system service in the electronic device acquires a first gesture attribute used to describe the sliding control type; and in the case that the first quantity of direction factors are the same, the first system service determines that the gesture to be recognized does not belong to the sliding control type.

[0290] The first quantity is related to a total number of gesture recognition points of the gesture to be recognized, and the first system service in the electronic device determines the first quantity through the quantity factor of the gesture to be recognized. Illustratively, the first quantity is greater than or equal to 1 / g times the total number of gesture recognition points, and less than or equal to the total number of gesture recognition points, g being a positive number greater than 1. For example, the first quantity is equal to the total number of gesture recognition points of the gesture to be recognized.

[0291] In one example, for a certain gesture to be recognized, the gesture to be recognized includes three gesture recognition points, each of the three gesture recognition points corresponding to a direction factor of left, and the three direction factors being the same; and the first system service determines that the gesture parameter corresponding to the direction factor satisfies the corresponding factor determination condition of the direction factor.

[0292] Optionally, the determination condition corresponding to the gravity center factor is that there are a second quantity of gesture recognition points, and for each gesture recognition point in the second quantity of gesture recognition points, the change amount of the gravity center factor corresponding to the gesture recognition point is greater than or equal to a first distance threshold; and the gesture control type corresponding to the gravity center factor is a zoom control type.

[0293] Illustratively, in the case that there are a second quantity of gesture recognition points, and for each gesture recognition point in the second quantity of gesture recognition points, the change amount of the gravity center factor corresponding to the gesture recognition point is greater than or equal to a first distance threshold, a first system service in the electronic device acquires a first gesture attribute used to represent the zoom control type; and in the case that there are a fourth quantity of gesture recognition points, and for each gesture recognition point in the fourth quantity of gesture recognition points, the change amount of the gravity center factor corresponding to the gesture recognition point is greater than or equal to the first distance threshold, the electronic device determines that the gesture to be recognized does not belong to the zoom control type. Illustratively, in the case that the gravity center factor does not satisfy the determination condition corresponding thereto, the electronic device acquires a first gesture attribute used to represent a rotation control type.

[0294] The fourth quantity is less than the second quantity (i.e., there is no second quantity of gesture recognition points), the second quantity being a positive integer, and the third quantity being a natural number.

[0295] The change amount of the gravity center factor is used to represent the change of the distance between the gesture recognition point and the gravity center of the geometric figure. For example, for a gesture recognition point, the change amount of the gravity center factor is related to the gravity center factor of the gesture recognition point at the first time and the gravity center factor of the gesture recognition point at the second time. The gravity center factor at the first time is obtained according to the gesture sensing information at the first time, and the gravity center factor at the second time is obtained according to the gesture sensing information at the second time.

[0296] For example, the change amount of the gravity center factor is equal to the gravity center factor at the second time minus the gravity center factor at the first time. For the first time, the second time and the gravity center factor, please refer to the introduction of step S320.

[0297] The second number is related to the total number of gesture recognition points. Optionally, the electronic device determines the second number through the number factor of the gesture to be recognized. For example, the second number is greater than or equal to 1 / h times of the total number of gesture recognition points, and less than or equal to the total number of gesture recognition points, h is a positive number greater than 1. For example, the second number is equal to the first number, or the second number is irrelevant to the first number.

[0298] Optionally, the determination condition corresponding to the gravity center factor is that there are a third number of gesture recognition points, and for each gesture recognition point in the third number of gesture recognition points, the change amount of the gravity center factor of the gesture recognition point is less than or equal to the second distance threshold. In this case, the gesture control type corresponding to the gravity center factor is the rotation control type.

[0299] The third number is related to the total number of gesture recognition points corresponding to the gesture to be recognized. For example, the third number is greater than or equal to half of the total number of gesture recognition points. For example, the second number is equal to the third number. The second distance threshold is pre-set. For example, the second distance threshold is equal to the first distance threshold, or the second distance threshold is any positive number irrelevant to the first distance threshold.

[0300] For example, in the case that there are a third number of gesture recognition points, and the change amount of the gravity center factor of each gesture recognition point in the third number of gesture recognition points is less than or equal to the second distance threshold, the electronic device determines that the gesture control type to which the gesture to be recognized belongs is the rotation control type; in the case that there is no third number of gravity center factors in the gravity center factors, the first system service obtains the first gesture attribute used to represent the rotation control type. For example, in the case that the gravity center factor does not satisfy the determination condition corresponding to the gravity center factor, the electronic device determines that the gesture to be recognized does not belong to the zoom control type.

[0301] In some embodiments, for a target gesture factor in the plurality of gesture factors, if the target gesture factor satisfies a corresponding determination condition, the first system service in the electronic device determines a first gesture attribute as a gesture attribute used to represent a gesture control type corresponding to the target gesture factor.

[0302] In a possible implementation, the first system service in the electronic device acquires the first gesture attribute according to at least one gesture factor, including: the first system service acquires a first gesture factor from the plurality of gesture factors, the first gesture factor being a gesture factor with a highest priority in the plurality of gesture factors; and in a case where the first gesture factor satisfies a corresponding determination condition, the first system service determines the first gesture attribute according to a gesture control type corresponding to the first gesture factor.

[0303] The determined first gesture attribute is used to represent the gesture control type corresponding to the first gesture factor.

[0304] For example, the plurality of gesture factors includes a direction factor and a gravity center factor, the direction factor has a higher priority than the gravity center factor, and the first gesture factor is the direction factor.

[0305] In another possible implementation, the first system service in the electronic device acquires the first gesture attribute according to at least one gesture factor, including: the first system service acquires a first gesture factor from the plurality of gesture factors, the first gesture factor being a gesture factor with a highest priority in the plurality of gesture factors; in a case where the first gesture factor does not satisfy a corresponding determination condition, the first system service acquires a second gesture factor from the plurality of gesture factors, the second gesture factor being a gesture factor with a highest priority in gesture factors that do not participate in gesture recognition in the plurality of gesture factors; and in a case where the second gesture factor satisfies a corresponding determination condition, the first system service determines the first gesture attribute according to a gesture control type corresponding to the second gesture factor.

[0306] The determined first gesture attribute is used to represent the gesture control type corresponding to the second gesture factor.

[0307] For example, the plurality of gesture factors includes a direction factor and a gravity center factor, the direction factor has a higher priority than the gravity center factor, and the second gesture factor is the gravity center factor in a case where the direction factor has been selected once.

[0308] Optionally, the electronic device can further cyclically execute the step 320 and the step 330. In each execution of the step 320, the electronic device determines part of the gesture factors in the at least one type of gesture factors according to the gesture sensing information. For example, in the execution of the step 320, the electronic device determines a gesture factor with a higher competitive priority according to the gesture sensing information according to the competitive priority; then, the electronic device executes the step S330; if the step 330 cannot identify the gesture control type of the to-be-identified gesture according to the gesture factor, the electronic device reselects a new gesture factor according to the competitive priority, and the electronic device identifies the step S320 and the step S330. The electronic device cyclically executes the step S320 and the step S330 in the above manner until the identification of the to-be-identified gesture is completed or the cycle exceeds a maximum cycle number.

[0309] Since the multiple types of gesture factors have the competitive priority, if the type of the to-be-identified gesture can be determined through the gesture factor with the higher competitive priority, the gesture factors with the relatively lower competitive priority do not participate in the gesture identification process. Therefore, the step S320 is executed in stages, and each time part of the gesture factors in the at least one type of gesture factors are calculated, which helps to ensure that all the calculated gesture factors can participate in the gesture identification process, thereby avoiding the situation that all the gesture factors are calculated once, but part of the gesture factors do not participate in the gesture identification.

[0310] In this way, the total amount of gesture factors that need to be calculated is reduced, thereby improving the calculation efficiency of the electronic device in the gesture identification process. Since the gesture factors with the higher competitive priority in the at least one type of gesture factors preferentially participate in the gesture identification process, the gesture factors with the higher competitive priority are preferentially calculated in the execution of the step S320 in stages, which does not affect the normal progress of the gesture identification process, helps to avoid the calculation of useless gesture factors, helps to improve the gesture identification efficiency of the electronic device, and improves the response speed of the gesture interaction.

[0311] In some embodiments, before the first system service in the electronic device acquires the first gesture according to the first gesture attribute and the at least one second gesture attribute, the first system service further includes: the first system service acquires a third gesture factor from the at least one type of gesture factors; and in a case where the third gesture factor satisfies an effectiveness condition, the first system service acquires the first gesture attribute according to the gesture control type corresponding to the selected gesture factor. Optionally, in a case where the third gesture factor does not satisfy the effectiveness condition, the first system service repeatedly executes the step of acquiring the gesture sensing information in response to the user operation to start until the first gesture is acquired or the user operation disappears.

[0312] Optionally, the third gesture factor is used to verify the effectiveness of the user operation. For example, the third gesture factor can be used to verify whether the user operation is used to implement the gesture control. For example, the type of the third gesture factor includes at least one of the following: a speed factor and a distance factor.

[0313] In one example, the validity condition corresponding to the speed factor is that the speed factor of the at least one gesture recognition point is greater than or equal to a speed threshold, and the validity condition corresponding to the distance factor is that the distance factor of the at least one gesture recognition point is greater than or equal to a second distance threshold.

[0314] The selected gesture factor refers to a gesture factor selected from the plurality of gesture factors in the process of acquiring the first gesture attribute according to the plurality of gesture factors. For example, the selected gesture factor is a currently selected target gesture factor. For example, the selected gesture factor includes the first gesture factor and the second gesture factor in the above embodiments.

[0315] Optionally, in the case where the speed factor is greater than or equal to the speed threshold and the distance factor is greater than or equal to the second distance threshold, the first system service in the electronic device continues to perform the step of acquiring the first gesture according to the first gesture attribute and the at least one second gesture attribute; in the case where the speed factor is less than the speed threshold or the distance factor is less than the second distance threshold, it indicates that the user operation does not move or the degree of movement can be ignored in a period of time. In this case, the first system service acquires gesture sensing information collected at the next detection time.

[0316] In the case where the third gesture factor meets the validity condition, the first gesture attribute is acquired according to the gesture control type corresponding to the selected gesture factor, including: acquiring the first gesture attribute for representing the gesture control type corresponding to the selected gesture factor.

[0317] The validity of the user operation is verified through the third gesture factor, so as to expand the gesture recognition in the case where the user operation produces a certain displacement or has a certain speed. The third gesture factor can filter out invalid user operations with small movement amplitudes, thereby helping to improve the accuracy of the gesture recognition process. In addition, in this way, it is also helpful to avoid the electronic device from identifying a user error touch as a user operation, thereby helping to avoid the case where the first application is notified to perform the first operation in the case where the user does not need to perform the first operation, resulting in abnormal operation of the electronic device.

[0318] In some embodiments, the first system service in the electronic device acquires the first gesture according to the first gesture attribute and the at least one second gesture attribute, including: the first system service continuously acquires n predicted gestures according to the corresponding first gesture attribute and the at least one second gesture attribute during the duration of the user operation, n being a positive integer; in the case where the n predicted gestures meet an accuracy condition, the first system service acquires the first gesture according to the predicted gesture.

[0319] The corresponding first gesture attribute and the at least one second gesture attribute are determined by the same gesture sensing information. The corresponding first gesture attribute and the at least one second gesture attribute can determine a predicted gesture.

[0320] The n predicted gestures are respectively determined according to each of the n gesture sensing information before the first gesture is determined.

[0321] Optionally, during the determination of the first gesture, the first system service acquires gesture sensing information every first time interval during the user operation. Before the first gesture is determined, for each acquired gesture sensing information, the first system service acquires at least one gesture factor according to the gesture sensing information. Then, the first system service acquires a first gesture attribute and at least one second gesture attribute according to the at least one gesture factor. The first gesture attribute and the at least one second gesture attribute mentioned here are the corresponding first gesture attribute and the at least one second gesture attribute. The first system service acquires a predicted gesture according to the corresponding first gesture attribute and the at least one second gesture attribute. The predicted gesture is a specific gesture.

[0322] For example, the corresponding first gesture attribute and the at least one second gesture attribute indicate that the zoom control type, pinch and 3 fingers, and then the 3-finger pinch is the predicted gesture determined this time.

[0323] For example, the n predicted gestures are respectively acquired according to n continuously acquired gesture sensing information. For example, the n continuously acquired gesture sensing information refers to n gesture sensing information acquired every first time interval.

[0324] The accuracy condition is related to the consistency between the n predicted gestures. Optionally, the accuracy condition includes that the n predicted gestures include m same predicted gestures, and m is a positive integer less than or equal to n. For example, the m same predicted gestures are respectively acquired according to m continuously acquired gesture sensing information. For example, the m predicted gestures are the latest continuously determined m predicted gestures in the n predicted gestures. m is a pre-set positive integer, such as m is equal to 3, 4, 5, 6, etc.

[0325] For example, in the case that the n predicted gestures include m same predicted gestures, the first system service determines any one of the m predicted gestures as the first gesture; in the case that the n predicted gestures do not include m same predicted gestures, the first system service re-executes the steps of acquiring gesture sensing information and acquiring predicted gestures according to the gesture sensing information until the first gesture is determined or the user operation disappears.

[0326] In one example, during the duration of the user operation, the first system service determines the predicted gesture according to the first gesture attribute and the second gesture attribute; the first system service updates the prediction times of the predicted gesture; in the case that the prediction times is less than the duration times threshold, the first system service repeats the step of acquiring the gesture sensing information in response to the user operation until the first gesture is acquired or the user operation disappears. In the case that the prediction times is equal to the duration times threshold, the first system service determines the predicted gesture as the first gesture.

[0327] The prediction times is used to represent the cumulative value of the same predicted gesture determined in the process of repeatedly performing the step S330. Optionally, after a certain predicted gesture is acquired, the first system service increases the prediction times of the predicted gesture by 1. Illustratively, in the case that the prediction times of the predicted gesture is less than the duration times threshold, the first system service resets the prediction times of other predicted gestures except the predicted gesture.

[0328] Illustratively, the first gesture is the subscription gesture whose prediction times first reaches the duration times threshold in the at least one subscription gesture.

[0329] Optionally, the duration times threshold is pre-set. For example, the duration times threshold is 3, 4, 5, 6, etc. By setting the duration times threshold, it is avoided that the electronic device matches the to-be-recognized gesture with an error gesture due to the fluctuation of the moving track of the to-be-recognized gesture, and thus the error of gesture control is avoided.

[0330] In one example, during the gesture recognition, the electronic device performs the step S330 for 3 times; in the first time of performing the step S330, the determined first gesture is the sliding gesture 1, in the second time of performing the step S330, the determined first gesture is the zoom gesture 1, and in the third time of performing the step S330, the determined first gesture is the zoom gesture 1; after the 3 times of performing the step S330, the prediction times of the sliding gesture 1 is equal to 1, and the prediction times of the zoom gesture 1 is equal to 2.

[0331] The embodiment sets and updates the prediction times, ensures that the gestures continuously recognized according to the gesture sensing information are consistent, and thus helps to eliminate the interference of the user operation jitter to the gesture recognition, and helps to improve the accuracy of the gesture recognition.

[0332] In some embodiments, taking a touch screen gesture as an example, the to-be-recognized gesture needs to go through at least three types of events from start to end, including a touch (Down) event, a movement (Move) event, and a lift (Up) event. The Down event is used to describe that the to-be-recognized gesture contacts with a sensing device (such as a touch screen) of the electronic device; the Move event is used to describe that the to-be-recognized gesture moves on the touch screen; and the Up event is used to describe that the to-be-recognized gesture leaves the touch screen. Optionally, step S310 is performed during the Down event and / or the Move event, and steps S320 and S330 are performed during the Move event. For example, each Move event corresponds to one gesture naked event, that is, in each Move event, the first system service obtains gesture sensing information once, and determines a corresponding predicted gesture based on the obtained gesture sensing information. In the case that the corresponding predicted gestures in the plurality of Move events are consistent, the first system service determines that the predicted gesture is the first gesture.

[0333] In this embodiment, the electronic device performs the gesture recognition step during the Down event and the Move event when detecting the to-be-recognized gesture, which helps to determine the gesture recognition result of the to-be-recognized gesture before the Up event, and helps to quickly determine the to-be-recognized gesture, so as to improve the response speed of the electronic device to the to-be-recognized gesture.

[0334] The following describes a logical diagram of at least one gesture recognition by an example. In this embodiment, there is a competition priority among the at least one type of gesture factor, and the gesture factors are arranged in descending order of the competition priority as follows: the quantity factor, the direction factor, and the gravity center factor. As shown in FIG. 9, the gesture recognition method includes at least the following steps, and the execution subject of the following steps is the first system service in the electronic device.

[0335] Step S910, after detecting the Down event of the to-be-recognized gesture, the first system service obtains and stores gesture sensing information at a second time. The second time can be an initial time after detecting the to-be-recognized gesture. Optionally, the gesture sensing information includes position information of at least one gesture recognition point.

[0336] Optionally, the electronic device obtains the gesture sensing information at the first time in response to a user operation.

[0337] Step S920, the first system service obtains and stores the gravity center factor at the second time according to the gesture sensing information at the second time. For the obtaining manner of the gravity center factor, refer to the above embodiment.

[0338] Step S930, after detecting the Move event of the to-be-recognized gesture, the electronic device obtains and stores the gesture sensing information at the first time.

[0339] In step S940, the first system service detects whether the gesture sensing information meets the competition recognition condition.

[0340] Optionally, the competition recognition condition is that the function gesture matched by the to-be-recognized gesture is not determined, and the to-be-recognized gesture moves. Illustratively, in the case that the function gesture matched by the to-be-recognized gesture is not determined and the to-be-recognized gesture moves, the first system service executes step S950. Illustratively, in the case that the to-be-recognized gesture does not move, the first system service executes step S930; in the case that the function gesture matched by the to-be-recognized gesture is determined, the first system service determines that the gesture recognition is completed.

[0341] In one example, in the case that the first gesture matched by the to-be-recognized gesture is determined, the first system service acquires updated gesture sensing information; and the first system service acquires motion parameters of the first gesture according to the updated gesture sensing information. The motion parameters of the first gesture are used to indicate the motion condition of the first gesture, and details are described below with reference to S350.

[0342] In step S950, the first system service acquires the direction factor at the first time according to the gesture sensing information at the first time.

[0343] Optionally, the first system service acquires the direction factor at the first time according to the gesture sensing information at the first time and reference sensing information. Illustratively, the reference sensing information is the gesture sensing information at the second time. The reference sensing information includes position reference information used to represent the historical position of the gesture recognition point.

[0344] Optionally, in step S950, the first system service further acquires the quantity factor and the speed factor at the first time according to the gesture sensing information at the first time.

[0345] In step S960, the first system service determines whether the direction factor at the first time meets the determination condition corresponding to the direction factor.

[0346] Optionally, if the direction factor at the first time meets the determination condition, the first system service executes step S970; if the direction factor at the first time does not meet the determination condition, the first system service executes step S980.

[0347] Optionally, before step S960 is executed, the first system service first determines which type of multi-finger gesture operation the to-be-recognized gesture belongs to according to the quantity factor; and after the multi-finger gesture operation to which the to-be-recognized gesture belongs is determined, the first system service executes step S960. For example, in the case that the quantity factor is 4, the first system service determines that the to-be-recognized gesture belongs to a four-finger gesture operation. For another example, in the case that the quantity factor is 3, the first system service determines that the to-be-recognized gesture belongs to a three-finger gesture operation.

[0348] As shown in FIG. 10, in the case that the to-be-recognized gesture belongs to the sliding control gesture, the moving direction of the gesture recognition point is basically consistent during the duration of the to-be-recognized gesture. Therefore, the gesture belonging to the sliding control type can be recognized by the determination condition corresponding to the sliding factor.

[0349] In step S970, the first system service acquires the first gesture attribute describing the sliding control type.

[0350] Optionally, during the execution of step S970, after determining that the direction factor meets the determination condition corresponding thereto, the first system service acquires the speed factor and / or the distance factor at the first time; in the case that the speed factor at the first time is greater than or equal to a speed threshold value, and / or the distance factor at the first time is greater than or equal to a second distance threshold value, the first system service determines that the first gesture control type is the sliding control type, otherwise, the first system service re-executes step S930. Exemplarily, the speed threshold value and the second distance threshold value are pre-set.

[0351] Since the user operation performed by the user is not very stable, the jitter of the to-be-recognized gesture is likely to provide interference information for the gesture recognition process, resulting in errors in gesture recognition. By setting the speed threshold value and the distance threshold value, the interference data caused by the jitter can be eliminated, and the accuracy of gesture recognition can be improved.

[0352] In step S980, the first system service acquires the gravity center factor at the first time according to the gesture sensing information at the first time.

[0353] In step S990, the first system service determines the change amount of the gravity center factor according to the gravity center factor at the first time and the gravity center factor at the second time of each gesture recognition point.

[0354] In step S991, the first system service determines whether the gravity center factor at the first time meets the determination condition corresponding to the gravity center factor.

[0355] Optionally, the determination condition corresponding to the gravity center factor is that the second number of gesture recognition points exist, and for each gesture recognition point in the second number of gesture recognition points, the change amount of the gravity center factor corresponding to the gesture recognition point is greater than or equal to a first distance threshold value. Optionally, in the case that the gravity center factor at the first time meets the determination condition corresponding to the gravity center factor, the electronic device executes step S992; in the case that the gravity center factor at the first time does not meet the determination condition corresponding to the gravity center factor, the electronic device executes step S993.

[0356] Please refer to FIG. 11 and FIG. 12 for comparison between the gesture of the zoom control type and the gesture of the rotation control type. As shown in FIG. 11, in the case that the gesture belongs to the zoom control type, the position of the center of gravity of the geometric figure does not change substantially during the user operation, and the distance between the gesture recognition point and the center of gravity gradually increases. The gesture to be recognized belonging to the zoom control type (such as the spread gesture and the pinch gesture) can be recognized by the factor determination condition corresponding to the center of gravity factor. As shown in FIG. 12, in the case that the gesture to be recognized belongs to the rotation control type, the position of the center of gravity of the geometric figure does not change substantially during the user operation, and the distance between the gesture recognition point and the center of gravity also does not change substantially.

[0357] In step S992, the first system service acquires the first gesture attribute for representing the zoom control type.

[0358] Optionally, in step S992, the first system service determines whether the first type is the pinch gesture or the spread gesture by the change direction of the value of the center of gravity factor. Exemplarily, in the case that the change direction of the center of gravity factor is value increase, the first system service determines that the first type is the spread gesture; in the case that the change direction of the value of the center of gravity factor is value decrease, the electronic device determines that the first type is the pinch gesture.

[0359] In step S993, the first system service acquires the first gesture attribute for representing the rotation control type.

[0360] Optionally, in step S993, the electronic device determines whether the first type is the clockwise rotation gesture or the counterclockwise rotation gesture according to the direction factor.

[0361] In step S994, the first system service acquires at least one second gesture attribute according to at least one gesture factor.

[0362] Optionally, the at least one second attribute includes the total number of gesture recognition points and the motion attribute. Exemplarily, the electronic device determines the total number of gesture recognition points according to the number factor in the at least one gesture factor.

[0363] Exemplarily, in the case that the first gesture attribute is for representing the sliding control type, the electronic device acquires the sliding direction attribute according to the direction factor. Exemplarily, in the case that the first gesture attribute is for representing the zoom control type, the electronic device acquires the zoom trend attribute according to the change amount of the center of gravity factor. Exemplarily, in the case that the first gesture attribute is for representing the rotation control type, the electronic device acquires the rotation direction attribute according to the direction factor.

[0364] In step S995, the first system service acquires the predicted gesture according to the first gesture attribute and the at least one second gesture attribute.

[0365] Optionally, the first system service combines the first gesture attribute and the at least one second gesture attribute to obtain a predicted gesture.

[0366] At step S996, the first system service determines whether the predicted number of times of the predicted gesture is equal to the sustained number threshold. Optionally, in a case where the predicted number of times of the first gesture is equal to the sustained number threshold, the first system service determines the predicted gesture as the first gesture; in a case where the predicted number of times of the first gesture is less than the sustained number threshold, the first system service re-executes step S930.

[0367] Optionally, before step S996, the first system service increases the predicted number of times of the predicted gesture by one, and in a case where the predicted number of times of the predicted gesture is determined to be less than the sustained number threshold at step S996, the first system service resets the predicted number of times of other predicted gestures.

[0368] Optionally, before the first system service re-executes step S930, the first system service updates the first time and the second time. For example, the first system service takes the first time as a new second time, and takes a time at which gesture sensing information is next acquired as a new first time.

[0369] At step S997, after detecting occurrence of an Up event of the gesture to be recognized, the first system service cleans up data required in the gesture recognition process.

[0370] Optionally, the data required in the gesture recognition process includes at least one of the following: gesture sensing information at the at least one time, gesture factors at the at least one time.

[0371] The embodiment determines at least one type of gesture factor through gesture sensing information, and determines a gesture attribute to which the gesture to be recognized belongs through the at least one type of gesture factor, and further acquires the first gesture. The gesture recognition logic of different gestures is unified through setting of the competition priority, which is helpful for subsequent expansion of the gesture recognition method. Moreover, various gestures are unified into one gesture recognition logic, so that after the electronic device detects the gesture to be recognized, the gesture recognition can be completed through the unified system service (such as the competition and recognition module). Compared with the gesture recognition completed by each application alone, the embodiment is helpful for avoiding conflicts existing in the gesture recognition process of each application.

[0372] In the embodiment, all gesture factors required in the gesture recognition process are not acquired at one time, but are acquired in multiple stages. For a certain gesture factor, the gesture factor is acquired through gesture sensing information before the gesture factor needs to be used. In this way, the type of gesture factor to be determined by the electronic device is reduced without affecting the gesture recognition flow, thereby helping to reduce the computing pressure of the electronic device.

[0373] In addition, by setting the threshold of the number of continuous times in the embodiment, interference of interference data accidentally detected during the process of the gesture to be recognized on the gesture recognition process can be avoided, and the reliability of the gesture recognition result can be improved.

[0374] In some embodiments, the step S330 is completed by a competition and recognition module in the electronic device. In the step S330, the competition and recognition module determines a first gesture factor from the plurality of gesture factors; in a case where the first gesture factor meets a corresponding determination condition, the competition and recognition module determines a first gesture attribute according to a gesture control type corresponding to the first gesture factor, determines at least one second gesture attribute according to at least one gesture factor, and obtains a first gesture according to the first gesture attribute and the second gesture attribute; in a case where the first gesture factor does not meet the corresponding determination condition, the competition and recognition module determines a new first gesture factor from the plurality of gesture factors, or, in a case where all the at least one gesture factor has been selected, the competition and recognition module executes the step S310, or determines that the current gesture recognition process fails.

[0375] In the step S340, the first system service reports information describing the first gesture to the first application, so that the first application executes a first operation corresponding to the first gesture.

[0376] The first application is an application subscribing to the first gesture. For details of the gesture subscription process, refer to the above embodiment.

[0377] In some embodiments, the first application is an application supporting the first gesture in the electronic device. For example, in a case where the first gesture is a rightward sliding gesture and the application 1 is provided with a function gesture of rightward sliding, the application 1 can be the first application. Alternatively, the first application is an application displayed in a user interface of the electronic device, or an application running in the background. For example, the electronic device determines the first application by using a gesture mapping table. For details of the gesture mapping table, refer to the above embodiment.

[0378] Alternatively, the first application is an application subscribing to the first gesture. For example, before the step S340, the first application notifies the first system service of subscription information, so as to subscribe to the first gesture from the first system service.

[0379] Alternatively, in a case where there are a plurality of candidate applications subscribing to the first gesture in the applications installed in the electronic device, the first system service selects one of the plurality of candidate applications as the first application. In an example, the electronic device includes an application priority list, the application priority list being used to indicate an order of sending gesture recognition results to the applications in the electronic device; the first system service selects a candidate application with the highest priority from the plurality of candidate applications as the first application according to the priority list.

[0380] In some embodiments, after receiving the subscription information from the application, the first system service determines at least one gesture supported by the application according to the subscription information, and determines whether there is an overlapping gesture in the gesture mapping table in the at least one gesture, the overlapping gesture being a function gesture supported by at least two applications; if there is an overlapping gesture, the first system service sets an application priority between the at least two applications supporting the overlapping gesture, and stores the application priority to the priority list.

[0381] Optionally, the first operation is used to implement a gesture control triggered by the first gesture. The first operation includes, but is not limited to, at least one of the following: refreshing display content, switching display page, adjusting brightness, adjusting volume, adjusting video playing progress, calling a background running application, moving an application to background running, adjusting display size of display content (such as enlarging or reducing a picture or text), adjusting a camera shooting parameter, intercepting display content, calling a system tool (such as entering a desktop arrangement tool by pinching operation), adjusting display angle of display content, etc.

[0382] The embodiment can avoid interference of gesture recognition to other applications by notifying the application subscribing to the first gesture of performing the corresponding operation by the system service without notifying the first gesture to other application programs in the electronic device.

[0383] The following describes an interaction process of each module in the electronic device and an application in a gesture recognition process by an embodiment. As shown in FIG. 13, the embodiment mainly includes the following steps:

[0384] S1310, the first application sends subscription information to the competition and recognition module.

[0385] Optionally, the subscription information is used to indicate at least one gesture supported by the first application. Exemplarily, for each gesture supported by the first application, the subscription information includes a type identifier of a gesture control type to which the gesture belongs and an attribute parameter of the gesture.

[0386] S1320, the competition and recognition module sends subscription result to the first application.

[0387] S1330, after the user makes a gesture to be recognized, the gesture sensing module acquires gesture sensing information in response to the user operation.

[0388] S1340, the gesture sensing module sends the gesture sensing information to the competition and recognition module.

[0389] S1350, the competition and recognition module acquires the first gesture according to the gesture sensing information.

[0390] S1360, the competition and recognition module reports information describing the first gesture to the first application.

[0391] S1370, the first application executes the first operation corresponding to the first gesture according to the information used to describe the first gesture.

[0392] The embodiment takes over the gesture recognition requirement by the competition and recognition module, and the competition and recognition module acquires the gesture recognition result according to the gesture sensing information. On the one hand, compared with sending the gesture sensing information to each application and completing the gesture recognition by the application according to the gesture sensing information, the embodiment helps to avoid the phenomenon that multiple applications simultaneously perform gesture recognition on the same to-be-recognized gesture, and helps to solve the repetition and conflict of multiple applications in the gesture recognition process.

[0393] On the other hand, the competition and recognition module sends the gesture recognition result to the application in the case that the application supports the functional gesture indicated by the gesture recognition result, and does not send the gesture recognition result to the application in the case that the application does not support the functional gesture indicated by the gesture recognition result, which helps to reduce the influence of the gesture recognition result on irrelevant applications. The competition and recognition module separates the application from the gesture operation of the user, thereby reducing the interference of irrelevant gestures on the application.

[0394] In some embodiments, the gesture recognition process provided by the embodiment can further include the following steps:

[0395] Step S350, in the case that the user operation is not ended, the first system service in the electronic device acquires updated gesture sensing information in response to the user operation.

[0396] Optionally, after determining the first gesture, the electronic device detects whether the user operation is ended; in the case that the user operation is not ended, the electronic device acquires updated gesture sensing information. In the case that the user operation is ended, there is no need to acquire gesture sensing information. For example, in the case that at least one gesture recognition point is not disappeared, the electronic device acquires updated gesture sensing information.

[0397] Illustratively, after acquiring the first gesture, if the to-be-recognized gesture is not ended (for example, the gesture recognition point is not disappeared), the gesture sensing module in the electronic device acquires gesture sensing information at a third time (that is, updated gesture sensing information).

[0398] Illustratively, the third time is later than the first time at which the gesture sensing information is acquired in step 310, and there is a difference between the gesture sensing information at the third time and the gesture sensing information at the first time. For example, the first time and the third time are different by 1 first time interval. For example, the time at which the gesture sensing module acquires gesture sensing information for the jth time is the first time, the time at which the gesture sensing module acquires gesture sensing information for the (j+1)th time is the third time, and j is a natural number.

[0399] Optionally, step S350 is performed after step S340, after the gesture recognition result is sent to the first application, the electronic device determines whether the to-be-recognized gesture satisfies the follow-hand condition. In a case where the to-be-recognized gesture satisfies the follow-hand condition, the electronic device performs step S350; in a case where the to-be-recognized gesture does not satisfy the follow-hand condition, the gesture recognition process is completed.

[0400] Exemplarily, the follow-hand condition is related to the first gesture. For example, the follow-hand condition includes that the first gesture belongs to a follow-hand gesture. The follow-hand gesture refers to a gesture whose implementation effect changes with the movement of the gesture. In one example, in a case where the first gesture belongs to the follow-hand gesture, the electronic device performs S350; in a case where the first gesture does not belong to the follow-hand gesture, the electronic device does not perform S350.

[0401] Exemplarily, the follow-hand condition includes at least one of the following: the first gesture belongs to a follow-hand gesture, and the gesture recognition point does not disappear. The follow-hand gesture refers to a gesture whose implementation effect changes with the movement of the gesture.

[0402] In one example, the follow-hand condition can be that the first gesture belongs to a follow-hand gesture, and the gesture recognition point does not disappear. In a case where the first gesture belongs to the follow-hand gesture, and the gesture recognition point does not disappear, the electronic device performs step S340 and then performs S350; in a case where the first gesture does not belong to the follow-hand gesture, or the gesture recognition point disappears, step S350 is not performed.

[0403] In step S360, the first system service in the electronic device acquires the motion parameter of the first gesture according to the updated gesture sensing information.

[0404] Optionally, the motion parameter of the first gesture is used to indicate the effect of performing the first operation. Different motion parameters of the first gesture can enable the first operation to implement different gesture control operations. Exemplarily, the motion parameter of the first gesture is also referred to as effect control information. The effect control information can be used to control the implementation precision of the first operation.

[0405] Optionally, the motion parameter of the first gesture is related to the gesture control type to which the first gesture belongs. Exemplarily, for the first gesture belonging to different gesture control types, the motion parameter of the first gesture includes different parameter contents.

[0406] In some embodiments, in a case where the first gesture belongs to a sliding control type, the motion parameter of the first gesture is used to indicate the change of the sliding distance; in a case where the first gesture belongs to a zooming control type, the motion parameter of the first gesture is used to indicate the change of the zooming ratio; in a case where the first gesture belongs to a rotating control type, the motion parameter of the first gesture is used to indicate the change of the rotating angle.

[0407] The change of the sliding distance is represented by a unit sliding distance. The unit sliding distance is used to indicate the sliding distance of the gesture of the sliding control type in a unit time length. For example, the unit time length refers to a time interval in which the gesture sensing information is acquired, i.e., the first time interval in the above embodiment.

[0408] The change of the scaling ratio is used to represent the ratio between the display size of the display content and a reference display size. For example, the reference display size is the display size of the display content before the user operation starts, or the display size of the display content before the first time interval. For the scaling gesture, the unit distance change amount is included in the effect control information. The unit distance change amount is used to represent the distance change amount between the gesture recognition point of the scaling gesture and the gravity center in a unit time.

[0409] The change of the rotation angle is used to represent the included angle between the display direction of the display content and a reference display direction. For example, for the rotation gesture, the unit rotation angle is included in the effect control information. The unit rotation angle is used to represent the rotation angle of the line connecting the gesture recognition point and the gravity center in a unit time, with the gravity center as the rotation center.

[0410] In some embodiments, the electronic device acquires a gesture factor at the third time according to the gesture sensing information at the third time; and the electronic device acquires the motion parameter effect control information of the first gesture according to the gesture factor at the third time.

[0411] Optionally, in the case where the first gesture belongs to the sliding control type, the electronic device acquires a distance factor at the third time according to the gesture sensing information at the third time; and the electronic device obtains a unit sliding distance by weightedly averaging the distance factors corresponding to the gesture recognition points at the third time.

[0412] Optionally, in the case where the first gesture belongs to the scaling control type, the electronic device acquires a gravity center factor at the third time according to the gesture sensing information at the third time; the electronic device acquires a change amount of the gravity center factor corresponding to each gesture recognition point at the third time according to the gravity center factor at the third time; and then, the electronic device obtains a motion parameter by weightedly averaging the change amounts of the gravity center factors at the third time. The determination method of the change amount of the gravity center factor is described above and will not be repeated here.

[0413] For example, the electronic device obtains the change amount of each gravity center factor at the third time according to the gravity center factor at the third time and a reference gravity center factor. The reference gravity center factor can be a gravity center factor at a fourth time, which is earlier than the third time. In one example, the fourth time is separated from the third time by the first time interval.

[0414] Optionally, in the case that the first gesture belongs to a rotation control gesture, the electronic device obtains a rotation unit angle according to the gesture sensing information at the third time, the gesture sensing information at the fourth time, and the position information of the geometric barycenter.

[0415] Illustratively, for any one of the at least one gesture recognition point, the electronic device determines a first straight line according to the position information of the gesture recognition point at the third time and the position information of the geometric barycenter, and determines a second straight line according to the position information of the gesture recognition point at the fourth time and the position information of the geometric barycenter; the electronic device obtains a change angle of the gesture recognition point according to the first straight line and the second straight line; and the electronic device obtains a rotation unit angle by weighted averaging the change angles of the gesture recognition points. For the explanation of the fourth time, please refer to the previous embodiment.

[0416] In some embodiments, the process of determining the effect control information according to the gesture sensing information is implemented by a competition and recognition module. The competition and recognition module receives the gesture sensing information sent by the gesture sensing module, and performs step S350 to obtain the motion parameters of the first gesture based on the gesture sensing information according to the corresponding calculation method of the function gesture indicated by the gesture recognition result.

[0417] After the motion parameters of the first gesture are determined, the first system service reports the motion parameters of the first gesture to the first application.

[0418] Step S370, the first application continues to perform the first operation according to the motion parameters of the first gesture.

[0419] Optionally, in the case that the first gesture belongs to a sliding control type, the first application refreshes the display position of the display content or refreshes the display content according to the motion parameters of the first gesture following the user operation. As shown in FIG. 14, the first application moves the display content from the display area 1410 to the display area 1420 according to the motion parameters of the first gesture.

[0420] Optionally, in the case that the first gesture belongs to a zoom control type, the first application changes the display size of the display content according to the motion parameters of the first gesture. As shown in FIG. 15, the first application reduces the display area 1510 of the display content to the display area 1520 according to the motion parameters of the first gesture following the user operation. In the case that the display content does not change, it is equivalent to reducing the display size of the display content.

[0421] As shown in FIG. 16, at a certain time, the first gesture corresponds to five gesture recognition points P1, P2, P3, P4 and P5, and the center of the geometric figure is G; the to-be-recognized gesture corresponds to five gesture recognition points P1', P2', P3', P4' and P5' at another time, and the center of the geometric figure is G'. During the pinching operation, the positions of the geometric centers G and G' change little or do not change. The center of the geometric figure is corresponded to the center of the display area, and the display area is reduced according to the change amount of the center of the geometric figure, so that the display area can be continuously reduced with the pinching operation.

[0422] Optionally, in a case where the function gesture indicated by the gesture recognition result belongs to a rotating gesture, the first application controls a display angle of the display content according to the effect control information.

[0423] In step S380, the electronic device repeatedly performs steps S350 to S370 until the to-be-recognized gesture disappears. Optionally, before repeatedly performing step S350, the electronic device takes the third time as a new fourth time and takes a time to which the next acquired gesture sensing information belongs as a new third time.

[0424] Steps S350 to S380 are used to complete the follow-up operation after the gesture recognition result is determined, so that the user can achieve more flexible function control through gestures.

[0425] FIG. 17 is an interaction schematic diagram of a follow-up process provided in an example embodiment of the present application. As shown in FIG. 17, the present embodiment at least includes the following steps, and the relevant steps S1310 to S1370 are described above. Optionally, if the to-be-detected gesture continues after step S1370 and the gesture recognition point does not disappear, the electronic device performs step S1380; if the gesture recognition point disappears, the electronic device determines that the gesture recognition process ends. In the present embodiment, the first system service is implemented as a competition and recognition module.

[0426] In step S1380, the gesture sensing module acquires updated gesture sensing information.

[0427] In step S1390, the gesture sensing module sends the updated gesture sensing information to the competition and recognition module. The competition and recognition module acquires the updated gesture sensing information from the gesture sensing module.

[0428] At step S1391, the competition and recognition module obtains the motion parameter of the first gesture according to the updated gesture sensing information. Optionally, after step S1391, the competition and recognition module re-recognizes the gesture according to the updated gesture sensing information (e.g., performs steps S920 to S996). In a case where the newly recognized gesture is the same as the first gesture obtained at step S1350, the competition and recognition module performs step S1392; in a case where the newly recognized gesture is different from the first gesture obtained at step S1350, the competition and recognition module obtains a new first application according to the newly recognized gesture, and performs from step S1360. In this way, the combined gesture can be recognized, and the gesture recognition method can adapt to more and more gestures.

[0429] At step S1392, the competition and recognition module reports the motion parameter of the first gesture to the first application.

[0430] At step S1393, the first application continues to perform the first operation according to the motion parameter of the first gesture.

[0431] The hand-following effect of the first application is controlled through the effect control information, which helps to improve the implementation effect of flexibly adjusting the preset function through the functional gesture.

[0432] The gesture recognition method provided by the embodiments of the present application is described in detail above in combination with FIGS. 3-17. The electronic device provided by the embodiments of the present application is described in detail below in combination with FIG. 18. All or part of any feature of any embodiment in the present application can be freely combined. The combined technical solutions also belong to the scope described in the present application.

[0433] In a possible design, FIG. 18 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. As shown in FIG. 18, the electronic device 1800 can include a transceiver unit 1801 and a processing unit 1802. The electronic device 1800 can be used to implement the functions of the electronic device involved in the above method embodiments.

[0434] Optionally, the processing unit 1802 is configured to support the electronic device 1800 to perform S310 and S340 in FIG. 3; and / or, support the electronic device 1800 to perform S350 to S380; and / or, support the electronic device to perform S910 to S997 in FIG. 9; and / or, support the electronic device 1800 to perform S1310 to S1370 shown in FIG. 13; and / or, support the electronic device 1800 to perform S1310 to S1393 shown in FIG. 17.

[0435] The transceiving unit can include a receiving unit and a sending unit, can be implemented by a transceiver or a transceiver-related circuit component, and can be a transceiver or a transceiver module. The operations and / or functions of each unit in the electronic device 1800 are respectively used to implement the corresponding procedures of the gesture recognition method described in the above method embodiments. All related contents of each step involved in the above method embodiments can be referred to the function description of the corresponding functional unit, and will not be repeated here for brevity.

[0436] Optionally, the electronic device 1800 shown in FIG. 18 can further include a storage unit (not shown in FIG. 18) having a program or instruction stored therein. When the processing unit 1802 executes the program or instruction, the electronic device 1800 shown in FIG. 18 can perform the gesture recognition method described in the above method embodiments.

[0437] The technical effects of the electronic device 1800 shown in FIG. 18 can refer to the technical effects of the gesture recognition method described in the above method embodiments, which will not be repeated here for brevity.

[0438] In addition to the form of the electronic device 1800, the technical solutions provided in the present application can also be a functional unit or a chip in the electronic device, or a device matched with the electronic device.

[0439] The embodiments of the present application also provide a chip system, comprising: a processor coupled with a memory, the memory being used to store a program or instruction, when the program or instruction is executed by the processor, the chip system implements the method in any of the above method embodiments.

[0440] Optionally, the processor in the chip system can be one or more. The processor can be implemented by hardware or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor which implements by reading software codes stored in the memory.

[0441] Optionally, the memory in the chip system can also be one or more. The memory can be integrated with the processor or can be separately arranged from the processor, and the embodiments of the present application are not limited. Exemplarily, the memory can be a non-transient processor, for example, a read-only memory (ROM), which can be integrated on the same chip as the processor or can be separately arranged on different chips. The embodiments of the present application do not make specific limitations on the type of the memory and the arrangement manner of the memory and the processor.

[0442] Exemplarily, the chip system can be a field programmable gate array (FPGA), can be an application specific integrated circuit (ASIC), can also be a system on chip (SoC), can also be a central processor unit (CPU), can also be a network processor (NP), can also be a digital signal processor (DSP), can also be a micro controller unit (MCU), can also be a programmable logic device (PLD) or other integrated chip.

[0443] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software. The method steps disclosed in combination with the embodiments of the present application can be directly embodied as hardware processor execution, or executed by a combination of hardware and software modules in the processor.

[0444] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. When the computer program is executed on a computer, the computer is caused to execute the above related steps to realize the gesture recognition method in the above embodiments.

[0445] The embodiments of the present application also provide a computer program product. When the computer program product is executed on a computer, the computer is caused to execute the above related steps to realize the gesture recognition method in the above embodiments.

[0446] In addition, the embodiments of the present application also provide an apparatus. The apparatus can be specifically a component or a module, and the apparatus can include one or more processors and memories connected thereto. The memories are used to store computer programs. When the computer programs are executed by the one or more processors, the apparatus executes the gesture recognition method in the above method embodiments.

[0447] Among them, the computer readable storage medium, computer program product or chip provided by the embodiments of the present application are used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved are referred to the beneficial effects in the corresponding method provided above, which will not be repeated here.

[0448] The steps of methods or algorithms described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM, flash memory, ROM, erasable programmable ROM, electrically programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can reside in an application-specific integrated circuit (ASIC).

[0449] Those skilled in the art can clearly understand the method provided by the above embodiments, for the convenience and brevity of description, only the above-mentioned functional modules are taken as examples. In actual application, the above-mentioned functions can be completed by different functional modules according to needs; that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0450] In several embodiments provided in the present application, it should be understood that the disclosed method can be implemented in other ways. The above-described device embodiments are only illustrative. For example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, modules or units, which can be electrical, mechanical or other forms.

[0451] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit. The computer readable storage medium includes, but is not limited to, any one of the following: U disk, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk, and various storage medium capable of storing program codes.

[0452] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any change or replacement within the technical scope disclosed by the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A gesture recognition method, characterized by, The method is executed by a first system service in an electronic device, and the method comprises: in response to a user operation for triggering a gesture control, acquiring gesture sensing information; acquiring a first gesture according to the gesture sensing information; reporting information describing the first gesture to a first application, so that the first application executes a first operation corresponding to the first gesture, the first application being an application subscribing to the first gesture.

2. The method of claim 1, wherein, After acquiring the first gesture according to the gesture sensing information, the method further comprises: in response to the user operation, acquiring updated gesture sensing information; acquiring motion parameters of the first gesture according to the updated gesture sensing information; reporting the motion parameters of the first gesture to the first application, so that the first application continues to execute the first operation.

3. The method of claim 2, wherein, The motion parameters of the first gesture are related to a gesture control type to which the first gesture belongs; in the case that the first gesture belongs to a sliding control type, the motion parameters of the first gesture are used to indicate a change in sliding distance, or, in the case that the first gesture belongs to a zooming control type, the motion parameters of the first gesture are used to indicate a change in zooming ratio, or, in the case that the first gesture belongs to a rotating control type, the motion parameters of the first gesture are used to indicate a change in rotating angle.

4. The method according to any one of claims 1 to 3, characterized in that, Before acquiring the gesture sensing information in response to the user operation, the method further comprises: acquiring subscription information of the first application, the subscription information being used to indicate at least one gesture supported by the first application, the at least one gesture including the first gesture.

5. The method of claim 4, wherein, The subscription information includes a type identifier, the type identifier being used to represent a gesture control type to which the first gesture belongs.

6. The method of claim 5, wherein, The information describing the first gesture includes the type identifier.

7. The method of claim 6, wherein, The information describing the first gesture further includes motion attributes of the first gesture and / or a total number of gesture recognition points of the first gesture; wherein the gesture control type of the first gesture is a sliding control type, and the motion attributes of the first gesture include a sliding direction; or the gesture control type of the first gesture is a zooming control type, and the motion attributes of the first gesture include a zooming trend attribute, the zooming trend attribute being opening or pinching; or the gesture control type of the first gesture is a rotating control type, and the motion attributes of the first gesture include a rotating direction attribute.

8. The method according to any one of claims 1 to 7, characterized in that, The acquiring of the first gesture according to the gesture sensing information comprises: acquiring at least one type of gesture factor according to the gesture sensing information, each type of the gesture factor being used to describe a gesture parameter in a corresponding dimension; acquiring the first gesture according to the at least one type of gesture factor.

9. The method of claim 8, wherein, The acquiring of the first gesture according to the at least one type of gesture factor comprises: acquiring a first gesture attribute according to the at least one type of gesture factor, the first gesture attribute being used to represent a first gesture control type to which the first gesture belongs; According to the at least one gesture factor, at least one second gesture attribute is acquired, the second gesture attribute being used to represent an attribute parameter associated with the first gesture control type; According to the first gesture attribute and the at least one second gesture attribute, the first gesture is acquired.

10. The method of claim 9, wherein, The at least one gesture factor includes a plurality of gesture factors with competitive priorities, the competitive priorities being used to represent an order in which the plurality of gesture factors participate in gesture recognition; The acquiring of the first gesture attribute according to the at least one gesture factor includes: A first gesture factor is acquired from the plurality of gesture factors, the first gesture factor being a gesture factor with a highest competitive priority in the plurality of gesture factors; In a case where the first gesture factor meets a corresponding determination condition, the first gesture attribute is determined according to a gesture control type corresponding to the first gesture factor.

11. The method of claim 9, wherein, The at least one gesture factor includes a plurality of gesture factors with competitive priorities, the competitive priorities being used to represent an order in which the plurality of gesture factors participate in gesture recognition; The acquiring of the first gesture attribute according to the at least one gesture factor includes: A first gesture factor is acquired from the plurality of gesture factors, the first gesture factor being a gesture factor with a highest competitive priority in the plurality of gesture factors; In a case where the first gesture factor does not meet a corresponding determination condition, a second gesture factor is acquired from the plurality of gesture factors, the second gesture factor being a gesture factor with a highest competitive priority in gesture factors that do not participate in gesture recognition in the plurality of gesture factors; In a case where the second gesture factor meets a corresponding determination condition, the first gesture attribute is determined according to a gesture control type corresponding to the second gesture factor.

12. The method according to claim 10 or 11, characterized in that, The acquiring of the first gesture attribute according to the at least one gesture factor further includes: A third gesture factor is acquired from the at least one gesture factor, the third gesture factor being used to verify validity of the user operation; In a case where the third gesture factor meets a validity condition, a first gesture attribute used to represent a gesture control type corresponding to a selected gesture factor is acquired, the selected gesture factor being the first gesture factor or the second gesture factor.

13. The method of claim 12, wherein, The third gesture factor includes at least one of: A distance factor used to represent a moving distance of at least one gesture recognition point indicated by the gesture sensing information; A speed factor used to represent a moving speed of the at least one gesture recognition point.

14. The method according to any one of claims 10 to 13, characterized in that, The plurality of gesture factors include a direction factor and a gravity center factor; The competitive priority of the direction factor is higher than the competitive priority of the gravity center factor; The gesture control type corresponding to the direction factor is a sliding control type, and the gesture control type corresponding to the gravity center factor is a rotating control type; or the gesture control type corresponding to the direction factor is a sliding control type, and the gesture control type corresponding to the gravity center factor is a zooming control type; The direction factor is used to represent a moving direction of at least one gesture recognition point indicated by the gesture sensing information, and the barycenter factor is used to represent a distance between the at least one gesture recognition point and a geometric barycenter in a case where a total number of the gesture recognition points is greater than 2, the geometric barycenter being a barycenter of a geometric figure with the at least one gesture recognition point as a vertex.

15. The method according to any one of claims 9 to 14, characterized in that, The first gesture is acquired according to the first gesture attribute and at least one second gesture attribute. n is a positive integer, and the n predicted gestures are acquired according to the corresponding first gesture attribute and at least one second gesture attribute during the user operation; The first gesture is acquired according to the n predicted gestures in a case where the n predicted gestures satisfy an accuracy condition.

16. The method according to any one of claims 9 to 15, characterized in that, The at least one second gesture attribute corresponding to the first gesture control type includes at least one of a total number of gesture recognition points indicated by the gesture sensing information and a motion attribute. The first gesture attribute includes at least one of a sliding control type, a zooming control type and a rotating control type. The motion attribute corresponding to the sliding control type includes a sliding direction attribute, the motion attribute corresponding to the zooming control type includes a zooming trend attribute, and the motion attribute corresponding to the rotating control type includes a rotating direction attribute. The total number of gesture recognition points is at least one of a single finger, a double finger, a triple finger, a quadruple finger and a quintuple finger, the sliding direction attribute is at least one of up, down, left, right, left up, right up, left down and right down, the zooming trend attribute is at least one of opening and pinching, and the rotating direction attribute is at least one of clockwise and counterclockwise.

17. The method of claim 16, wherein, The information for describing the first gesture includes first information and second information, the first information being determined according to the first gesture attribute and the motion attribute, and the second information being the total number of gesture recognition points.

18. The method according to any one of claims 8 to 17, characterized in that, The at least one gesture factor includes at least one of: a number factor used to represent a total number of at least one gesture recognition point indicated by the gesture sensing information; a direction factor used to represent a moving direction of the at least one gesture recognition point; a distance factor used to represent a moving distance of the at least one gesture recognition point; a barycenter factor used to represent a distance between the at least one gesture recognition point and a geometric barycenter in a case where a total number of the gesture recognition points is greater than 2, the geometric barycenter being a barycenter of a geometric figure with the at least one gesture recognition point as a vertex; a speed factor used to represent a moving speed of the at least one gesture recognition point.

19. The method of claim 18, wherein, The gesture sensing information is used to indicate position information of at least one gesture recognition point. The at least one gesture factor is acquired according to the gesture sensing information, including: In a case where the barycenter factor is included in the at least one gesture factor, position information of a geometric barycenter is acquired according to the position information of the at least one gesture recognition point; The barycenter factor corresponding to each gesture recognition point in the at least one gesture recognition point is acquired according to the position information of the at least one gesture recognition point and the position information of the geometric barycenter.

20. The method of claim 18, wherein, The gesture sensing information is used to indicate position information of at least one gesture recognition point; The obtaining of the at least one gesture factor according to the position information of the gesture recognition point comprises: In the case that the at least one gesture factor comprises a relative factor, for each relative factor, a corresponding relative factor of each gesture recognition point in the at least one gesture recognition point is obtained according to the position information of the at least one gesture recognition point and position reference information, the relative factor being at least one of the direction factor, the speed factor and the distance factor.

21. The method according to any one of claims 1 to 20, characterized in that, The first system service is located at a system framework layer in the electronic device.

22. An electronic device, comprising: The electronic device comprises: a display screen for interface display; a transceiver for sending and receiving radio signals; a memory for storing computer program instructions; a processor for executing the computer program instructions to support the electronic device to implement the method according to any one of claims 1 to 21.

23. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and when the computer program runs on a computer, the method according to any one of claims 1 to 21 is implemented.

24. A computer program product comprising instructions, wherein: When the computer program product runs on a computer, the computer is caused to execute the method according to any one of claims 1 to 21.

Images