Gesture recognition method and device

Abstract

The embodiment of the invention discloses a gesture recognition method and device. The gesture recognition method comprises the following steps: capturing a gesture event to obtain parameter information of the gesture event; the method is realized through a desktop application and comprises the following steps: when determining that a gesture event meets a full-screen gesture recognition requirement according to parameter information of the gesture event, performing full-screen gesture recognition according to the parameter information of the gesture event, and performing state switching according to the recognized full-screen gesture. According to the embodiment of the invention, the operation of the user is uniformly planned and designed at the first initiation point-desktop application of the operation of the user, the recognition and state switching of the full-screen gesture are put at the application end, the visual effect design space of each page is larger, more visual effect and interaction scenes which better conform to the operation of the user can be designed, and the user experience is improved. And other visual effect resources are controlled to be coordinated and consistent in full-screen gesture recognition and state switching through the system level, so that disjunction is avoided.

Description

[0001] This application is a divisional application with the application date of "2019.07.03", application number "201910596336.1", and application name "A gesture recognition method and device". Technical Field

[0002] The embodiments of the present invention relate to but are not limited to terminal technology, and in particular to a gesture recognition method and device. Background Art

[0003] The development of user operation buttons in smart terminals has experienced a transition from physical button manipulation to software virtual buttons that make the most of the screen. The latest development trend is to use the entire screen as a space for users to operate, completely abandon the button prompts that occupy space on the screen, and directly recognize the user's gestures to respond to the user's operations. In this way, the software can make full use of the size of the mobile phone screen, display its own interface to the greatest extent, and greatly improve the user experience. There are two main related full-screen gesture recognition solutions. One is directly implemented at the system level, with high processing efficiency, but the interactivity with the desktop is not high and the disconnection is serious; the other is implemented on the application side. Compared with the system-level implementation solution, the application itself is more flexible, can freely implement animations, has stronger interactivity with users, is disconnected from the system, and cannot recognize gestures once the application is in an inactive state. Summary of the invention

[0004] The embodiments of the present invention provide a gesture recognition method and device, which can effectively combine the application end and the system level to realize gesture recognition without being out of touch.

[0005] An embodiment of the present invention provides a gesture recognition method, including:

[0006] Implemented at the system level: Capture gesture events to obtain parameter information of gesture events;

[0007] Implemented through desktop application: When it is determined that the gesture event meets the full-screen gesture recognition requirements based on the parameter information of the gesture event, the full-screen gesture is recognized based on the parameter information of the gesture event, and the state is switched based on the recognized full-screen gesture.

[0008] An embodiment of the present invention provides a gesture recognition device, including:

[0009] The event capture module is used to achieve at the system level: capturing gesture events to obtain parameter information of gesture events;

[0010] A gesture recognition module, used to implement, through a desktop application: when it is determined according to the parameter information of the gesture event that the gesture event meets the full-screen gesture recognition requirement, recognition of the full-screen gesture according to the parameter information of the gesture event;

[0011] The state switching module is used to implement, through a desktop application, state switching according to a recognized full-screen gesture.

[0012] An embodiment of the present invention provides a gesture recognition device, including a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions are executed by the processor, any of the above-mentioned gesture recognition methods is implemented.

[0013] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned gesture recognition methods are implemented.

[0014] The embodiments of the present invention include: realizing through the system level: capturing gesture events to obtain parameter information of gesture events; realizing through desktop applications: when it is determined that the gesture event meets the full-screen gesture recognition requirements based on the parameter information of the gesture event, recognizing the full-screen gesture based on the parameter information of the gesture event, and switching the state based on the recognized full-screen gesture. The embodiments of the present invention make unified planning and design of user operations at the first starting point of user operations—the desktop application, and place the recognition and state switching of full-screen gestures on the application side. The visual effect design space of each page is larger, and more visual effects and interactive scenes that are more in line with user operations can be designed, and interact with the system level, and other visual effect resources are controlled through the system level to coordinate and be consistent in the full-screen gesture recognition and state switching, so as not to be out of touch.

[0015] In another embodiment, identifying a full-screen gesture based on the moving distance and / or moving speed of a moving event relative to a pressing event includes at least one of the following: when the foreground application is a desktop application when the gesture event is captured, identifying the full-screen gesture of the desktop application based on the moving distance and / or moving speed of the moving event relative to the pressing event; when the foreground application is a non-desktop application when the gesture event is captured, identifying the full-screen gesture of the non-desktop application based on the moving distance and / or moving speed of the moving event relative to the pressing event. This solution can identify full-screen gestures of desktop applications and full-screen gestures of non-desktop applications, and will not fail to recognize full-screen gestures because the foreground application is a non-desktop application.

[0016] Other features and advantages of the embodiments of the present invention will be described in the following description, and partly become apparent from the description, or be understood by implementing the embodiments of the present invention. The purposes and other advantages of the embodiments of the present invention can be achieved and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the technical solutions of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the embodiments of the present invention, they are used to explain the technical solutions of the embodiments of the present invention and do not constitute a limitation on the technical solutions of the embodiments of the present invention.

[0018] Figure 1 A flowchart of a gesture recognition method proposed by an embodiment of the present invention;

[0019] Figure 2 This is an exemplary flow chart of a gesture recognition process according to an embodiment of the present invention;

[0020] Figure 3 An exemplary flow chart of a process of performing gesture recognition and state switching through a desktop gesture consumer according to an embodiment of the present invention;

[0021] Figure 4 An exemplary flow chart of a process of performing gesture recognition and state switching through a non-desktop gesture consumer according to an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of the structure of a gesture recognition device according to another embodiment of the present invention;

[0023] Figure 6 FIG. 4 is an exemplary schematic diagram of a gesture recognition device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other at will.

[0025] The steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions. Also, although a logical sequence is shown in the flowchart, in some cases, the steps shown or described can be performed in a sequence different from that shown here.

[0026] With the popularity of smart phones, the global user group of smart terminals will exceed 2 billion. With such a huge number, users' demand for differentiation is also increasing day by day. Under such changes, a variety of personalized differentiated products have emerged one after another, from a wide variety of mobile phone styles, changes in hardware levels of colors, to various customized systems and software, all of which are to meet the differentiated needs of users to the greatest extent. At present, mainstream smart terminals have withdrawn their own full-screen design solutions. The original intention of this solution is to use the entire screen as a space for users to operate, completely abandon the button prompts that occupy the position on the screen, leave more space for applications, and give users a better visual enjoyment. However, there is some room for improvement in the consistency and fluency of interaction in the mainstream design solution.

[0027] Currently, there are two main mainstream full-screen gesture recognition solutions.

[0028] 1. Capture gesture events, recognize gestures, and switch states directly at the system level. In full-screen mode, all states are controlled directly at the system level. The advantage of this solution is that it solves problems at the system level and has a priority advantage in event processing, that is, it handles events at the first time and avoids layer-by-layer transmission, which is the most efficient way. However, the disadvantages of this solution are also obvious. The interactivity with the various applications involved at the system level is very poor. Although the final state switching is implemented by the application side, the specific switching process is still controlled by the system level. The application side can only execute according to the control instructions at the system level. The system level cannot control the change state of the sub-interface in the interface in detail. The visual effect of the state switching process is relatively rough, resulting in some interactions and scene designs that optimize the user experience. It is not easy to implement and the cost of implementation is very high. The interaction between multiple interfaces and applications during the implementation process greatly increases the overhead, gradually turning the advantages of this implementation solution into disadvantages. In a highly competitive market environment, the ultimate goal of a function implementation is to provide a good user experience and a comfortable user usage scenario, so this solution has no competitiveness. In contrast, the second implementation solution is more competitive in the market.

[0029] Second, realize the capture of gesture events, recognition of gestures and state switching on the application side or application terminal. This solution has greater flexibility, and it is also more flexible in the visual effect design and scene design stages. However, the full-screen gesture recognition solution implemented on the application side also has the problem of disconnection. Although the visual effect interaction process is better, there are defects in the system switching, that is, in other applications that are separated from application management, once the application is in an inactive state, gesture events cannot be captured, and gesture recognition cannot be realized, resulting in gesture recognition failure.

[0030] See also Figure 1 An embodiment of the present invention provides a gesture recognition method, comprising:

[0031] Step 100, implemented at the system level: capturing gesture events to obtain parameter information of gesture events.

[0032] In the embodiment of the present invention, a high-level window (i.e., gesture recognition hotspot) is used at the top of the desktop to capture gesture events. This solution can recognize full-screen gestures of desktop applications and full-screen gestures of non-desktop applications, and will not fail to recognize full-screen gestures because the foreground application is a non-desktop application.

[0033] In the embodiment of the present invention, the gesture event may be a touch event or a non-touch event, which is not limited in the embodiment of the present invention.

[0034] In the embodiment of the present invention, parameter information of the gesture event is included in the MotionEvent object.

[0035] In the embodiment of the present invention, the parameter information of the gesture event includes all parameter information of the user operating the touch screen, including the position, the event, the event type, the finger information of the touch, etc. These parameters are important parameters for gesture recognition.

[0036] In one exemplary instance, the position refers to the positions of all touch points of a gesture event.

[0037] In one exemplary instance, the occurrence of an event refers to whether the event occurs:

[0038] In an exemplary instance, the event type refers to whether the event that occurs is a down event, a move event, or an up event.

[0039] In an exemplary embodiment, the finger information on which the touch occurs refers to the number of fingers on which the touch occurs, etc.

[0040] Step 101, implemented through desktop application: when it is determined according to the parameter information of the gesture event that the gesture event meets the full-screen gesture recognition requirement, the full-screen gesture is recognized according to the parameter information of the gesture event, and the state is switched according to the recognized full-screen gesture.

[0041] In the embodiment of the present invention, the full-screen gesture refers to an action originally defined by the system layer, mainly switching to a desktop application (Home), displaying a recently used application (Recent), and returning to a recently used application (Back).

[0042] In an exemplary embodiment, the full-screen gesture recognition requirements include: the finger information of the touch is 1, and the moving direction of the gesture event is a preset direction, such as up, down, left, right, etc.

[0043] In an embodiment of the present invention, identifying a full-screen gesture according to parameter information of a gesture event includes:

[0044] When it is detected that the gesture event needs to be consumed, and the gesture event includes a press event, a move event, and a lift event, determining the moving distance and / or moving speed of the lift event relative to the press event according to parameter information of the lift event;

[0045] The full-screen gesture is identified according to a moving distance and / or a moving speed of the lift event relative to the press event.

[0046] In an exemplary embodiment, detecting a gesture event requires consuming:

[0047] The current state is normal mode or non-editing mode, and the press event occurs in the gesture recognition hot zone.

[0048] In an exemplary embodiment, the gesture recognition hotspot refers to a predefined area, which may be any area on the screen.

[0049] In another embodiment of the present invention, when the foreground application is a non-desktop application when the gesture event is captured, and the gesture event is a press event, the method further includes: switching the current state to a state of entering a recent application.

[0050] In the embodiment of the present invention, identifying a full-screen gesture according to a moving distance and / or moving speed of a lift event relative to a press event includes at least one of the following:

[0051] When the moving distance is greater than a first preset distance threshold, determining that the gesture is a first gesture, such as a pull-up gesture;

[0052] When the moving distance is less than or equal to the first preset distance threshold, and the moving speed is greater than a preset speed threshold, determining that the gesture is a second gesture, such as a quick pull-up gesture;

[0053] When the moving speed is less than or equal to the preset speed threshold, the process is terminated or the relevant parameters of gesture recognition are reinitialized, such as the current state of the mobile terminal, the first preset distance threshold, the second preset distance threshold and the preset speed threshold.

[0054] The state switching according to the recognized gesture includes at least one of the following:

[0055] When the gesture is a first gesture, such as a pull-up gesture, it is determined that the current state needs to be switched to a state of entering a recent application, and an animation effect of switching from the current state to a state of entering a recent application is executed;

[0056] When the gesture is a second gesture, such as a quick pull-up gesture, it is determined that the current state needs to be switched to the desktop, an animation of switching from the current state to the desktop is executed, and the current page of the desktop is moved to a designated homepage.

[0057] In an exemplary embodiment, the first preset distance threshold is related to the display size of the current screen (ie, the number of pixels contained in each dp) and the display direction of the screen.

[0058] In an exemplary embodiment, it is necessary to record the position coordinates Pdown of the pressing event.

[0059] In another embodiment of the present invention, after switching the state according to the recognized full-screen gesture, the method further includes: reinitializing relevant parameters of gesture recognition, such as the current state of the mobile terminal, the first preset distance threshold, the second preset distance threshold and the preset speed threshold.

[0060] In another embodiment of the present invention, the method further includes: synchronizing the current state.

[0061] In another embodiment of the present invention, when the gesture event includes only the press event and the move event, the identifying of the full-screen gesture according to the parameter information of the gesture event further includes:

[0062] When the moving distance of the last captured moving event relative to the pressing event is greater than the first preset distance threshold, determining the sum of the moving distances of the n moving events according to the parameter information of the last captured n moving events;

[0063] When the sum of the moving distances of the n moving events is less than a second preset distance threshold, determining that the gesture is a third gesture, such as a pause gesture;

[0064] The state switching according to the recognized gesture includes:

[0065] When the gesture is a third gesture, such as a pause gesture, it is determined that the current state needs to be switched to a state of entering a recent application, and an animation effect of switching from the current state to a state of entering a recent application is executed.

[0066] In another embodiment of the present invention, when the gesture event includes only the press event and the move event, the identifying of the full-screen gesture according to the parameter information of the gesture event further includes:

[0067] When the movement speed of the last captured movement event relative to the press event is greater than the preset speed threshold, determining that the full-screen gesture is a second gesture;

[0068] The state switching according to the recognized full-screen gesture includes:

[0069] When the full-screen gesture is the second gesture, it is determined that the current state needs to be switched to the desktop, an animation of switching from the current state to the desktop is executed, and the current page of the desktop is moved to a designated homepage.

[0070] In another embodiment of the present invention, after switching the state according to the recognized full-screen gesture, the method further includes: resetting all gesture recognition states.

[0071] In another embodiment of the present invention, before capturing the gesture event to obtain parameter information of the gesture event, the method further includes: initializing the entire system.

[0072] Specifically, initializing the entire system includes:

[0073] When the desktop application starts, get the current navigation bar type from the system;

[0074] When the current navigation bar type is a full-screen gesture type, initialize the following parameters: whether the current desktop is in the foreground (whether it is visible to the user), whether the system's event distribution pipeline is connected (connected in service mode, if not connected, a connection request is required), whether the internal event distribution module pipeline is initialized (if not, it needs to be created and initialization parameters are passed), the current state of the mobile terminal, the first preset distance threshold, the second preset distance threshold and the preset speed threshold.

[0075] The current state of the mobile terminal includes: the screen size of the mobile terminal, the current horizontal or vertical screen state of the mobile terminal, and the gesture recognition hot zone parameters of the mobile terminal.

[0076] The current horizontal or vertical screen state of the mobile terminal includes: horizontal screen state or vertical screen state;

[0077] The gesture recognition hotspot of a mobile terminal refers to a rectangular space defined in association with the size of the mobile terminal and the horizontal or vertical state of the screen.

[0078] In another embodiment of the present invention, the method further comprises:

[0079] Register a monitoring object, and monitor whether the navigation bar type switches through the monitoring object. When it is monitored that the navigation bar type switches, re-execute the step of obtaining the current navigation bar type from the system.

[0080] The embodiment of the present invention uniformly plans and designs user operations at the first starting point of user operations - desktop applications, and places the recognition and state switching of full-screen gestures on the application side. The visual effect design space of each page is larger, and more visual effects and interactive scenes that are more in line with user operations can be designed. It can also interact with the system level, and other visual effect resources can be controlled at the system level to be coordinated and consistent in full-screen gesture recognition and state switching to avoid disconnection.

[0081] In one illustrative example, Figure 2 As shown, the gesture recognition process includes:

[0082] Step 200: Capture a gesture event at the system level and generate a MotionEvent object, which includes parameter information of the gesture event.

[0083] In this step, the parameter information of the gesture event includes all parameter information of the user operating the touch screen, including the position, the event, the event type, the finger information of the touch, etc. These parameters are important parameters for gesture recognition.

[0084] In one exemplary instance, the position refers to the positions of all touch points of a gesture event.

[0085] In one exemplary instance, the occurrence of an event refers to whether the event occurs:

[0086] In an exemplary instance, the event type refers to whether the event that occurs is a down event, a move event, or an up event.

[0087] In an exemplary embodiment, the finger information on which the touch occurs refers to the number of fingers on which the touch occurs, etc.

[0088] Step 201: When the full-screen gesture recognition requirements are met, a gesture event consumer is selected according to the current operation scenario, and the Mot ionEvent object is forwarded to the selected gesture event consumer. Specifically, when the current operation scenario is that the foreground application is a desktop application, a desktop gesture consumer is selected, the Mot ionEvent object is forwarded to the desktop gesture consumer, and steps 202 and 204 are continued; when the current operation scenario is that the foreground application is a non-desktop application, a non-desktop gesture consumer is selected, the Mot ionEvent object is forwarded to the non-desktop gesture consumer, and steps 203 and 204 are continued.

[0089] In an exemplary embodiment, the operation scenario includes at least one of the following:

[0090] The desktop application recognizes full-screen gestures when it is in the foreground, that is, recognizes full-screen gestures on the desktop;

[0091] A non-desktop application recognizes full-screen gestures when it is in the foreground, that is, the user has opened the non-desktop application and the non-desktop application is in an activated state.

[0092] Step 202: The desktop gesture consumer recognizes and switches the state of the full-screen gesture.

[0093] Step 203: Recognize and switch the state of the full-screen gesture through the non-desktop gesture consumer.

[0094] Step 204: Perform state synchronization. Specifically, synchronize the switched state to the system and other applications.

[0095] In one illustrative example, Figure 3 As shown, the process of identifying and switching the full-screen gestures through the desktop gesture consumer includes:

[0096] Step 300: Monitor whether gesture events need to be consumed. Monitoring rules: the current state can execute full-screen gestures, that is, the desktop is in normal mode (or non-editing mode); whether the press event occurs in the gesture recognition hot zone.

[0097] In this step, the gesture recognition hot zone refers to a predefined area, which can be any area on the screen.

[0098] Step 301, when a gesture event that needs to be consumed is monitored, it is necessary to first identify the press (EVENT_DOWN) event, move event and lift event in the gesture event, record the position coordinates Pdown of the EVENT_DOWN event, and initialize the first preset distance threshold, which is defined here as Ts l ide.

[0099] In this step, the first preset distance threshold is related to the current display size (ie, the number of pixels contained in each dp) and the current display direction of the screen.

[0100] Step 302: When the gesture event includes a press event, a move event, and a lift event, determine whether the moving distance of the lift event relative to the press event is greater than a first preset distance threshold. When the moving distance is greater than the first preset distance threshold, determine that the current state needs to be switched to the state Srecent of the most recent application, and execute the animation from the current state to Srecent; when the moving distance is less than or equal to the first preset distance threshold, continue to execute step 303.

[0101] Step 303: Determine whether the moving speed of the lift event relative to the press event is greater than a preset speed threshold. When the moving speed is greater than the preset speed threshold, determine that the current state needs to be switched to the desktop Shome, execute the animation effect of switching from the current state to Shome, and move the current page of the desktop to the homepage specified by the user; when the moving speed is less than or equal to the preset speed threshold, execute step 306.

[0102] Step 304 : When the gesture event only includes the press event and the move event, the user's action is still in progress, that is, the EVENT_UP event is not captured, that is, the user does not raise his hand at this time, and the subsequent events need to be recognized again. The specific recognition steps include step 305 .

[0103] Step 305: When the moving distance of the last moving event relative to the pressing event is greater than the first preset distance threshold, the above judgment condition cannot be used to handle this. At this time, it is necessary to retrieve the parameter information of the n most recent gesture events from the parameter information of the previously cached gesture events, and monitor whether the sum of the moving distances Dad of the n gesture events is less than the second preset distance threshold. When the sum of the moving distances Dad of the n gesture events is less than the second preset distance threshold, it is determined that the current state needs to be switched to the state Srecent of entering the most recent application, and an animation of switching from the current state to the state of entering the most recent application is executed, and all gesture recognition states are reset. There is no need to continue waiting for the EVENT_UP event. When the sum of the moving distances of the n gesture events is greater than or equal to the second preset distance threshold, return to step 300 to continue execution.

[0104] When the moving distance of the last gesture event relative to the press event is less than or equal to the first preset distance threshold, or the moving speed of the last movement event relative to the press event is less than or equal to the preset speed threshold, return to step 300 to continue monitoring the movement (EVENT_MOVE) or EVENT_UP event.

[0105] When the movement speed of the last move event relative to the press event is greater than the preset speed threshold, the mark bit is recorded and the event is consumed; when the mark bit is T, it is determined that the current state needs to be switched to the desktop Shome, and the animation effect of switching from the current state to Shome is executed, and the current page of the desktop is moved to the homepage specified by the user.

[0106] Step 306: reset all gesture recognition states and wait for the next recognition.

[0107] In one illustrative example, Figure 4 As shown, the process of identifying and switching the full-screen gesture through the non-desktop gesture consumer includes:

[0108] Step 400: Monitor whether gesture events need to be consumed. Monitoring rules: the current state can execute full-screen gestures, that is, the desktop is in normal mode (or non-editing mode); whether the press event occurs in the gesture recognition hot zone; the foreground application is a non-desktop application.

[0109] In this step, the gesture recognition hot zone refers to a predefined area, which can be any area on the screen.

[0110] Step 401, when a gesture event that needs to be consumed is monitored, it is necessary to first identify the press (EVENT_DOWN) event, move event and lift event in the gesture event, record the position coordinates Pdown of the EVENT_DOWN event, and initialize the first preset distance threshold, which is defined here as Ts l ide.

[0111] In this step, the first preset distance threshold is related to the current display size (ie, the number of pixels contained in each dp) and the current display direction of the screen.

[0112] Step 402: When a press event is received, the current state is switched to the state Srecent of the most recent application.

[0113] Step 403: When the gesture event includes a press event, a move event, and a lift event, determine whether the moving distance of the lift event relative to the press event is greater than a first preset distance threshold. When the moving distance is greater than the first preset distance threshold, determine that the current state needs to be switched to the state Srecent of the most recent application, and execute the animation from the current state to Srecent; when the moving distance is less than or equal to the first preset distance threshold, continue to execute step 404.

[0114] Step 404: determine whether the moving speed of the lift event relative to the press event is greater than a preset speed threshold. When the moving speed is greater than the preset speed threshold, determine that the current state needs to be switched to the desktop Shome, execute the animation effect of switching from the current state to Shome, and move the current page of the desktop to the homepage specified by the user; when the moving speed is less than or equal to the preset speed threshold, execute step 407.

[0115] Step 405: When the gesture event only includes the press event and the move event, the user's action is still in progress, that is, the EVENT_UP event is not captured, that is, the user does not raise his hand at this time, and the subsequent events need to be recognized again. The specific recognition steps include step 406.

[0116] Step 406: When the moving distance of the last moving event relative to the pressing event is greater than the first preset distance threshold, the above judgment condition cannot be used to handle this. At this time, it is necessary to retrieve the parameter information of the n most recent gesture events from the parameter information of the gesture events cached previously, and monitor whether the sum of the moving distances Dad of the n gesture events is less than the second preset distance threshold. When the sum of the moving distances Dad of the n gesture events is less than the second preset distance threshold, it is determined that the current state needs to be switched to the state Srecent of entering the most recent application, and an animation is executed to switch from the current state to the state of entering the most recent application. All gesture recognition states are reset, and there is no need to continue waiting for the EVENT_UP event. When the sum of the moving distances of the n gesture events is greater than or equal to the second preset distance threshold, return to step 400 to continue execution.

[0117] When the moving distance of the last gesture event relative to the press event is less than or equal to the first preset distance threshold, or the moving speed of the last move event relative to the press event is less than or equal to the preset speed threshold, return to step 400 to continue monitoring the move (EVENT_MOVE) or EVENT_UP event.

[0118] When the movement speed of the last move event relative to the press event is greater than the preset speed threshold, the mark bit is recorded and the event is consumed; when the mark bit is T, it is determined that the current state needs to be switched to the desktop Shome, and the animation effect of switching from the current state to Shome is executed, and the current page of the desktop is moved to the homepage specified by the user.

[0119] Step 407: reset all gesture recognition states and wait for the next recognition.

[0120] Another embodiment of the present invention provides a gesture recognition device, including a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions are executed by the processor, any of the above gesture recognition methods is implemented.

[0121] Another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of any of the above-mentioned gesture recognition methods are implemented.

[0122] See also Figure 5 Another embodiment of the present invention provides a gesture recognition device, comprising:

[0123] The event capture module 501 is used to achieve, at the system level: capturing gesture events to obtain parameter information of gesture events;

[0124] The gesture recognition module 502 is used to implement, through a desktop application: when it is determined according to the parameter information of the gesture event that the gesture event meets the full-screen gesture recognition requirement, recognize the full-screen gesture according to the parameter information of the gesture event;

[0125] The state switching module 503 is used to implement, through a desktop application, state switching according to a recognized full-screen gesture.

[0126] In the embodiment of the present invention, a high-level window (i.e., gesture recognition hotspot) is used at the top of the desktop to capture gesture events. This solution can recognize full-screen gestures of desktop applications and full-screen gestures of non-desktop applications, and will not fail to recognize full-screen gestures because the foreground application is a non-desktop application.

[0127] In the embodiment of the present invention, the gesture event may be a touch event or a non-touch event, which is not limited in the embodiment of the present invention.

[0128] In the embodiment of the present invention, parameter information of the gesture event is included in the MotionEvent object.

[0129] In the embodiment of the present invention, the parameter information of the gesture event includes all parameter information of the user operating the touch screen, including the position, the event, the event type, the finger information of the touch, etc. These parameters are important parameters for gesture recognition.

[0130] In one exemplary instance, the position refers to the positions of all touch points of a gesture event.

[0131] In one exemplary instance, the occurrence of an event refers to whether the event occurs:

[0132] In an exemplary instance, the event type refers to whether the event that occurs is a down event, a move event, or an up event.

[0133] In an exemplary embodiment, the finger information on which the touch occurs refers to the number of fingers on which the touch occurs, etc.

[0134] In the embodiment of the present invention, the full-screen gesture refers to an action originally defined by the system layer, mainly switching to a desktop application (Home), displaying a recently used application (Recent), and returning to a recently used application (Back).

[0135] In an exemplary embodiment, the full-screen gesture recognition requirements include: the finger information of the touch is 1, and the moving direction of the gesture event is a preset direction, such as up, down, left, right, etc.

[0136] In the embodiment of the present invention, the gesture recognition module 502 is specifically used to realize the recognition of the full-screen gesture according to the parameter information of the gesture event in the following manner:

[0137] When it is detected that the gesture event needs to be consumed, and the gesture event includes a press event, a move event, and a lift event, determining the moving distance and / or moving speed of the lift event relative to the press event according to parameter information of the lift event;

[0138] The full-screen gesture is identified according to a moving distance and / or a moving speed of the lift event relative to the press event.

[0139] In an exemplary embodiment, the gesture recognition module 502 is specifically configured to implement the consumption required to detect a gesture event in the following manner:

[0140] The current state is normal mode or non-editing mode, and the press event occurs in the gesture recognition hot zone.

[0141] In an exemplary embodiment, the gesture recognition hotspot refers to a predefined area, which may be any area on the screen.

[0142] In another embodiment of the present invention, the state switching module 503 is further used for: when the foreground application is a non-desktop application when the gesture event is captured, and when the gesture event is a press event, switching the current state to a state of entering a recent application.

[0143] In the embodiment of the present invention, the gesture recognition module 502 is specifically configured to recognize a full-screen gesture according to a moving distance and / or moving speed of a lift event relative to a press event in at least one of the following ways:

[0144] When the moving distance is greater than a first preset distance threshold, determining that the gesture is a first gesture, such as a pull-up gesture;

[0145] When the moving distance is less than or equal to the first preset distance threshold, and the moving speed is greater than a preset speed threshold, determining that the gesture is a second gesture, such as a quick pull-up gesture;

[0146] When the moving speed is less than or equal to the preset speed threshold, the process is terminated or the relevant parameters of gesture recognition are reinitialized, such as the current state of the mobile terminal, the first preset distance threshold, the second preset distance threshold and the preset speed threshold.

[0147] The state switching module 503 is specifically configured to implement the state switching according to the recognized full-screen gesture by adopting at least one of the following methods:

[0148] When the gesture is a first gesture, such as a pull-up gesture, it is determined that the current state needs to be switched to a state of entering a recent application, and an animation effect of switching from the current state to a state of entering a recent application is executed;

[0149] When the gesture is a second gesture, such as a quick pull-up gesture, it is determined that the current state needs to be switched to the desktop, an animation of switching from the current state to the desktop is executed, and the current page of the desktop is moved to a designated homepage.

[0150] In an exemplary embodiment, the first preset distance threshold is related to the display size of the current screen (ie, the number of pixels contained in each dp) and the display direction of the screen.

[0151] In an exemplary embodiment, it is necessary to record the position coordinates Pdown of the pressing event.

[0152] In another embodiment of the present invention, the initialization module 504 is also used to: after state switching according to the recognized full-screen gesture, reinitialize relevant parameters of gesture recognition, such as the current state of the mobile terminal, the first preset distance threshold, the second preset distance threshold and the preset speed threshold.

[0153] In another embodiment of the present invention, the state switching module 503 is further used to: synchronize the current state.

[0154] In another embodiment of the present invention, the gesture recognition module 502 is further used for:

[0155] When the gesture event includes only the press event and the move event, when the moving distance of the last captured move event relative to the press event is greater than the first preset distance threshold, determining the sum of the moving distances of the n move events according to the parameter information of the last captured n move events;

[0156] When the sum of the moving distances of the n moving events is less than a second preset distance threshold, determining that the gesture is a third gesture, such as a pause gesture;

[0157] The state switching module 503 is also used for:

[0158] When the gesture is a third gesture, such as a pause gesture, it is determined that the current state needs to be switched to a state of entering a recent application, and an animation effect of switching from the current state to a state of entering a recent application is executed.

[0159] In another embodiment of the present invention, the gesture recognition module 502 is further used for:

[0160] When the gesture event includes only the press event and the move event, when the movement speed of the last captured move event relative to the press event is greater than the preset speed threshold, determining that the full-screen gesture is the second gesture;

[0161] The state switching according to the recognized full-screen gesture includes:

[0162] When the full-screen gesture is the second gesture, it is determined that the current state needs to be switched to the desktop, an animation of switching from the current state to the desktop is executed, and the current page of the desktop is moved to a designated homepage.

[0163] In another embodiment of the present invention, it further includes: an initialization module 504, which is used to initialize the entire system.

[0164] Specifically, the initialization module 504 is specifically used for:

[0165] When the desktop application starts, get the current navigation bar type from the system;

[0166] When the current navigation bar type is a full-screen gesture type, initialize the following parameters: whether the current desktop is in the foreground (whether it is visible to the user), whether the system's event distribution pipeline is connected (connected in service mode, if not connected, a connection request is required), whether the internal event distribution module pipeline is initialized (if not, it needs to be created and initialization parameters are passed), the current state of the mobile terminal, the first preset distance threshold, the second preset distance threshold and the preset speed threshold.

[0167] The current state of the mobile terminal includes: the screen size of the mobile terminal, the current horizontal or vertical screen state of the mobile terminal, and the gesture recognition hot zone parameters of the mobile terminal.

[0168] The current horizontal or vertical screen state of the mobile terminal includes: horizontal screen state or vertical screen state;

[0169] The gesture recognition hotspot of a mobile terminal refers to a rectangular space defined in association with the size of the mobile terminal and the horizontal or vertical state of the screen.

[0170] In another embodiment of the present invention, the initialization module 504 is further used to:

[0171] Register a monitoring object, and monitor whether the navigation bar type switches through the monitoring object. When it is monitored that the navigation bar type switches, re-execute the step of obtaining the current navigation bar type from the system.

[0172] The embodiment of the present invention uniformly plans and designs user operations at the first starting point of user operations - desktop applications, and places the recognition and state switching of full-screen gestures on the application side. The visual effect design space of each page is larger, and more visual effects and interactive scenes that are more in line with user operations can be designed. It can also interact with the system level, and other visual effect resources can be controlled at the system level to be coordinated and consistent in full-screen gesture recognition and state switching to avoid disconnection.

[0173] In one illustrative example, Figure 6 As shown, the gesture recognition device further includes an event distribution module 505, which is used to identify the current operation scenario and distribute the parameter information of the gesture event to the gesture recognition submodule 601 of the desktop application or the gesture recognition submodule 602 of the non-desktop application according to the current operation scenario;

[0174] Specifically, when the current operation scenario is that a full-screen gesture is recognized when a desktop application is in the foreground, the parameter information of the gesture event is distributed to the gesture recognition submodule 601 of the desktop application; when the current operation scenario is that a full-screen gesture is recognized when a non-desktop application is in the foreground, the parameter information of the gesture event is distributed to the gesture recognition submodule 602 of the non-desktop application;

[0175] The gesture recognition module 502 includes a gesture recognition submodule 601 for desktop applications and a gesture recognition submodule 602 for non-desktop applications;

[0176] The desktop application gesture recognition submodule 601 is used to recognize full-screen gestures for gesture events captured when the desktop application is in the foreground;

[0177] The gesture recognition submodule 602 of the non-desktop application is used to recognize the full-screen gestures for the gesture events captured when the non-desktop application is in the foreground;

[0178] The state switching module 503 includes a state control submodule 603 and a motion effect control submodule 604;

[0179] The state control submodule 603 is used to switch states and synchronize states according to the gesture recognition results of the gesture recognition submodule 601 of the desktop application or the gesture recognition submodule 602 of the non-desktop application;

[0180] The motion effect control submodule 604 is used to control the motion effect during the state switching process.

[0181] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0182] Although the implementation methods disclosed in the embodiments of the present invention are as above, the contents described are only implementation methods adopted to facilitate understanding of the embodiments of the present invention, and are not intended to limit the embodiments of the present invention. Any technician in the field to which the embodiments of the present invention belong may make any modifications and changes in the form and details of implementation without departing from the spirit and scope disclosed in the embodiments of the present invention, but the scope of patent protection of the embodiments of the present invention shall still be subject to the scope defined in the attached claims.

Claims

1. A gesture recognition method, comprising: Capturing gesture events through a gesture recognition hotspot at the top of the desktop to obtain parameter information of the gesture event, wherein the gesture recognition hotspot is used to recognize full-screen gestures of desktop applications and full-screen gestures of non-desktop applications; When it is determined according to the parameter information of the gesture event that the gesture event meets the full-screen gesture recognition requirement, if the current state is in the normal mode or the non-editing mode, the gesture event includes a press event, a move event, and a lift event, and the press event occurs in the gesture recognition hot zone, the moving distance of the lift event relative to the press event is determined according to the parameter information of the lift event; When the moving distance is greater than a first preset distance threshold, the full-screen gesture is determined to be a first gesture, and based on the first gesture, it is determined that the current state needs to be switched to a state of entering a recent application, and an animation effect of switching from the current state to a state of entering a recent application is executed.

2. The method according to claim 1, characterized in that When determining the moving distance of the lift event relative to the press event according to the parameter information of the lift event, the moving speed of the lift event relative to the press event is also determined; The method further comprises: When the moving distance is less than or equal to the first preset distance threshold, and the moving speed is greater than the preset speed threshold, the full-screen gesture is determined to be a second gesture, and according to the second gesture, it is determined that the current state needs to be switched to the desktop, and an animation effect of switching from the current state to the desktop is executed.

3. The method according to claim 1, characterized in that Before capturing gesture events through the gesture recognition hotspot at the top layer of the desktop to obtain parameter information of the gesture events, the method further includes: Initialize the entire system.

4. The method according to claim 3, characterized in that The initialization setting of the entire system includes: When the desktop application starts, get the current navigation bar type from the system; When the current navigation bar type is a full-screen gesture type, the following parameters are initialized: whether the current desktop is in the foreground, whether the system event distribution pipeline is connected, whether the module pipeline of the internal event distribution is initialized, the current state of the mobile terminal, the first preset distance threshold, the second preset distance threshold and the preset speed threshold; The current state of the mobile terminal includes: the screen size of the mobile terminal, the current horizontal or vertical screen state of the mobile terminal, and the gesture recognition hot zone parameters of the mobile terminal.

5. The method according to claim 4, characterized in that The method further comprises: Register a monitoring object, and monitor whether the navigation bar type switches through the monitoring object. When it is monitored that the navigation bar type switches, re-execute the step of obtaining the current navigation bar type from the system.

6. The method according to claim 1, characterized in that When the gesture event is captured, the foreground application is a non-desktop application, and the gesture event is a press event, and the method further includes: The current state is switched to a state of entering a recent application.

7. The method according to claim 1, characterized in that When the gesture event includes only a press event and a move event, the method further includes: When the moving distance of the last captured moving event relative to the pressing event is greater than the first preset distance threshold, determining the sum of the moving distances of the n moving events according to the parameter information of the n last captured moving events; When the sum of the moving distances of the n moving events is less than a second preset distance threshold, determining that the full-screen gesture is a third gesture; It is determined according to the third gesture that the current state needs to be switched to a state of entering a recent application, and an animation effect of switching from the current state to a state of entering a recent application is executed.

8. The method according to claim 1, characterized in that When the gesture event includes only a press event and a move event, the method further includes: When the movement speed of the last captured movement event relative to the press event is greater than a preset speed threshold, determining that the full-screen gesture is a second gesture; According to the second gesture, it is determined that the current state needs to be switched to the desktop, and an animation effect of switching from the current state to the desktop is executed.

9. A gesture recognition method, comprising: Capturing gesture events through a gesture recognition hotspot at the top of the desktop to obtain parameter information of the gesture event, wherein the gesture recognition hotspot is used to recognize full-screen gestures of desktop applications and full-screen gestures of non-desktop applications; When it is determined according to the parameter information of the gesture event that the gesture event meets the full-screen gesture recognition requirement, if the current state is in the normal mode or the non-editing mode, the gesture event includes a press event, a move event, and a lift event, and the press event occurs in the gesture recognition hot zone, the moving speed of the lift event relative to the press event is determined according to the parameter information of the lift event; When the moving distance is less than or equal to a first preset distance threshold, and the moving speed is greater than a preset speed threshold, the full-screen gesture is determined to be a second gesture, and according to the second gesture, it is determined that the current state needs to be switched to the desktop, and an animation of switching from the current state to the desktop is executed.

10. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, the gesture recognition method according to any one of claims 1 to 9 is implemented.

11. A computer-readable storage medium storing computer-executable instructions, characterized in that: The computer executable instructions are used to execute the gesture recognition method as described in any one of claims 1 to 9.

12. A computer program product, comprising a computer program or computer instructions, characterized in that The computer program or the computer instruction is stored in a computer-readable storage medium, and a processor of an electronic device reads the computer program or the computer instruction from the computer-readable storage medium, and the processor executes the computer program or the computer instruction, so that the electronic device performs the gesture recognition method as described in any one of claims 1 to 9.

Images