Gesture Posture Synchronization Detection and Control Method, System, Medium and Mobile Terminal

Abstract

The present invention provides a gesture and posture synchronization detection and control method, system, medium, and mobile terminal, including: a motion information acquisition step: acquiring the motion information of the mobile terminal; a posture analysis step: analyzing to obtain posture information according to the motion information; a gesture analysis step: analyzing to obtain gesture information according to the motion information; a control analysis step: obtaining control information according to the gesture information and the posture information. The present invention organically combines gesture information and posture information, and obtains a control instruction according to the combination relationship between the two, greatly expanding the number of instructions and adding more fun to the use of toys.

Description

Technical Field

[0001] The present invention relates to the field of gesture control, and specifically, to a gesture posture synchronization detection control method, system, medium, and mobile terminal. Background Art

[0002] Patent document CN104941203A discloses a toy based on gesture trajectory recognition and its recognition and control method. By setting a gesture trajectory recognition mechanism, the gesture trajectory of a user is recognized, and the corresponding functions of the toy are controlled according to different gesture trajectories of the user. Without using a remote control or contacting the toy, only gesture actions are required for operation, which is simple and convenient, and can recognize the swiping trajectories of the user's gestures, such as forward, backward, left, right, circular gestures, etc. The user can achieve multiple different function controls with a single gesture. However, patent document CN104941203A can only achieve separate gesture control.

[0003] Patent document CN106178538A provides an intelligent toy control system and method based on posture detection, which relates to the field of toys. It is characterized in that the system includes: a control end for wearing on a human body to detect human motion control commands and a toy end for controlling the operation of the toy according to the control commands sent by the control end and for controlling the operation of the toy according to the recognition results of a voice recognition device. However, patent document CN106178538A can only achieve separate posture control. Summary of the Invention

[0004] Aiming at the defects in the prior art, the purpose of the present invention is to provide a gesture posture synchronization detection control method, system, medium, and mobile terminal.

[0005] According to a gesture posture synchronization detection control method provided by the present invention, it includes:

[0006] A motion information acquisition step: acquiring the motion information of the mobile terminal;

[0007] A posture analysis step: analyzing to obtain posture information according to the motion information;

[0008] A gesture analysis step: analyzing to obtain gesture information according to the motion information;

[0009] A control analysis step: obtaining control information according to the gesture information and the posture information.

[0010] Preferably, the motion information includes acceleration sensing information, wherein the acceleration sensing information indicates the acceleration of the mobile terminal;

[0011] In the posture analysis step, the acceleration components related to the posture are extracted from the acceleration sensing information and denoted as posture acceleration components; the posture acceleration components are matched with the posture preset information to obtain the posture information.

[0012] In the gesture analysis step, the interference acceleration components of the posture acceleration components are extracted from the acceleration sensing information and denoted as gesture acceleration components; the gesture acceleration components are matched with the gesture preset information to obtain the gesture information.

[0013] Preferably, in the posture analysis step, after the posture acceleration components are matched with the posture preset information to obtain the posture information, the correction information corresponding to the gesture information is found, and the posture information is corrected by using the correction information.

[0014] Wherein, the correction information is used to correct the acceleration deviation caused by the action deviation caused by the limitation of the human joint mobility.

[0015] Preferably, in the control analysis step, according to the gesture information and the posture information, the current combination relationship of the gesture and the posture is obtained, and according to the current combination relationship, the corresponding control instruction is obtained, wherein the control instruction instructs the actuator to execute the preset action.

[0016] The current combination relationship includes a position relationship, wherein the position relationship is used to indicate the position of the mobile terminal in the gesture action when the gesture posture is generated.

[0017] A gesture and posture synchronous detection control system according to the present invention includes:

[0018] A motion information acquisition module: acquiring the motion information of the mobile terminal;

[0019] A posture analysis module: analyzing and obtaining the posture information according to the motion information;

[0020] A gesture analysis module: analyzing and obtaining the gesture information according to the motion information;

[0021] A control analysis module: obtaining the control information according to the gesture information and the posture information.

[0022] Preferably, the motion information includes acceleration sensing information, wherein the acceleration sensing information indicates the acceleration of the mobile terminal.

[0023] In the posture analysis module, the acceleration components related to the posture are extracted from the acceleration sensing information and denoted as posture acceleration components; the posture acceleration components are matched with the posture preset information to obtain the posture information.

[0024] In the gesture analysis module, an interference acceleration component of the attitude acceleration component is extracted from the acceleration sensing information and denoted as the gesture acceleration component; the gesture acceleration component is matched with the gesture preset information to obtain gesture information.

[0025] Preferably, in the attitude analysis module, after matching the attitude acceleration component with the attitude preset information to obtain attitude information, correction information corresponding to the gesture information is found, and the attitude information is corrected by using the correction information;

[0026] Wherein, the correction information is used to correct the acceleration deviation caused by the action deviation caused by the limitation of the human joint mobility.

[0027] Preferably, in the control analysis module, according to the gesture information and the attitude information, the current combination relationship of the gesture and the attitude is obtained, and according to the current combination relationship, a corresponding control instruction is obtained, wherein the control instruction instructs the actuator to execute a preset action;

[0028] The current combination relationship includes a position relationship, wherein the position relationship is used to indicate the position of the mobile terminal in the gesture action when the gesture posture is generated.

[0029] According to a computer-readable storage medium storing a computer program provided by the present invention, when the computer program is executed by a processor, the steps of the method are implemented.

[0030] According to an intelligent terminal provided by the present invention, the intelligent terminal includes the computer-readable storage medium storing the computer program, or includes the gesture and attitude synchronization detection control system.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] The present invention organically combines gesture information and attitude information, and obtains a control instruction according to the combination relationship between the two, greatly expanding the number of instructions and adding more fun to the use of the toy. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objects and advantages of the present invention will become more apparent:

[0034] Figure 1 It is a flowchart of the steps of the gesture and attitude synchronization detection control method provided by the present invention. DETAILED DESCRIPTION

[0035] The present invention will be described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that those of ordinary skill in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0036] A gesture and posture synchronization detection and control method provided by the present invention includes:

[0037] A motion information acquisition step: acquiring the motion information of a mobile terminal; the mobile terminal can be a handheld terminal such as a smart phone, a smart watch, a tablet computer, smart glasses, a smart bracelet, a smart toy, a smart remote control, etc. When a user holds the handheld terminal, the posture of the mobile terminal can be changed by actions such as tilting. At the same time, when the user holds the handheld terminal, gestures can be made by driving the mobile terminal to follow the hand through actions such as rotating the wrist and waving the arm. When the mobile terminal changes its posture or makes a gesture, motion information will be generated. Or when, for example, the posture remains unchanged, it is also considered to represent a static motion state of the mobile device relative to the user's hand, and this is also used as motion information. In this way, just a simple change or not can combine the gesture and the posture to form four types of motion information: both change, both do not change, one changes and the other does not change, and one does not change and the other changes. Correspondingly, the four types of motion information can allow the user to issue four control instructions correspondingly. In a preferred example, the mobile terminal is a smart wearable device, for example, a smart wearable device worn on the forearm. Then, there is a corresponding relationship between the posture of the smart wearable device and the posture of the wearer's forearm. Therefore, according to the posture of the smart wearable device itself, the posture of the wearer's body part can also be obtained as the posture information for subsequent analysis and control.

[0038] A posture analysis step: analyzing and obtaining posture information according to the motion information; the posture information reflects the actions of the user with a relatively small amplitude, usually the actions of the wrist, such as tilting or shaking.

[0039] A gesture analysis step: analyzing and obtaining gesture information according to the motion information; the gesture information reflects the actions of the user with a relatively large amplitude. Such actions usually require the cooperation of the elbow joint and the shoulder joint to make. In particular, regarding the mobile terminal as a mass point, the motion trajectory of this mass point is used as the gesture.

[0040] Control analysis step: Obtain control information based on the gesture information and posture information. In the present invention, it is not that one control information is obtained from the gesture information and another control information is obtained from the posture information. Instead, based on the combination of gesture information and posture information at the same moment or within the same time period, control information is obtained. Herein, the same time period may be the time period between the start time and the end time when a complete gesture is made. For example, when the gesture information meets the requirements, that is, the gesture is considered standard, while the posture information does not meet the requirements, that is, the posture is considered not to be standard, then a combination of gesture information and posture information that meets the requirements cannot be obtained, and the control information desired by the user cannot be generated.

[0041] The present invention will be described in more detail below.

[0042] The motion information includes acceleration sensing information, wherein the acceleration sensing information indicates the acceleration of the mobile terminal; the acceleration sensing information is acquired through an acceleration sensor in the mobile terminal. In the posture analysis step, the acceleration component related to the posture is extracted from the acceleration sensing information and denoted as the posture acceleration component; the posture acceleration component is matched with the posture preset information to obtain the posture information. In the gesture analysis step, the interference acceleration component of the posture acceleration component is extracted from the acceleration sensing information and denoted as the gesture acceleration component; the gesture acceleration component is matched with the gesture preset information to obtain the gesture information. More specifically, there may be one or several acceleration sensors. For a mobile terminal with only one acceleration sensor, Patent Document CN107014386B discloses a solution capable of separating the acceleration related to the posture of the aircraft from the acceleration unrelated to the posture. Those skilled in the art can at least refer to Patent Document CN107014386B to separately extract the acceleration components corresponding to the posture information and gesture information. For a mobile terminal with multiple, for example, two, acceleration sensors, the acceleration information corresponding to the posture information and gesture information can be obtained through the two acceleration sensors.

[0043] Further, in the posture analysis step, after matching the posture acceleration components with the preset posture information to obtain the posture information, the correction information corresponding to the gesture information is found, and the posture information is corrected using the correction information; wherein, the correction information is used to correct the acceleration deviation caused by the action deviation due to the limitation of the human joint mobility. In a preferred example, the preset posture information is a set of acceleration component templates, and the posture information preset corresponding to the template in the acceleration component template set that best matches the posture acceleration component is used as the posture information obtained after the posture acceleration component is matched. However, due to the limitation of the human joint mobility, especially the limitation of the wrist joint and elbow joint of the hand, it is difficult to make the expected actions in all directions, which is also related to the flexibility of the user's arm, or it is difficult to exert force at some angles, etc. Therefore, the present invention corrects the acceleration deviation caused by the action deviation due to the limitation of the human joint mobility, so as to correct the slightly non-standard actions made by the user into standard actions, which is convenient for the subsequent matching calculation of the computer and avoids misjudgment. Among them, the types of action deviations and the correction values of each type of action deviation are preset and form a table, and the computer can obtain the corresponding correction scheme for the corresponding action deviation by looking up the table for correction.

[0044] More specifically, in the control analysis step of the present invention, according to the gesture information and the posture information, the current combination relationship of the gesture and posture is obtained, and according to the current combination relationship, the corresponding control instruction is obtained, wherein the control instruction instructs the actuator to perform a preset action. For example, if the current combination relationship is that the gesture is to draw a circle and the posture is to always remain horizontal, then the corresponding control instruction obtained is to command the toy car to stop moving forward, and the toy car as the actuator performs the preset action of stopping. Another example is that if the current combination relationship is that the gesture is to draw an 8 and the posture is to shake left and right, then the corresponding control instruction obtained is to command the toy car to accelerate forward in an 8 shape. In a preferred example, the current combination relationship includes a position relationship, wherein the position relationship is used to indicate the position of the mobile terminal in the gesture action when the gesture posture is generated. For example, the current combination relationship including the position relationship is that the gesture is to draw a triangle, the gesture at the top angle of the triangle is forward inclination, the gesture at the lower left corner of the triangle is left inclination, and the gesture at the lower right corner of the triangle is right inclination, then the corresponding control instruction obtained is to play music.

[0045] The present invention also provides a gesture and posture synchronous detection control system, and the gesture and posture synchronous detection control system can be realized by executing the step flow of the gesture and posture synchronous detection control method. Those skilled in the art can understand the gesture and posture synchronous detection control method as a preferred embodiment of the gesture and posture synchronous detection control system.

[0046] A gesture and posture synchronization detection control system provided according to the present invention includes:

[0047] A motion information acquisition module: to acquire the motion information of the mobile terminal;

[0048] A posture analysis module: to analyze and obtain posture information according to the motion information;

[0049] A gesture analysis module: to analyze and obtain gesture information according to the motion information;

[0050] A control analysis module: to obtain control information according to the gesture information and posture information.

[0051] Preferably, the motion information includes acceleration sensing information, wherein the acceleration sensing information indicates the acceleration of the mobile terminal;

[0052] In the posture analysis module, extract the acceleration component related to the posture from the acceleration sensing information, denoted as the posture acceleration component; match the posture acceleration component with the posture preset information to obtain the posture information;

[0053] In the gesture analysis module, extract the interference acceleration component of the posture acceleration component from the acceleration sensing information, denoted as the gesture acceleration component; match the gesture acceleration component with the gesture preset information to obtain the gesture information.

[0054] Preferably, in the posture analysis module, after matching the posture acceleration component with the posture preset information to obtain the posture information, find the correction information corresponding to the gesture information, and use the correction information to correct the posture information;

[0055] Wherein, the correction information is used to correct the acceleration deviation caused by the action deviation caused by the limitation of the human joint mobility.

[0056] Preferably, in the control analysis module, according to the gesture information and posture information, obtain the current combination relationship of the gesture and posture, and according to the current combination relationship, obtain the corresponding control instruction, wherein the control instruction instructs the actuator to execute the preset action;

[0057] The current combination relationship includes a position relationship, wherein the position relationship is used to indicate the position of the mobile terminal in the gesture action when the gesture posture is generated.

[0058] According to the present invention, there is also provided a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the method are implemented.

[0059] According to the present invention, there is also provided an intelligent terminal, which includes the computer-readable storage medium storing the computer program, or includes the gesture posture synchronization detection control system described above.

[0060] Those skilled in the art know that in addition to implementing the systems, devices and their respective modules provided by the present invention in the form of pure computer-readable program codes, the method steps can be logically programmed to enable the systems, devices and their respective modules provided by the present invention to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. to achieve the same program. Therefore, the systems, devices and their respective modules provided by the present invention can be regarded as a kind of hardware components, and the modules included therein for implementing various programs can also be regarded as the structures within the hardware components; the modules for implementing various functions can also be regarded as either software programs for implementing the methods or the structures within the hardware components.

[0061] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. Without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A gesture posture synchronization detection and control method, characterized in that, Including: Motion information acquisition step: acquiring the motion information of the mobile terminal; Posture analysis step: analyzing to obtain posture information according to the motion information; Gesture analysis step: analyzing to obtain gesture information according to the motion information; Control analysis step: obtaining control information according to the gesture information and the posture information; The actions reflected by the posture information include the actions of the user's wrist; The motion information includes acceleration sensing information, wherein the acceleration sensing information indicates the acceleration of the mobile terminal; In the posture analysis step, extracting the acceleration component related to the posture from the acceleration sensing information, denoted as the posture acceleration component; matching the posture acceleration component with the posture preset information to obtain the posture information; In the gesture analysis step, extracting the interference acceleration component of the posture acceleration component from the acceleration sensing information, denoted as the gesture acceleration component; matching the gesture acceleration component with the gesture preset information to obtain the gesture information; In the posture analysis step, after matching the posture acceleration component with the posture preset information to obtain the posture information, finding the correction information corresponding to the gesture information, and using the correction information to correct the posture information; Wherein, the correction information is used to correct the acceleration deviation caused by the action deviation caused by the limitation of the human joint mobility; Wherein, the control information is obtained according to the combination of the gesture information and the posture information at the same moment or in the same time period.

2. The gesture and posture synchronous detection and control method according to claim 1, wherein In the control analysis step, according to the gesture information and the posture information, obtaining the current combination relationship of the gesture and the posture, and according to the current combination relationship, obtaining the corresponding control instruction, wherein the control instruction instructs the actuator to execute a preset action; The current combination relationship includes a position relationship, wherein the position relationship is used to indicate the position of the mobile terminal in the gesture action when the gesture posture is generated.

3. A gesture posture synchronization detection and control system, characterized in that, Including: Motion information acquisition module: acquiring the motion information of the mobile terminal; Posture analysis module: analyzing to obtain posture information according to the motion information; Gesture analysis module: analyzing to obtain gesture information according to the motion information; Control analysis module: obtaining control information according to the gesture information and the posture information; The actions reflected by the posture information include the actions of the user's wrist; In the posture analysis module, after matching the posture information with the posture preset information to obtain the posture information, finding the correction information corresponding to the gesture information, and using the correction information to correct the posture information; The motion information includes acceleration sensing information, wherein the acceleration sensing information indicates the acceleration of the mobile terminal; In the posture analysis module, extracting the acceleration component related to the posture from the acceleration sensing information, denoted as the posture acceleration component; matching the posture acceleration component with the posture preset information to obtain the posture information; In the gesture analysis module, an interference acceleration component of the attitude acceleration component is extracted from the acceleration sensing information and denoted as the gesture acceleration component; the gesture acceleration component is matched with gesture preset information to obtain gesture information; In the attitude analysis module, after the attitude acceleration component is matched with attitude preset information to obtain attitude information, correction information corresponding to the gesture information is found, and the attitude information is corrected by using the correction information; wherein, the correction information is used to correct the acceleration deviation caused by the motion deviation caused by the limitation of the human joint mobility; wherein, the control information is obtained according to the combination of the gesture information and the attitude information at the same moment or in the same time period.

4. The gesture attitude synchronous detection control system according to claim 3, wherein In the control analysis module, according to the gesture information and the attitude information, the current combination relationship of the gesture and the attitude is obtained, and according to the current combination relationship, a corresponding control instruction is obtained, wherein the control instruction instructs the actuator to execute a preset action; The current combination relationship includes a position relationship, wherein the position relationship is used to indicate the position of the mobile terminal in the gesture action when the gesture posture is generated.

5. A computer-readable storage medium storing a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 2.

6. An intelligent terminal, characterized in that, The intelligent terminal includes the computer-readable storage medium storing the computer program according to claim 5, or includes the gesture attitude synchronous detection control system according to claim 3 or 4.

Images