Posture recognition method and device, electronic equipment and storage medium

By deploying a companion program earlier than the visual inertial unit starts in the mobile terminal, the initialization parameters in the stable state are obtained, and the inaccuracy of the parameter caused by the unstable state of the visual inertial unit is solved, which improves the accuracy of posture recognition sex.

CN119025874APending Publication Date: 2024-11-26BEIJING XIAOMI ROBOT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310575271.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-19
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The mobile terminal may be in an unstable state when started, resulting in inaccurate initialization parameters acquired by the visual inertia unit, which in turn affects the accuracy of posture recognition.

Method used

A companion program is deployed in a mobile terminal, which is earlier than the visual inertial unit starts, and is used to obtain initialization parameters when the terminal is in a stable state, and to obtain these parameters directly from the companion program when the visual inertial unit is initialized.

Benefits of technology

The accuracy of the initialization parameters obtained in the stable state of the terminal is ensured, and the inaccuracy caused by the unstable state is avoided due to the visual inertial unit's own acquisition of parameters is avoided, which improves the accuracy of posture recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025874A_ABST
    Figure CN119025874A_ABST
Patent Text Reader

Abstract

The invention relates to a posture recognition method and device, electronic equipment and a storage medium. The method is applied to the mobile terminal, a visual inertia unit and an accompanying program different from the visual inertia unit are deployed in the mobile terminal, and the accompanying program is started earlier than the visual inertia unit. The method comprises the following steps: the accompanying program obtains an initialization parameter used for initialization by the visual inertia unit under the condition that the mobile terminal is in a stable state; and the visual inertia unit obtains the initialization parameters from the adjoint program under the condition that an initialization condition is met, and carries out initialization based on the obtained initialization parameters so as to output attitude data of the terminal based on an initialization result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of terminal technology, and in particular to a gesture recognition method and device, an electronic device, and a storage medium. Background Art

[0002] When a mobile terminal is in motion, it is necessary to identify its posture at all times. For example, for a robot, it is necessary to identify its posture at all times so as to perform positioning or navigation according to the identified posture. Summary of the invention

[0003] The present disclosure provides a gesture recognition method and device, an electronic device, and a storage medium, which can accurately initialize a visual inertial unit configured in a mobile terminal.

[0004] According to a first aspect of the present disclosure, a gesture recognition method is provided, which is applied to a mobile terminal, wherein a visual inertia unit and a companion program different from the visual inertia unit are deployed in the mobile terminal, wherein the companion program is started earlier than the visual inertia unit; the method comprises:

[0005] The companion program acquires, when the mobile terminal is in a stable state, initialization parameters used by the visual inertial unit for initialization;

[0006] When the initialization condition is met, the visual inertial unit obtains the initialization parameter from the companion program, and performs initialization based on the obtained initialization parameter, so as to output the posture data of the terminal based on the initialization result.

[0007] According to a second aspect of the present disclosure, there is provided a gesture recognition device, applied to a mobile terminal, the device comprising: a visual inertia unit and a companion program different from the visual inertia unit;

[0008] The companion program is started earlier than the visual inertia unit, and is used to obtain initialization parameters used by the visual inertia unit for initialization when the mobile terminal is in a stable state;

[0009] The visual inertial unit is used to obtain the initialization parameters from the companion program when the initialization conditions are met, and to perform initialization based on the obtained initialization parameters, so as to output the posture data of the terminal based on the initialization result.

[0010] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0011] processor;

[0012] a memory for storing processor-executable instructions;

[0013] The processor implements the method as described in the first aspect by running the executable instructions.

[0014] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0015] In the technical solution disclosed in the present invention, a companion program different from the visual inertial unit can be additionally deployed in the mobile terminal to obtain initialization parameters used by the visual inertial unit for initialization. The companion program is started earlier than the visual inertial unit. On this basis, the companion program can obtain the above initialization parameters when the mobile terminal is in a stable state, and the visual inertial unit can directly obtain the initialization parameters from the companion program when the initialization conditions are met for initialization, and then output the posture data of the terminal based on the initialization result.

[0016] It should be understood that the present disclosure additionally sets up an independent companion program for obtaining initialization parameters, and the companion program is started earlier than the visual inertia unit, so that the present disclosure can obtain the above-mentioned initialization parameters when the terminal is in a stable state before the visual inertia unit is started, and the visual inertia unit can directly obtain the initialization parameters from the companion program when initialization is required, avoiding the problem in the related art that the visual inertia unit is required to obtain the initialization parameters for initialization by itself, and the terminal may be in an unstable state when the initialization conditions are met, resulting in inaccurate initialization parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0018] Figure 1 is a flow chart of a gesture recognition method shown in an exemplary embodiment of the present disclosure;

[0019] Figure 2 is a flow chart of a robot posture correction method shown in an exemplary embodiment of the present disclosure;

[0020] Figure 3 is a block diagram of a gesture recognition device shown in an exemplary embodiment of the present disclosure;

[0021] Figure 4 is a block diagram of another gesture recognition device shown in an exemplary embodiment of the present disclosure;

[0022] Figure 5 It is a schematic structural diagram of an electronic device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0024] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0025] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0026] A visual inertial unit, also known as a visual inertial odometry or visual inertial navigation odometer, can be used to identify the posture of a mobile terminal and determine the mileage of the mobile terminal based on continuously acquired posture information.

[0027] Since the visual inertial unit combines the sensor data of multiple sensors such as visual sensors and motion sensors during the gesture recognition process, the gesture recognition accuracy is higher. Therefore, most of the current mobile terminals perform gesture recognition through the visual inertial unit.

[0028] However, although the visual inertial unit has a high accuracy in the process of gesture recognition, since the gesture recognition operation of the visual inertial unit is performed based on the initialization result, the accuracy of the gesture recognition result is also affected by the initialization result. It should be understood that if the initialization result is inaccurate, it will not be guaranteed that the gesture information obtained based on the initialization result is accurate.

[0029] In the related art, the visual inertial unit usually obtains initialization parameters for initialization by itself when it is started, so as to initialize itself based on the obtained initialization parameters. However, when the visual inertial unit is started, the mobile terminal may be in an unstable state. For example, the mobile terminal may be in a complex motion such as high-speed motion, repeated motion with frequent changes in direction, etc. It should be understood that in such an unstable state, the parameters obtained by the sensor are usually inaccurate, which causes the initialization parameters obtained by the visual inertial unit to be inaccurate. Accordingly, the initialization result obtained by the visual inertial unit based on the obtained initialization parameters is also inaccurate.

[0030] It can be seen that in the related art, there is a problem that the initialization result of the visual inertial unit is inaccurate, which leads to inaccurate posture recognition operation of the visual inertial unit.

[0031] From the above introduction, it can be seen that the reason why the related technology has the problem of inaccurate initialization results is that the related technology requires the visual inertial unit to obtain initialization parameters by itself when it is started, and the mobile terminal may be in an unstable state when the visual inertial unit is started.

[0032] In view of this, the present disclosure additionally deploys a companion program in the mobile terminal that is different from the visual inertia unit and is used to obtain the above initialization parameters, wherein the companion program is started earlier than the visual inertia unit. On this basis, the companion program can obtain the initialization parameters used for initialization by the visual inertia unit when the mobile terminal is in a stable state, without the need for the visual inertia unit to temporarily obtain the initialization parameters when the initialization conditions such as being started are met.

[0033] It should be understood that, since the present disclosure deploys a companion program dedicated to obtaining initialization parameters, the present disclosure can obtain accurate initialization parameters at any time when "the mobile terminal is detected to be in a stable state", avoiding the problem of inaccurate initialization parameters obtained in the related art due to the limitation of the acquisition time of initialization parameters such as "when the visual inertial unit is started". It can be seen that the way of setting the companion program in the present disclosure can effectively avoid the problem of inaccurate initialization results in the related art due to the need for the visual inertial unit to obtain initialization parameters by itself.

[0034] The following is an introduction to the various steps of the gesture recognition method proposed in the present disclosure.

[0035] Figure 1 The flowchart of a gesture recognition method is shown in an exemplary embodiment of the present disclosure. The method is applied to a mobile terminal, in which a visual inertia unit and a companion program different from the visual inertia unit are deployed, wherein the companion program is started earlier than the visual inertia unit. Figure 1As shown, the method may include the following steps:

[0036] Step 102: When the mobile terminal is in a stable state, the companion program obtains initialization parameters used by the visual inertial unit for initialization.

[0037] In the present disclosure, the mobile terminal being in a stable state may indicate that the mobile terminal is not in a state where the speed of movement or the moving direction does not change frequently. For example, the stable state may indicate that the moving speed / acceleration of the mobile terminal is less than a preset speed / acceleration, and / or the stable state indicates that the moving direction of the mobile terminal is less than a preset amplitude within a preset time.

[0038] In the present disclosure, since the companion program needs to provide initialization parameters to the visual inertia unit so that the visual inertia unit can be used to initialize itself, the companion program can be started earlier than the visual inertia unit to ensure that the visual inertia unit can obtain the initialization parameters from the companion program at any time.

[0039] In the present disclosure, the startup timing of the companion program needs to be earlier than the visual inertia unit to ensure that the companion program can provide accurate initialization parameters at any time after the visual inertia unit is started. There are no other rigid requirements, which can be determined by those skilled in the art according to actual needs. For example, the companion program can be set to: start automatically when the mobile terminal is turned on. In this case, the requirement of "providing initialization parameters to the visual inertia unit at any time" can be met to the greatest extent; for another example, the companion program can be set to: start when the terminal is in a specific state, for example, the specific state can be when the mobile terminal is detected to be in a stationary state. In this case, it can be guaranteed that the initialization parameters obtained after startup are more accurate. For another example, the specific state can be when the mobile terminal receives a motion instruction but has not yet started moving. Of course, the above examples are only illustrative, and the present disclosure does not limit how to set the startup timing of the companion program.

[0040] In the present disclosure, it is possible to determine whether the mobile terminal is in a stable state based on the motion parameters of the mobile terminal. In one case, the present disclosure can determine that the mobile terminal is in a stable state when the acquired motion parameters represent that the mobile terminal is in a stationary state; in another case, the present disclosure can determine that the mobile terminal is in a stable state when the acquired motion parameters represent that the mobile terminal is in a low-speed motion state with a motion speed lower than a preset speed; in another case, the present disclosure can determine that the terminal is in a stable state when the acquired motion parameters represent that the mobile terminal is in a regular motion state. In the first two cases, the acquired motion parameters can be speed parameters of the mobile terminal, and the mobile terminal is determined to be in a stable state as long as the speed parameters do not reach the preset speed; and in the third case, the conditions for being in regular motion can be set according to actual needs, for example, the motion state with a constant speed but a periodic change in speed direction can be determined as a regular motion, and for another example, the motion state with a constant speed direction and a periodic change in speed can be determined as a regular motion. It should be understood that under regular motion, the change of parameters is also predictable, so the acquisition of the above-mentioned initialization parameters can be determined more accurately. Of course, the above examples are only illustrative. How to determine whether the mobile terminal is in a stable state based on the motion parameters can be determined by those skilled in the art according to actual needs, and the present disclosure does not limit this.

[0041] In addition to determining whether the mobile terminal is in a stable state based on the motion parameters of the mobile terminal, it is also possible to determine whether the mobile terminal is in a stable state based on other information. For example, since the mobile terminal performs corresponding operations based on various instructions and then starts to move, it will not move when it does not receive a motion instruction. Therefore, it is also possible to determine whether the mobile terminal is in a stable state based on whether it receives a motion instruction. For example, when the mobile terminal does not receive a motion instruction, it can be determined that the mobile terminal is in a stable state of stillness, and the accompanying program can obtain initialization parameters.

[0042] Of course, the above examples are merely illustrative. The specific information used to determine whether the mobile terminal is in a stable state and how to determine whether the mobile terminal is in a stable state based on various information are all illustrative and can be determined by technical personnel in this field according to actual needs. The present disclosure does not limit this.

[0043] It should be emphasized that the above operation of determining whether the mobile terminal is in a stable state can be executed by any component in the mobile terminal according to actual conditions. For example, it can be executed by the central processor of the mobile terminal or by the above-mentioned accompanying program, and the present disclosure does not impose any restrictions on this.

[0044] In the present disclosure, the initialization parameters obtained by the accompanying program may be related to the initialization requirements of the visual inertial unit. Since the most important thing in the initialization process of the visual inertial unit is to determine the direction of gravity and the zero bias of the terminal, when the accompanying program determines that the mobile terminal is in a stable state, the initialization parameters obtained may be the direction of gravity and the zero bias of the mobile terminal.

[0045] It should be noted that zero bias refers to the parameter value used to initially correct the posture or motion state of the mobile terminal. For example, when the mobile terminal is in a stationary state, the detected motion parameters such as linear acceleration and angular acceleration should be zero, but the actual detected acceleration value may not be zero, and the zero bias can be the parameter value used to correct the acceleration in the stationary state to zero. For example, the obtained zero bias can include the zero bias of linear acceleration and the zero bias of angular acceleration.

[0046] In the present disclosure, the companion program can obtain sensor parameters in the mobile terminal to obtain initialization parameters based on the obtained sensor parameters. For example, the mobile terminal can be equipped with an IMU (Inertial Measurement Unit), then the companion program can obtain acceleration parameters from the IMU to determine the initialization parameters used for initialization by the visual inertial unit based on the acceleration parameters, for example, the above-mentioned gravity direction and zero bias can be calculated based on the obtained angular acceleration and linear acceleration as initialization parameters. It is worth noting that the IMU can include a gyroscope and an accelerometer, and essentially obtains sensor parameters from the accelerometer and the gyroscope.

[0047] In addition to obtaining sensor parameters from the IMU to determine the initialization parameters, other sensor parameters can also be obtained to determine the initialization parameters. For example, the accompanying program can also obtain sensor parameters from sensors such as speed sensors, displacement sensors, and visual sensors to determine the above initialization parameters. Of course, the above examples are only illustrative, and what kind of sensor parameters to obtain for determining the initialization parameters can be determined by those skilled in the art according to actual needs, and the present disclosure does not limit this.

[0048] Step 104: When the initialization condition is met, the visual inertial unit obtains the initialization parameters from the companion program, and performs initialization based on the obtained initialization parameters, so as to output the posture data of the terminal based on the initialization result.

[0049] In the present disclosure, an initialization condition can be set for the visual inertial unit, so that when the initialization condition is met, the initialization parameters are obtained from the companion program, and the initialization is performed based on the obtained initialization parameters. For example, the initialization condition may be that the visual inertial unit is started; for another example, the initialization condition may be that the posture data output by the visual inertial unit is inaccurate. Of course, this example is only illustrative, and how to set the initialization condition can be determined by those skilled in the art according to actual needs, and the present disclosure does not limit this.

[0050] Among them, when the initialization condition is that the visual inertia unit is activated, it may include a variety of situations. For example, it may be a situation where the visual inertia unit is activated for the first time after the mobile terminal is turned on; for another example, it may be a situation where the visual inertia unit is restarted when the visual inertia unit fails. It should be understood that any situation of restarting the visual inertia unit is deemed to be satisfied by the initialization condition. The specific situation can be determined by those skilled in the art according to actual needs, and the present disclosure does not limit this.

[0051] When the initialization condition is that the posture data output by the visual inertial unit is inaccurate, it can be determined whether the output posture data is accurate in a variety of ways.

[0052] For example, the mobile terminal may also include an odometer for determining the mileage value of the mobile terminal. Then, the mobile terminal may obtain the mileage value output by the odometer, and compare the obtained mileage value with the mileage value calculated based on the posture data output by the visual inertial unit, and then obtain the mileage difference between the two. If the difference between the two is large, for example, the calculated mileage difference exceeds the preset difference, it is determined that the posture data output by the visual inertial unit is inaccurate.

[0053] In this example, the odometer deployed in the mobile terminal can be one of the conventional odometers such as a wheel odometer, a foot odometer, etc. The specific odometer to be used can be determined by technical personnel in this field according to actual needs, and this disclosure does not impose any restrictions on this.

[0054] In addition to comparing mileage to determine whether the posture data is accurate, other methods can also be used to determine whether the posture data is accurate. For example, when the mobile terminal moves along a preset route, that is, when the output posture data is used for navigation, if the posture data is accurate, it must be able to move along the preset route without colliding with other objects. In this case, if the mobile terminal detects a collision with other objects, it can be determined that the output posture data is inaccurate and needs to be reinitialized.

[0055] Of course, the above examples are only illustrative. How to determine whether the posture data output by the visual inertial unit is accurate can be determined by those skilled in the art according to actual needs, and the present disclosure does not limit this.

[0056] It is worth noting that the present disclosure may set only one of the initialization conditions described above, or may set multiple initialization conditions at the same time, so as to perform the initialization operation of the visual inertial unit when any initialization condition is met, and the present disclosure does not limit this.

[0057] In the present disclosure, when the initialization condition includes that the posture data output by the visual inertial unit is inaccurate and the initialization condition is met, since the reinitialization process requires a certain amount of time, the visual inertial unit cannot output the posture data, and there is a certain continuity in the posture change during the movement of the mobile terminal, therefore, the present disclosure can also restore the posture data during the reinitialization process based on the posture data before and after the reinitialization. For example, the mobile terminal can obtain "historical posture data output by the visual inertial unit before determining that the posture data output by the visual inertial unit is inaccurate", as well as the current posture data obtained after the reinitialization, and based on the obtained historical posture data and current posture data, determine the posture data of the mobile terminal during the reinitialization process, for example, the historical posture data and the current posture data can be input into the Kalman filter equation to fit the posture data during the reinitialization process. Of course, this example is only illustrative, and how to obtain the posture data of the mobile terminal during the reinitialization process can be determined by those skilled in the art according to actual needs, and the present disclosure does not limit this.

[0058] It should be noted that the mobile terminal in the present disclosure can be of any type, for example, it can be an emerging mobile terminal such as a robot, a smart car, a drone, or a traditional mobile terminal such as a smart phone, a tablet computer, etc. It should be understood that any mobile terminal equipped with a visual inertial unit can be used as the execution subject of the present disclosure, and the specific type of mobile terminal to which the present disclosure is applied can be determined by those skilled in the art according to actual needs, and the present disclosure does not limit this.

[0059] It can be known from the above technical solution that the present disclosure can deploy a companion program in the mobile terminal that is different from the visual inertial unit and is dedicated to obtaining the initialization parameters used for initialization by the visual inertial unit, and the companion program is started earlier than the visual inertial unit. On this basis, the companion program can obtain the above initialization parameters when the mobile terminal is in a stable state, so that when the visual inertial unit needs to be initialized, the initialization parameters for initialization can be directly obtained from the companion program. In this way, it can be ensured that the initialization parameters used for initialization by the visual inertial unit are accurate initialization parameters obtained when the mobile terminal is in a stable state, avoiding the problem that the related technology obtains the initialization parameters by itself when the visual inertial unit is started, and the initialization parameters may be obtained when the mobile terminal is in an unstable state, resulting in inaccurate initialization parameters, which in turn causes the problem of inaccurate posture data output by the visual inertial unit based on the initialization result.

[0060] Below, the technical solution of the present disclosure is introduced by taking "initializing the visual inertial unit assembled in the robot" as an example. For the convenience of description, "VIO" is used to represent the visual inertial unit, and "Bias" is used to represent the zero bias of the robot.

[0061] Figure 2 The present invention is a flowchart of a robot posture correction method according to an exemplary embodiment of the present invention.

[0062] like Figure 2 As shown, the method may include the following steps:

[0063] Step 201: The robot responds to a power-on instruction and performs a power-on operation.

[0064] In this embodiment, the robot can be powered on by manual remote control, or when a power button is detected to be triggered, the robot can be powered on. In addition, the robot can be powered on according to a preset timed power-on strategy, which is not limited in this embodiment.

[0065] Step 202 , the robot starts the pre-assembled accompanying program during the boot process.

[0066] In this embodiment, the robot can start a pre-installed companion program when it is turned on. For example, the companion program can be started in the BIOS stage of the robot to ensure that the initialization parameters can be updated according to the continuously monitored robot status during the movement of the robot.

[0067] Step 203, the accompanying program checks whether the robot is in a stationary state; if so, jump to step 204, otherwise, continue to monitor the movement state of the robot.

[0068] In this embodiment, the robot being stationary can be regarded as the above-mentioned stable state. On this basis, the accompanying program can monitor the robot's motion state to obtain the initialization parameters of VIO when the robot is in a stationary state. For example, a preset speed can be set, and when it is detected that the robot's motion speed is lower than the preset speed, the robot is assumed to be in a stationary state. At this time, the initialization parameters such as gravity direction and Bias can be obtained.

[0069] Step 204, the accompanying program updates the recorded gravity direction and the robot's Bias.

[0070] In this embodiment, the robot may be equipped with an IMU, and the IMU may include an accelerometer and a gyroscope. Therefore, the accompanying program may obtain accelerometer parameters and gyroscope parameters to calculate the gravity direction and Bias according to the accelerometer parameters and the gyroscope parameters.

[0071] It is worth noting that the operation of obtaining and updating the gravity direction and Bias is performed every time the companion program detects that the robot is in a stationary state. It is a continuous operation, not just performed once.

[0072] Step 205, the robot starts VIO.

[0073] In this embodiment, when the robot starts VIO, VIO must be initialized to perform posture recognition based on the initialization result. For example, the initialization operation of VIO can be: establishing a three-dimensional coordinate system based on the gravity direction and Bias, so as to represent the posture data based on the coordinates and vectors in the three-dimensional coordinate system in the subsequent posture recognition process.

[0074] Step 206, VIO obtains gravity direction and Bias from the companion program.

[0075] In this embodiment, when VIO is initialized, it directly obtains the gravity direction and Bias from the companion program for initialization without obtaining the initialization parameters by itself.

[0076] It is worth noting that the operation of obtaining the gravity direction and Bias from the accompanying program in this embodiment is performed when the initialization conditions are met, including "the above-mentioned robot is turned on" and the "mileage difference exceeds the preset difference" introduced later.

[0077] Step 207: VIO is initialized based on the acquired gravity direction and Bias.

[0078] Step 208: VIO continuously outputs the robot's posture data and visual mileage value based on the initialization result.

[0079] In this embodiment, VIO can also obtain accelerometer parameters and gyroscope parameters from the IMU, and image parameters from the visual sensor (such as a camera) to output the robot's posture data and visual mileage value.

[0080] It is worth noting that VIO outputting posture data is a continuous operation, which predicts the posture of the next moment based on the posture of the previous moment and the acquired sensor parameters, rather than each posture data output directly relying on the initialization result.

[0081] Step 209: The robot continuously obtains the wheel speed mileage value output by the wheel speed odometer.

[0082] In this embodiment, the robot may also be equipped with a wheel speed odometer to obtain the mileage of the robot. In this embodiment, its output may be referred to as a wheel speed mileage value. Although the wheel speed mileage value output by the wheel speed odometer has a certain error and is usually not used as an accurate mileage value, it will not differ too much from the actual mileage. Therefore, in this embodiment, the output of the VIO can be judged based on the wheel speed mileage value to determine whether it is accurate.

[0083] Step 210 , the robot calculates the mileage difference between the wheel speed mileage value and the visual mileage value.

[0084] In this embodiment, the robot can calculate the mileage difference between the wheel speed mileage value and the visual mileage value to determine whether the visual mileage value output by the VIO is accurate based on the mileage difference.

[0085] In step 211, the robot determines whether the mileage difference exceeds a preset difference; if so, it jumps to step 206, otherwise, the VIO continues to output based on the current initialization result.

[0086] In this embodiment, after the mileage difference is calculated, the mileage difference can be compared with the preset difference. If the comparison result shows that the mileage difference exceeds the preset difference, it proves that the posture data output by the VIO is inaccurate. Therefore, the VIO can obtain the gravity direction and Bias from the companion program for reinitialization.

[0087] It is worth noting that the above-mentioned operations of calculating the mileage difference and determining whether the mileage difference exceeds the preset difference can be performed by any component included in the robot, for example, it can be performed by the VIO itself, by a companion program, or by the CPU, and this embodiment does not impose any restrictions on this.

[0088] It can be seen from the above technical solution that the present disclosure can deploy a companion program different from VIO in a mobile terminal such as a robot, which is dedicated to obtaining initialization parameters for VIO initialization. Among them, the companion program can be automatically started when the machine is turned on, ensuring that accurate initialization parameters can be obtained when the mobile terminal is in a static state, so that when VIO needs to be initialized, it can directly obtain the initialization parameters from the companion program without having to temporarily obtain them by itself, thereby ensuring the accuracy of the initialization parameters and avoiding the problem in the related art that the initialization parameters need to be temporarily obtained by VIO when initialization is required, and the mobile terminal is not in a stable state when obtaining, resulting in inaccurate initialization parameters.

[0089] Figure 3 is a block diagram of a gesture recognition device shown in an exemplary embodiment of the present disclosure. The device is applied to a mobile terminal, referring to Figure 3 , the device includes: a visual inertia unit 301 and a companion program 302 different from the visual inertia unit 301;

[0090] A companion program 302 is started before the visual inertia unit is started, and is used to obtain initialization parameters used by the visual inertia unit for initialization when the mobile terminal is in a stable state;

[0091] The visual inertial unit 301 is used to obtain the initialization parameters from the companion program when the initialization conditions are met, and perform initialization based on the obtained initialization parameters, so as to output the posture data of the terminal based on the initialization result.

[0092] Optionally, the accompanying program 302 starts automatically when the mobile terminal is turned on.

[0093] Optionally, the companion program 302 is further used to:

[0094] The gravity direction and the zero bias of the mobile terminal are obtained as initialization parameters used by the visual inertial unit for initialization.

[0095] Optionally, the companion program 302 is further used to:

[0096] Acceleration parameters are acquired from an inertial sensor included in the terminal, so as to determine initialization parameters used by the visual inertial unit for initialization based on the acquired acceleration parameters.

[0097] Optionally, the initialization condition includes at least one of the following:

[0098] The visual inertial unit is activated;

[0099] The posture data output by the visual inertial unit is inaccurate.

[0100] like Figure 4 As shown, Figure 4 is a block diagram of another gesture recognition device shown in an exemplary embodiment of the present disclosure. Figure 3 Based on the illustrated embodiment, it further includes: an acquisition unit 303.

[0101] Optionally, also include:

[0102] An acquiring unit 303 is configured to acquire motion parameters of the terminal;

[0103] Among them, when the acquired motion parameters represent that the mobile terminal is in at least one of the following states, it is determined that the terminal is in a stable state: a stationary state, a low-speed motion state with a motion speed lower than a preset speed, and a regular motion state.

[0104] Optionally, the acquiring unit 303 is further used to:

[0105] Obtaining a mileage value output by an odometer included in the mobile terminal, and calculating a mileage difference between the obtained mileage value and a mileage value calculated based on the posture data output by the visual inertial unit;

[0106] Wherein, when the mileage difference is greater than a preset difference, it is determined that the posture data output by the visual inertial unit is inaccurate.

[0107] Optionally, after reinitialization when the posture data output by the visual inertial unit is inaccurate, the acquisition unit 303 is further used to:

[0108] Acquire historical posture data output by the visual-inertial unit before determining that the posture data output by the visual-inertial unit is inaccurate, and current posture data acquired after reinitialization;

[0109] Based on the historical posture data and the current posture data, the posture data of the mobile terminal during the reinitialization process is determined.

[0110] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0111] Accordingly, the present disclosure also provides a posture recognition device, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to implement a posture recognition method as described in any of the above embodiments, for example, the method may include: instructing the companion program to obtain initialization parameters used for initialization by the visual inertia unit when the mobile terminal is in a stable state; instructing the visual inertia unit to obtain the initialization parameters from the companion program when the initialization conditions are met, and initialize based on the obtained initialization parameters, so as to output the posture data of the terminal based on the initialization result.

[0112] Accordingly, the present disclosure also provides an electronic device, which includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions for implementing a posture recognition method as described in any of the above embodiments. For example, the method may include: instructing the companion program to obtain initialization parameters used for initialization by the visual inertia unit when the mobile terminal is in a stable state; instructing the visual inertia unit to obtain the initialization parameters from the companion program when the initialization conditions are met, and to initialize based on the obtained initialization parameters, so as to output the posture data of the terminal based on the initialization result.

[0113] Figure 5 1 is a block diagram of an apparatus 500 for implementing a gesture recognition method according to an exemplary embodiment. For example, the apparatus 500 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0114] Reference Figure 5 , the device 500 may include one or more of the following components: a processing component 502 , a memory 504 , a power component 506 , a multimedia component 508 , an audio component 510 , an input / output (I / O) interface 512 , a sensor component 514 , and a communication component 516 .

[0115] The processing component 502 generally controls the overall operation of the device 500, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 502 may include one or more modules to facilitate the interaction between the processing component 502 and other components. For example, the processing component 502 may include a multimedia module to facilitate the interaction between the multimedia component 508 and the processing component 502.

[0116] The memory 504 is configured to store various types of data to support operations on the device 500. Examples of such data include instructions for any application or method operating on the device 500, contact data, phone book data, messages, pictures, videos, etc. The memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0117] The power supply component 506 provides power to the various components of the device 500. The power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 500.

[0118] The multimedia component 508 includes a screen that provides an output interface between the device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front camera and / or a rear camera. When the device 500 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0119] The audio component 510 is configured to output and / or input audio signals. For example, the audio component 510 includes a microphone (MIC), and when the device 500 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 504 or sent via the communication component 516. In some embodiments, the audio component 510 also includes a speaker for outputting audio signals.

[0120] I / O interface 512 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0121] The sensor assembly 514 includes one or more sensors for providing various aspects of status assessment for the device 500. For example, the sensor assembly 514 can detect the open / closed state of the device 500, the relative positioning of components, such as the display and keypad of the device 500, the sensor assembly 514 can also detect the position change of the device 500 or a component of the device 500, the presence or absence of user contact with the device 500, the orientation or acceleration / deceleration of the device 500, and the temperature change of the device 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 514 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0122] The communication component 516 is configured to facilitate wired or wireless communication between the device 500 and other devices. The device 500 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (New Radio) or a combination thereof. In an exemplary embodiment, the communication component 516 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0123] In an exemplary embodiment, the apparatus 500 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.

[0124] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, and the instructions can be executed by the processor 520 of the device 500 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0125] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0126] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0127] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A gesture recognition method, It is characterized in that Applied to a mobile terminal, wherein a visual inertia unit and a companion program different from the visual inertia unit are deployed in the mobile terminal, wherein the companion program is started earlier than the visual inertia unit; the method comprises: The companion program acquires, when the mobile terminal is in a stable state, initialization parameters used by the visual inertial unit for initialization; When the initialization condition is met, the visual inertial unit obtains the initialization parameter from the companion program, and performs initialization based on the obtained initialization parameter, so as to output the posture data of the terminal based on the initialization result.

2. The method according to claim 1, It is characterized in that The accompanying program automatically starts when the mobile terminal is turned on.

3. The method according to claim 1, It is characterized in that Also includes: Acquiring motion parameters of the terminal; Wherein, when the acquired motion parameter indicates that the mobile terminal is in at least one of the following states, it is determined that the terminal is in a stable state: A stationary state, a low-speed movement state with a movement speed lower than a preset speed, or a regular movement state.

4. The method according to claim 1, It is characterized in that The obtaining of initialization parameters used by the visual inertial unit for initialization includes: The gravity direction and the zero bias of the mobile terminal are obtained as initialization parameters used by the visual inertial unit for initialization.

5. The method according to claim 1, It is characterized in that The obtaining of initialization parameters used by the visual inertial unit for initialization includes: Acceleration parameters are acquired from an inertial sensor included in the terminal, so as to determine initialization parameters used by the visual inertial unit for initialization based on the acquired acceleration parameters.

6. The method according to claim 1, It is characterized in that The initialization condition includes at least one of the following: The visual inertial unit is activated; The posture data output by the visual inertial unit is inaccurate.

7. The method according to claim 6, It is characterized in that Also includes: Obtaining a mileage value output by an odometer included in the mobile terminal, and calculating a mileage difference between the obtained mileage value and a mileage value calculated based on the posture data output by the visual inertial unit; Wherein, when the mileage difference is greater than a preset difference, it is determined that the posture data output by the visual inertial unit is inaccurate.

8. The method according to claim 6, It is characterized in that After reinitialization when the posture data output by the visual inertial unit is inaccurate, the method further includes: Acquire historical posture data output by the visual-inertial unit before determining that the posture data output by the visual-inertial unit is inaccurate, and current posture data acquired after reinitialization; Based on the historical posture data and the current posture data, the posture data of the mobile terminal during the reinitialization process is determined.

9. A gesture recognition device, It is characterized in that Applied to a mobile terminal, the device comprises: a visual inertia unit and a companion program different from the visual inertia unit; The companion program is started earlier than the visual inertia unit, and is used to obtain initialization parameters used by the visual inertia unit for initialization when the mobile terminal is in a stable state; The visual inertial unit is used to obtain the initialization parameters from the companion program when the initialization conditions are met, and to perform initialization based on the obtained initialization parameters, so as to output the posture data of the terminal based on the initialization result.

10. An electronic device, It is characterized in that include: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 8 by running the executable instructions.

11. A computer-readable storage medium having computer instructions stored thereon, It is characterized in that When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.