A control method, system, device, and storage medium based on attitude calculation using inertial sensors.
By calculating the attitude using inertial sensors, the head posture is mapped to the game control input, solving the portability and naturalness issues of wearable devices in game control, and achieving highly compatible and highly accurate wireless control.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN SKYWORTH DIGITAL TECH CO LTD
- Filing Date
- 2025-12-30
- Publication Date
- 2026-05-26
AI Technical Summary
In existing technologies, wearable devices cannot effectively utilize inertial sensors for game control mapping, resulting in shortcomings in the grip, portability, and naturalness of the controller in wearable devices or scenarios where users need to free their hands.
By collecting acceleration and angular velocity data, the attitude of the inertial sensor is calculated and mapped to relative directional displacement data. This data is then transmitted wirelessly to the host computer. Combined with complementary filtering and baseline calibration, the accuracy and stability of the attitude calculation are improved.
A wireless and portable attitude calculation method has been developed, which is suitable for wearable devices such as smart glasses. It improves the accuracy of attitude calculation, reduces the false touch rate, and has high compatibility, enabling it to be recognized as handle input in multiple systems.
Smart Images
Figure CN122083931A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of interactive control, and in particular to a control method, system, device, and storage medium based on attitude calculation using inertial sensors. Background Technology
[0002] With the development of mobile games, cloud gaming, and immersive applications such as AR / VR / MR, users' gaming scenarios have gradually shifted from traditional fixed locations to outdoor and mobile environments. Therefore, users' demands for "portability," "natural interaction," and "immersive experience" are constantly increasing. Traditional wireless input for games mostly uses Bluetooth game controllers (Bluetooth Classic or BLE, typically using HID Profile). Bluetooth game controllers remain the mainstream gaming input method due to their physical joysticks, button combinations, and wide system compatibility, offering advantages such as fast response, high accuracy, and direct compatibility with numerous games. However, in wearable devices (smart glasses) or scenarios where users need to free their hands, the controller's grip, portability, and naturalness have significant shortcomings.
[0003] Against this backdrop, smart glasses and head-mounted devices are becoming increasingly popular, and they generally integrate inertial measurement units (IMUs, including accelerometers and gyroscopes, with some devices also including magnetometers). These sensors can capture the wearer's head posture information in real time. If the head posture can be stably mapped to a virtual joystick (X / Y) input consistent with game semantics and transmitted wirelessly to the host device, a "handset-free, natural, and wireless" game directional control experience can be achieved, or other handheld controllers can be used to assist in directional control.
[0004] However, in existing technologies, the main control mapping mainly uses devices that are easy to map, such as mice, and there is a lack of technology for using wearable devices to collect posture data for mapping input control. Summary of the Invention
[0005] This invention provides a control method, system, device, and storage medium based on inertial sensor attitude calculation to solve the problem that existing interactive systems cannot be controlled using wearable devices.
[0006] This invention provides a control method based on attitude calculation using inertial sensors.
[0007] A control method based on attitude calculation using inertial sensors includes the following steps: Acceleration and angular velocity are collected, and motion attitude data is obtained by solving the motion attitude based on the acceleration and angular velocity. The motion posture data is calculated to obtain the relative attitude angle change, and the relative attitude angle change is mapped to the relative direction displacement to obtain the relative direction displacement data. The relative direction displacement data is sent to the host computer; The host receives and uses the relative direction displacement data.
[0008] A further technical solution involves, after acquiring acceleration and angular velocity data, performing the following steps: If the acceleration and angular velocity data are lower than the set minimum values, they are considered to be 0; If the acceleration and angular velocity data exceed the set minimum values, continue with the subsequent steps.
[0009] A further technical solution is that the relative attitude angle change includes any two or all of the yaw angle change, pitch angle change, and roll angle change; The relative directional displacement includes X-direction displacement and Y-direction displacement.
[0010] A further technical solution is that the specific steps for performing linear scaling are as follows: The ratio of the change in yaw angle to the full yaw amplitude is taken as the displacement in the X direction, and the ratio of the change in pitch angle to the full pitch amplitude is taken as the displacement in the Y direction. The displacements in the X and Y directions are limited.
[0011] A further technical solution involves performing the following steps before acquiring the acceleration and angular velocity: Identify the current scene state, and perform baseline calibration after the current scene state meets the conditions of the scene to be calibrated. Record the current yaw angle reference position and pitch angle reference position as the reference position.
[0012] A further technical solution is that the current scenario state satisfies the conditions of the scenario to be calibrated, meaning that the scenario to be calibrated satisfies any of the following scenario states: first power-on pairing, manual button calibration, or short-term static zero bias.
[0013] A further technical solution involves performing the following steps after the real-time sampling and attitude calculation: Filtering, smoothing and output quantization: The relative direction displacement data obtained by mapping is subjected to a first-order low-pass filter to obtain the relative direction output displacement data; The relative direction output displacement data is used as input.
[0014] In addition, the present invention provides an attitude calculation system based on inertial sensors.
[0015] An attitude calculation system based on inertial sensors includes an inertial sensing unit, an attitude calculation module, a mapping module, a drift compensation module, and a transmission module. The inertial sensing unit is used to collect acceleration and angular velocity; The attitude calculation module is used to fuse the collected acceleration and angular velocity data to calculate and output the motion attitude; The mapping module is used to map the relative change in motion posture into relative directional displacement data output. The drift compensation module is used to record the current yaw angle reference position and pitch angle reference position, and to perform baseline calibration; The transmission module is used to send the relative direction displacement data to the host.
[0016] In addition, the present invention provides a wearable device for attitude calculation based on inertial sensors.
[0017] A wearable device based on inertial sensor attitude calculation includes an inertial sensor attitude calculation system as described above.
[0018] In one aspect, the present invention also provides a computer-readable storage medium.
[0019] A computer-readable storage medium storing a computer program that, when executed by a processor, implements a control method based on inertial sensor attitude calculation as described above.
[0020] The above technical solution has the following advantages: This application converts motion posture into relative attitude angle changes, providing a wireless and portable attitude calculation method, offering another implementation method for device operation. Furthermore, this application can utilize wireless transmission, balancing mobility and portability, making it suitable for wearable devices such as smart glasses. Further, this application introduces complementary filtering, dead zone filtering, and output filtering steps into the attitude calculation method, improving the accuracy of attitude calculation. It also adds an automatic calibration step, which can suppress drift and jitter, reducing the false touch rate. The method of this application has high compatibility: if using a BLE HID Profile, it can be recognized as a gamepad by most systems without additional drivers; if using a custom GATT, it can be adapted to more scenarios through the host app. Attached Figure Description
[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0022] Figure 1 This is a flowchart illustrating an embodiment of the attitude calculation and control method based on inertial sensors according to the present invention. Figure 2This is a block diagram of an inertial sensor-based attitude calculation system according to one embodiment of the present invention. Detailed Implementation
[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0024] In one embodiment, such as Figure 1 As shown, a control method based on attitude calculation using inertial sensors is provided, comprising the following steps: It is understood that the control method based on inertial sensor attitude calculation disclosed in this application can be applied to any device equipped with an inertial sensor, information processing unit, and transmission module, including VR glasses, smart glasses, and headphone devices.
[0025] As an example, when applying the control method to smart glasses, it is understood that the smart glasses can control applications on the smart glasses through the inertial sensor attitude calculation control method, and can also transmit the calculated data to other host devices, such as computer hosts, game consoles, controlled tablets, etc. for control.
[0026] As an example, the specific steps are as follows: use smart glasses to collect changes in the user's head posture, map the input from the joystick on the controller, and control the game console.
[0027] S1: Power on and pairing; This process is consistent with the connection of a typical wireless device. For example, the device powers on to initialize the inertial sensor and communication protocol stack, enters pairing mode, or automatically connects to the nearest paired host.
[0028] As an example, the inertial sensor employs a 9-axis IMU (gyroscope + accelerometer + magnetometer) to achieve long-term stable correction of absolute heading (Yaw), and the communication protocol is the BLE protocol.
[0029] It is understandable that when smart glasses or other data acquisition terminals control applications on smart glasses or other data acquisition terminals through sensor attitude calculation control methods, pairing is not required. The data is transmitted and used within the smart glasses. Therefore, pairing is not a necessary step in this control method.
[0030] As an example, the IMU sampling rate can be adjusted from 100 to 500 Hz. In this embodiment, the sampling rate fs is 200 Hz. In a preferred embodiment, a baseline calibration step is performed after power-on pairing and before data acquisition begins.
[0031] S2: Identify the current scene state. Once the current scene state meets the conditions for the scene to be calibrated, perform baseline calibration. The specific steps are as follows: remind the user to keep the device stationary, and the device records the current yaw angle reference position and pitch angle reference position as the reference position (Pitch0, Yaw0).
[0032] Understandably, the application scenarios for the baseline calibration step can be set according to requirements. It can be started automatically in scenarios such as initial power-on pairing and automatic short-term static zero bias update, or it can be calibrated manually by the user by pressing a button, or it can be calibrated when any of the above scenarios are met.
[0033] The purpose of baseline calibration is to address baseline drift that occurs after prolonged use or upon startup, and calibration can improve data accuracy.
[0034] As an example, the automatic short-time stationary zero-bias update condition is: stationary condition: angular velocity RMS < ω_stable and lasts for T_stable.
[0035] Where ω_stable is the stable angle and T_stable is the stable time.
[0036] S3: Real-time sampling and attitude calculation, collecting acceleration and angular velocity, and calculating motion attitude; As an example, acceleration and angular velocity are acquired at a frequency of fs Hz; complementary filtering or Madgwick algorithm is used to output yaw angle Pitch(t) and pitch angle Yaw(t).
[0037] The complementary filtering coefficient α can be taken as 0.96~0.98, and in this embodiment it is taken as 0.97; S4: Calculate the mapping, calculate the relative attitude angle change from the collected attitude data, and map the relative attitude angle to the relative directional displacement; The relative attitude angle change includes any two or all of the following: yaw angle change, pitch angle change, and roll angle change; In a preferred embodiment, after collecting acceleration and angular velocity data, dead zone filtering is performed.
[0038] It is understood that the dead zone filtering step can be performed in any step between the sampling and mapping steps, and the determination method is as follows: If the yaw angle change and pitch angle change are lower than the set minimum value, i.e., |Δ|<δ (dead zone), then the yaw angle change and pitch angle change are considered to be 0; If the set minimum value is exceeded, the subsequent steps will continue to map the relative attitude angle to the relative directional displacement; In daily use, it is impossible for users to remain completely still. In order to filter out erroneous input caused by slight movements of the user, a minimum relative attitude angle setting has been added.
[0039] As an example, the dead zone δ can be taken as 1~4°, and as an example, it can be taken as 2°; If |Δ| ≥ δ, then perform the mapping: It is understood that the mapping can adopt various methods such as linear mapping, exponential mapping, and dynamic mapping to adapt to different game or application operation requirements.
[0040] As an example, relative attitude angle changes include yaw angle changes and pitch angle changes, and relative directional displacements include X-direction displacement and Y-direction displacement.
[0041] As an example, a linear proportional mapping is used between relative attitude angle changes and relative directional displacements, specifically including the following steps: The ratio of the change in yaw angle to the full yaw amplitude is taken as the displacement in the X direction, and the ratio of the change in pitch angle to the full pitch amplitude is taken as the displacement in the Y direction. The displacements in the X and Y directions are limited.
[0042] As an example, the specific calculation steps are as follows: X_raw = ΔYaw / θ_max; Y_raw = -ΔPitch / φ_max; Where X_raw represents the displacement data in the X direction; Y_raw represents the displacement data in the Y direction; ΔPitch represents the pitch angle change; ΔYaw represents the change in yaw angle; θ_max is the configured yaw full angle, which can be configured from 20° to 45°. As an example, it is set to 30°, which corresponds to the joystick full angle. φ_max is the configured full pitch angle, which can be configured from 15° to 30°. As an example, it is set to 20°, which corresponds to the full pitch angle of the joystick.
[0043] In a preferred embodiment, the following steps are employed in the linear scaling mapping: The mapped input is clipped to [-1, 1], specifically: X = clip(ΔYaw / θ_max, -1,1); Y = clip(-ΔPitch / φ_max, -1,1); In a preferred embodiment, after real-time sampling and attitude calculation, the following steps are performed: S5: Filtering, smoothing and output quantization. The mapped displacement data is subjected to a first-order low-pass filter to obtain the output displacement data. The relative direction output displacement data is used as the input. As an example, a first-order low-pass filter is applied to X_raw / Y_raw (displacement data in the X and Y directions) to obtain X_out / Y_out (output displacement data in the X and Y directions), and then X_out / Y_out is mapped to the HID joystick values of a standard Bluetooth game controller. X_byte = round(127 * X_out), Y_byte = round(127 * Y_out).
[0044] As an example, a first-order low-pass filter (exponential smoothing) is used: X_out(t) = β X_out(t-Δt) + (1-β) X(t).
[0045] The typical range of β is 0.80 to 0.92, which is used to balance response speed and stability; in this embodiment, it is taken as 0.85.
[0046] S6: Data transmission, sending the mapped relative direction displacement data to the host; As an example, the frame includes a sequence number, timestamp, X_byte, Y_byte, mode_flag, and CRC; it is sent to the host via Notify (every 10~20 ms). The transmission is performed via BLE. The BLE transmission period can be adjusted based on the chip and power consumption, using either 10 ms (100 Hz) or 20 ms (50 Hz). In this embodiment, 10 ms (100 Hz) is used.
[0047] S7: Host parsing and input; the host accepts and uses relative directional displacement data.
[0048] It is understood that the host can be any device that requires operation using relative directional displacement, such as a computer host, a game console, a controlled tablet, etc. If the device is a BLE HID game console, the console will recognize the input relative directional displacement as gamepad input data and use it directly; If it is a custom GATT, the host-side app parses it and injects the value into the Gamepad API or simulates keyboard / mouse input. Understandably, HID is preferred during use to improve compatibility.
[0049] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0050] In one embodiment, see Figure 2 This paper provides an inertial sensor-based attitude calculation system, which corresponds one-to-one with the inertial sensor-based attitude calculation and control method described in the above embodiments.
[0051] An attitude calculation system based on inertial sensors includes an inertial sensing unit, an attitude calculation module, a mapping module, a drift compensation module, and a transmission module. The inertial sensing unit is used to collect acceleration and angular velocity; As an example, a three-axis accelerometer (ax, ay, az) and a three-axis gyroscope (gx, gy, gz) are used. In a further preferred example, a triaxial magnetometer (mx, my, mz) is added for heading correction.
[0052] Sensor sampling rate: 100~200 Hz.
[0053] The attitude calculation module is used to fuse accelerometer and gyroscope data to calculate and output motion attitude; In a further preferred example, the attitude calculation module uses complementary filtering or fusion algorithms such as Madgwick / Mahony to combine gyroscope and accelerometer (and optional magnetometer) to obtain real-time Euler angles or quaternions: Pitch(t), Yaw(t), Roll(t).
[0054] The mapping module is used to map the relative change in motion posture into relative directional displacement data output. As an example, it works by using a linear mapping: X_raw = ΔYaw / θ_max, Y_raw = -ΔPitch / φ_max; In a further preferred example, the mapping module is used to perform dead zone (δ) and clipping processing on the relative changes in motion attitude: Specifically, if the relative change in motion posture |Δ| < δ, then the output is 0; And, the mapped input is clipped to [-1, 1], specifically as follows: X = clip(ΔYaw / θ_max, -1,1); Y = clip(-ΔPitch / φ_max, -1,1); The transmission module is used to send relative displacement data to the host.
[0055] As an example, BLE HID Profile (which the system recognizes as a gamepad) or custom GATTService (which requires the host app to resolve) are preferred.
[0056] In a further preferred example, the attitude calculation and control system also includes a drift compensation module; The drift compensation module is used by the device to record the current yaw angle reference position and pitch angle reference position, and to perform baseline calibration.
[0057] In a further preferred example, the drift compensation module also includes a magnetometer to assist in long-term correction of the yaw.
[0058] In another embodiment, a wearable device for attitude calculation is provided, which includes an inertial sensor attitude calculation system.
[0059] The various modules in a wearable device for inertial sensor attitude calculation can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or they can be stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.
[0060] In another embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program implements the attitude calculation and control methods described in the above embodiments; to avoid repetition, these details will not be repeated here. Alternatively, when executed by a processor, the computer program implements the functions of each module / unit in the wearable device embodiment described above for attitude calculation; to avoid repetition, these details will not be repeated here. The computer-readable storage medium can be non-volatile or volatile.
[0061] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0062] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0063] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A control method based on attitude calculation using inertial sensors, characterized in that, Includes the following steps: Acceleration and angular velocity are collected, and motion attitude data is obtained by solving the motion attitude based on the acceleration and angular velocity. The motion posture data is calculated to obtain the relative attitude angle change, and the relative attitude angle change is mapped to the relative direction displacement to obtain the relative direction displacement data. The relative direction displacement data is sent to the host computer; The host receives and uses the relative direction displacement data.
2. The control method based on inertial sensor attitude calculation as described in claim 1, characterized in that, After collecting acceleration and angular velocity data, perform the following steps: If the acceleration and angular velocity data are lower than the set minimum values, they are considered to be 0; If the acceleration and angular velocity data exceed the set minimum values, continue with the subsequent steps.
3. The control method based on inertial sensor attitude calculation as described in claim 1, characterized in that, The relative attitude angle change includes any two or all of the yaw angle change, pitch angle change, and roll angle change; The relative directional displacement includes X-direction displacement and Y-direction displacement.
4. The control method based on inertial sensor attitude calculation as described in claim 3, characterized in that, The mapping of the relative attitude angle change to the relative directional displacement specifically involves linearly proportionally mapping the relative attitude angle change to the relative directional displacement, including the following steps: The ratio of the change in yaw angle to the full yaw amplitude is taken as the displacement in the X direction, and the ratio of the change in pitch angle to the full pitch amplitude is taken as the displacement in the Y direction. The displacements in the X and Y directions are limited.
5. The control method based on inertial sensor attitude calculation as described in claim 1, characterized in that, Before acquiring acceleration and angular velocity, perform the following steps: Identify the current scene state, and perform baseline calibration after the current scene state meets the conditions of the scene to be calibrated. Record the current yaw angle reference position and pitch angle reference position as the reference position.
6. The control method based on attitude calculation using inertial sensors as described in claim 5, characterized in that, The current scenario state meets the conditions for the scenario to be calibrated, meaning that the scenario to be calibrated meets any of the following scenario states: first power-on pairing, manual button calibration, or short-term static zero bias.
7. The control method based on inertial sensor attitude calculation as described in claim 1, characterized in that, After the real-time sampling and attitude calculation, the following steps are performed: Filtering, smoothing and output quantization: The relative direction displacement data obtained by mapping is subjected to a first-order low-pass filter to obtain the relative direction output displacement data; The relative direction output displacement data is used as input.
8. An attitude calculation system based on inertial sensors, characterized in that, It includes an inertial sensing unit, an attitude calculation module, a mapping module, a drift compensation module, and a transmission module; The inertial sensing unit is used to collect acceleration and angular velocity; The attitude calculation module is used to fuse the collected acceleration and angular velocity data to calculate and output the motion attitude; The mapping module is used to map the relative change in motion posture into relative directional displacement data output. The drift compensation module is used to record the current yaw angle reference position and pitch angle reference position, and to perform baseline calibration; The transmission module is used to send the relative direction displacement data to the host.
9. A wearable device for attitude calculation based on inertial sensors, characterized in that, This includes the attitude calculation system based on inertial sensors as described in claim 8.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a control method based on inertial sensor attitude calculation as described in any one of claims 1 to 7.