An Attitude Control Method and System for an Intelligent Handheld Console

Abstract

The present invention discloses a method and a system for attitude control of an intelligent handheld device. The method specifically includes: collecting the original attitude data of the intelligent handheld device in real time, performing periodic calibration on the original attitude data through a dynamic zero-drift correction algorithm to obtain corrected attitude data; performing attitude calculation and time window sampling on the corrected attitude data to form first attitude data, and inputting the first attitude data into a time series prediction algorithm for prediction calculation to obtain attitude change prediction data; performing fusion processing on the visual data, ultrasonic data, lidar data, and IMU data of the intelligent handheld device to generate high-precision positioning information; based on the attitude change prediction data, high-precision positioning information, and user historical operation data, dynamically generating personalized attitude control instructions by using a user behavior modeling algorithm and a reinforcement learning algorithm. The present invention realizes precise, dynamic, and personalized control of the attitude of the intelligent handheld device, improving the user experience and operation accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent handheld game consoles, and in particular to a posture control method and system for an intelligent handheld game console. Background Art

[0002] With the rapid popularization of smart devices, handheld devices are increasingly used in gaming, virtual reality (VR), augmented reality (AR) and other fields, providing users with a more immersive and interactive experience. However, the existing handheld gesture control technology still faces a series of challenges, which affects the user experience and device performance.

[0003] First, error accumulation and delay are major challenges in handheld gesture control. When used for a long time or performing rapid and continuous actions, the accuracy of gesture control will be affected due to the inherent errors of the sensor and the system response delay. This error accumulation not only reduces the accuracy of gesture control, but may also cause system response lag, which seriously affects the user experience.

[0004] Secondly, the problem of positioning accuracy in complex environments is also a difficult problem that needs to be solved urgently. In low light, magnetic field interference or complex outdoor environments, the accuracy and stability of traditional sensors and positioning technologies will be greatly reduced. This not only limits the application of handheld game consoles in a variety of environments, but may also lead to inaccurate positioning information, thereby affecting the reliability of posture control.

[0005] Furthermore, the problem of user personalized needs is also a major shortcoming of the existing handheld gesture control algorithm. Different users have different control habits and gesture change patterns, but the existing algorithms are often difficult to adapt to the needs of all users. As a result, in some user scenarios, gesture control may not achieve the best effect, reducing user experience satisfaction.

[0006] In addition, in AR applications, handheld devices need to provide accurate visual feedback to guide user operations, but existing technologies often have difficulty maintaining the accuracy and real-time nature of visual feedback in complex environments or during rapid movement. At the same time, although tactile feedback technology can enhance the user's operational perception, how to effectively combine it with the gesture control algorithm to achieve accurate and comfortable tactile feedback is also an urgent problem to be solved. Summary of the invention

[0007] The purpose of the present invention is to provide a posture control method and system for an intelligent handheld game console, which realizes accurate, dynamic and personalized control of the posture of the intelligent handheld game console, improves user experience and operation accuracy, and solves at least one of the above-mentioned prior art problems.

[0008] In a first aspect, the present invention provides a method for controlling a posture of an intelligent handheld game console, the method specifically comprising:

[0009] Collect the original attitude data of the intelligent handheld device in real time according to the accelerometer, gyroscope and magnetometer, and perform periodic calibration on the original attitude data through a dynamic zero-drift correction algorithm to obtain corrected attitude data;

[0010] Perform attitude calculation and time window sampling on the corrected attitude data to form first attitude data, and input the first attitude data into a time series prediction algorithm for prediction calculation to obtain attitude change prediction data;

[0011] Use the Bayesian filtering algorithm to fuse the visual data, ultrasonic data, lidar data and IMU data of the intelligent handheld device to generate high-precision positioning information;

[0012] Based on the attitude change prediction data, the high-precision positioning information and the user's historical operation data, use the user behavior modeling algorithm and the reinforcement learning algorithm to dynamically generate personalized attitude control instructions;

[0013] According to the AR visual feedback data and tactile feedback data of the intelligent handheld device, perform deviation correction processing on the personalized attitude control instructions.

[0014] In a second aspect, the present invention provides an attitude control system for an intelligent handheld device, and the system specifically includes:

[0015] A first attitude control module, configured to collect the original attitude data of the intelligent handheld device in real time according to the accelerometer, gyroscope and magnetometer, and perform periodic calibration on the original attitude data through a dynamic zero-drift correction algorithm to obtain corrected attitude data;

[0016] A second attitude control module, configured to perform attitude calculation and time window sampling on the corrected attitude data to form first attitude data, and input the first attitude data into a time series prediction algorithm for prediction calculation to obtain attitude change prediction data;

[0017] A third attitude control module, configured to use the Bayesian filtering algorithm to fuse the visual data, ultrasonic data, lidar data and IMU data of the intelligent handheld device to generate high-precision positioning information;

[0018] A fourth attitude control module, configured to dynamically generate personalized attitude control instructions based on the attitude change prediction data, the high-precision positioning information and the user's historical operation data by using the user behavior modeling algorithm and the reinforcement learning algorithm;

[0019] A fifth attitude control module, configured to perform deviation correction processing on the personalized attitude control instructions according to the AR visual feedback data and tactile feedback data of the intelligent handheld device.

[0020] In a third aspect, the present invention provides a computer device, including: a memory, a processor, and a computer program stored on the memory. When the computer program is executed on the processor, it implements the attitude control method of the intelligent handheld device as described in any one of the above methods.

[0021] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it implements the attitude control method of the intelligent handheld device as described in any one of the above methods.

[0022] Compared with the prior art, the present invention has at least one of the following technical effects:

[0023] 1. By integrating various sensor data and combining technologies such as time series prediction, Bayesian filtering, user behavior modeling, and reinforcement learning, the present invention realizes precise, dynamic, and personalized control of the attitude of the intelligent handheld device, improving the user experience and operation accuracy.

[0024] 2. By periodically calibrating the original attitude data, the present invention effectively reduces the influence of sensor zero drift on attitude measurement, improving the accuracy and stability of attitude data.

[0025] 3. By setting the determination conditions for the static state of the accelerometer and the static determination conditions for the gyroscope, the present invention can accurately determine whether the sensor is in a static state, thereby triggering zero drift correction and avoiding inaccurate data caused by mis-triggering correction in a dynamic environment.

[0026] 4. By combining dynamic compensation and temperature compensation to compensate the zero drift reference value, the present invention further improves the accuracy of attitude data, especially in different environmental temperatures, and can maintain stable measurement performance.

[0027] 5. By performing standardization processing on the first attitude data and training and inferring the LSTM prediction model, the present invention can accurately predict future attitude changes, providing forward-looking data support for attitude control.

[0028] 6. By fusing various sensor data through the Bayesian filtering algorithm, the present invention generates high-precision positioning information, improving the spatial positioning accuracy and robustness of the intelligent handheld device.

[0029] 7. By combining user historical operation data, attitude change prediction data, and high-precision positioning information, and adopting user behavior modeling and reinforcement learning algorithms, the present invention can dynamically generate personalized attitude control instructions that conform to user operation habits and game scenarios, improving the comfort and response speed of operations.

[0030] 8. The present invention obtains AR visual feedback data and tactile feedback data, performs fusion processing to generate a comprehensive deviation vector, and corrects the deviation of the personalized attitude control instruction based on this vector, further improving the accuracy of attitude control and the user experience. At the same time, the introduction of the adaptive weight coefficient makes the correction process more flexible and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0032] Figure 1 FIG. 9 is a schematic flowchart of an attitude control method for an intelligent handheld device provided by an embodiment of the present invention;

[0033] Figure 2 FIG. 13 is a schematic structural diagram of an attitude control system for an intelligent handheld device provided by an embodiment of the present invention;

[0034] Figure 3 FIG. 17 is a schematic structural diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0035] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0036] It should be understood that when used in the specification and appended claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0037] It should also be understood that the term " / and" as used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0038] As used in the specification and appended claims of this application, the term "if" may be construed, depending on the context, as "when", "once", "in response to determining", or "in response to detecting". Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be construed, depending on the context, to mean "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]".

[0039] In addition, in the description of the specification and appended claims of this application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0040] Reference to "one embodiment" or "some embodiments" or the like described in the specification of this application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0041] In the embodiments of this application, the execution subject of the process includes a terminal device. The terminal device includes, but is not limited to: devices such as servers, computers, smart phones, and tablet computers that can execute the methods disclosed in this application. Figure 1 The flowchart of the attitude control method of the intelligent handheld device disclosed in an embodiment of the present invention is shown and described in detail as follows:

[0042] S101, collect the original attitude data of the intelligent handheld device in real time according to the accelerometer, gyroscope, and magnetometer, and perform periodic calibration on the original attitude data through a dynamic zero-drift correction algorithm to obtain corrected attitude data.

[0043] In this embodiment, the intelligent handheld console is built-in with an accelerometer, a gyroscope, and a magnetometer. These sensors can collect the attitude data of the handheld console in three-dimensional space in real time, including acceleration, angular velocity, and magnetic field direction, etc. Among them, the accelerometer is responsible for collecting the acceleration data of the handheld console in the X-axis, Y-axis, and Z-axis directions, the gyroscope is responsible for collecting the rotational angular velocity data of the handheld console around the X-axis, Y-axis, and Z-axis, and the magnetometer is responsible for collecting the magnetic field direction data of the environment where the handheld console is located, which is used to assist in determining the absolute direction of the handheld console. The dynamic zero-drift correction algorithm is applied to periodically calibrate the original attitude data. This algorithm is based on the principle of automatic zero-offset voltage compensation. It assumes that the zero-offset voltage of the sensor is a slowly changing signal, and uses the zero-drift voltage of the previous moment to cancel the zero-offset voltage of the adjacent next moment, so as to achieve automatic zero-drift compensation. The frequency of periodic calibration can be adjusted according to the usage environment and accuracy requirements of the handheld console. For example, in a scenario with a stable environment and high accuracy requirements, a higher calibration frequency can be set; while in a scenario with a complex environment and low accuracy requirements, a lower calibration frequency can be set. The corrected attitude data processed by the dynamic zero-drift correction algorithm is output to the operating system or relevant application programs of the handheld console, and these data can be used to implement more accurate gesture recognition, game control, navigation positioning and other functions.

[0044] In this embodiment, by periodically calibrating the original attitude data through the dynamic zero-drift correction algorithm, the influence of sensor zero-drift can be effectively eliminated, thereby improving the accuracy of the attitude data, which helps to implement more accurate gesture recognition, game control and other functions, and enhances the user experience. Periodic calibration can ensure that the sensor can maintain stable performance output in different environments, which helps the intelligent handheld console to maintain stable attitude recognition ability in various complex scenarios. By reducing the influence of sensor zero-drift, the wear and failure rate of the sensor can be reduced, which helps to extend the service life of the intelligent handheld console and reduce the maintenance cost of users.

[0045] S102, perform attitude calculation and time window sampling on the corrected attitude data to form first attitude data, and input the first attitude data into a time series prediction algorithm for prediction calculation to obtain attitude change prediction data.

[0046] In this embodiment, the corrected attitude data is first processed by an attitude calculation algorithm. The attitude calculation algorithm uses the data fusion technology of the accelerometer, gyroscope, and magnetometer, and combines the complementarity between the sensors to accurately calculate the real-time attitude of the handheld console. During the attitude calculation process, the noise and errors of the sensors, as well as the cross-interference between them, need to be considered. Through appropriate filtering algorithms and fusion algorithms, the accuracy and stability of the attitude calculation can be effectively improved.

[0047] Perform time window sampling on the data after attitude calculation. Time window sampling is a commonly used signal processing technique that extracts the characteristics of a signal in different time periods by segmenting a continuous signal. In this embodiment, time window sampling is used to extract attitude data within a certain time period from the attitude calculation data to form first attitude data. The size of the time window can be adjusted according to actual application requirements to balance prediction accuracy and computational efficiency.

[0048] Input the first attitude data into a time series prediction algorithm for prediction calculation. A time series prediction algorithm is a mathematical model that can use historical data to predict future data. In this embodiment, an appropriate time series prediction algorithm can be selected, such as an autoregressive moving average (ARMA) model, an autoregressive integrated moving average (ARIMA) model, a recurrent neural network (RNN), etc. These algorithms can capture the time dependence and trend in the attitude data, thereby achieving accurate prediction of future attitude changes.

[0049] The time series prediction algorithm outputs attitude change prediction data. These data represent the predicted values of the handheld console's attitude in a future period of time. The attitude change prediction data can be used in various application scenarios, such as anticipation in game control, attitude prediction in virtual reality, and attitude tracking in motion analysis.

[0050] In this embodiment, through attitude calculation and time window sampling, more accurate and stable attitude data can be extracted, providing high-quality historical data for the time series prediction algorithm, which helps to improve the accuracy of the prediction algorithm and achieve more accurate prediction of future attitude changes. Time window sampling enables the algorithm to process continuously changing attitude data and output prediction results in real time, which helps to enhance the real-time performance of the system, enabling the handheld console to quickly respond to attitude changes and improve the user experience. By selecting an appropriate time window size and prediction algorithm, while ensuring prediction accuracy, the computational efficiency can be optimized, which helps to reduce the computational burden of the system and improve the overall performance of the system.

[0051] S103, use the Bayesian filtering algorithm to fuse the visual data, ultrasonic data, lidar data, and IMU data of the intelligent handheld console to generate high-precision positioning information.

[0052] In this embodiment, the intelligent handheld console is equipped with visual sensors (such as cameras), ultrasonic sensors, lidar sensors, and IMU sensors, which collect information about the environment around the handheld console in real time, including images, distances, 3D point clouds, and accelerations / angular velocities.

[0053] Input visual data, ultrasonic data, lidar data, and IMU data into the Bayesian filtering algorithm. The Bayesian filtering algorithm is based on Bayes' theorem and combines prior knowledge and observed data to recursively update the posterior probability distribution of the system state. In this embodiment, the system state includes the position and attitude of the handheld device, and the observed data comes from various sensors. The Bayesian filtering algorithm generates more accurate and stable positioning information by fusing data from different sensors. After being fused and processed by the Bayesian filtering algorithm, high-precision positioning information is output, including the three-dimensional position, attitude (such as pitch angle, yaw angle, and roll angle), and motion speed of the handheld device, etc.

[0054] In this embodiment, the Bayesian filtering algorithm can fuse data from different sensors, make full use of the complementarity of various sensors, and through fusion processing, can eliminate the errors and uncertainties of a single sensor, thereby improving the positioning accuracy. The Bayesian filtering algorithm has good robustness in dealing with noise and outliers. In a complex environment, a single sensor may be interfered with or blocked, resulting in inaccurate data. However, the Bayesian filtering algorithm can reduce this impact by fusing data from multiple sensors to ensure the stability of the positioning information.

[0055] S104, based on the attitude change prediction data, the high-precision positioning information, and the user historical operation data, use the user behavior modeling algorithm and the reinforcement learning algorithm to dynamically generate personalized attitude control instructions.

[0056] In this embodiment, record the user's operation habits, preferences, and behavior patterns in common scenarios when using the device in the past to form user historical operation data. Use the Hidden Markov Model (HMM) or other user behavior modeling algorithms to model the user historical operation data. Analyze the user's behavior characteristics in different scenarios, identify the user's operation habits and preferences. Establish a user behavior model for predicting the user's possible future behaviors.

[0057] Define the state space, action space, and reward function of the reinforcement learning. The state space includes information such as the current attitude, position, and speed. The action space includes a set of attitude control instructions. The reward function is designed based on factors such as the user behavior model and task objectives to evaluate the pros and cons after executing a certain action. Select a suitable reinforcement learning algorithm (such as Q-learning, Deep Q-Network DQN, Proximal Policy Optimization PPO, etc.) for training. Use the attitude change prediction data and high-precision positioning information as inputs, and continuously optimize the attitude control instructions through the reinforcement learning algorithm to make them more in line with the user's expectations and task objectives. Dynamically generate attitude control instructions that conform to the user's habits and preferences according to the outputs of the user behavior model and the reinforcement learning algorithm. In practical applications, adjust the attitude control instructions according to real-time sensor data and user behavior prediction to achieve a more personalized control experience.

[0058] In this embodiment, through user behavior modeling and reinforcement learning algorithms, the generated posture control instructions are more in line with the user's operation habits and preferences, thereby improving the user experience. The dynamically generated posture control instructions can be adjusted according to different scenarios and user behaviors, enhancing the device's adaptability to different environments and users.

[0059] S105. Perform deviation correction processing on the personalized posture control instructions according to the AR visual feedback data and tactile feedback data of the intelligent handheld device.

[0060] In this embodiment, the intelligent handheld device is equipped with an AR visual sensor and a tactile sensor, which can obtain the user's visual feedback data and tactile feedback data in real time. Using the AR visual feedback data, the visual deviation during the user's interaction with the handheld device is detected in real time. Through image recognition and target tracking technologies, the difference between the user's actual actions and expected actions in the AR environment is analyzed. According to the magnitude and direction of the visual deviation, the personalized posture control instructions are fine-tuned to more accurately reflect the user's intention. The tactile sensor can sense the force and direction when the user touches the handheld device. By analyzing the tactile feedback data, it is judged whether the user encounters resistance or discomfort during actual operation. If it is detected that the tactile feedback does not match the expectation, such as excessive force or deviation in direction, the personalized posture control instructions are further corrected to ensure the comfort and accuracy of the operation. The results of the AR visual feedback data correction and tactile feedback data correction are integrated to form the final personalized posture control instructions. The corrected instructions are output to the control system of the handheld device to achieve more precise and user-friendly posture control.

[0061] In this embodiment, by combining the AR visual feedback data and tactile feedback data, the user's intention and operation deviation can be more accurately identified, thereby improving the operation accuracy. The corrected personalized posture control instructions are more in line with the user's actual needs and operation habits, thus enhancing the comfort and satisfaction of the user experience in AR applications.

[0062] In some embodiments, in the above step S101, the periodic calibration of the original posture data by the dynamic zero-drift correction algorithm to obtain the calibrated posture data specifically includes:

[0063] Set the zero-drift detection conditions and calibration period;

[0064] When the zero-drift condition is triggered or in each calibration period, calculate the zero-drift reference value according to the original posture data of multiple sampling points, and perform zero-drift compensation on the zero-drift reference value through the dynamic compensation formula and temperature compensation formula to obtain the first zero-drift data;

[0065] Perform posture calculation on the first zero-drift data to obtain the calibrated posture data.

[0066] In this embodiment, a threshold is set. When the sensor has no signal input (such as when the attitude sensor is stationary), if the fluctuation of its output value exceeds this threshold, the zero-drift detection condition is triggered. For example, for a six-axis attitude sensor, it can be set that when it is in a stationary state, if the fluctuation of the raw data of the three axes of the gyroscope exceeds a certain range (such as ±0.05° / s), the zero-drift detection is triggered.

[0067] According to the characteristics of the sensor and the usage environment, a fixed time interval is set as the calibration period. For example, it can be set to calibrate once a day, or once every 100 working hours. The setting of the calibration period should ensure the accuracy and stability of the sensor during long-term use.

[0068] When the zero-drift detection condition is triggered or in each calibration period, the zero-drift reference value is calculated by collecting the raw attitude data of multiple sampling points. The number of sampling points can be set as needed. For example, collect the data of 100 or more sampling points. The collected raw data is processed, such as removing outliers and calculating the average value, to obtain the zero-drift reference value. This reference value represents the offset of the sensor when there is no signal input.

[0069] According to the dynamic characteristics of the sensor, a dynamic compensation formula is designed to correct the zero-drift reference value. This formula can be dynamically adjusted based on factors such as the historical data of the sensor and the current working environment. Considering the influence of temperature change on the performance of the sensor, a temperature compensation formula is designed to further correct the zero-drift reference value. This formula can calculate the compensation amount according to the difference between the current temperature and the standard temperature. The results of dynamic compensation and temperature compensation are added together to obtain the first zero-drift data, which has been preliminarily corrected for zero drift. The first zero-drift data is input into the attitude solution algorithm for attitude calculation. This algorithm can calculate the real-time attitude of the object according to the output data of the sensor. During the attitude solution process, the corrected zero-drift data is used to correct the attitude to obtain more accurate corrected attitude data.

[0070] In this embodiment, through the zero-drift detection and compensation technology, the zero point position of the sensor can be tracked and corrected in real time to ensure the accuracy and stability of the measurement results, which is particularly important for occasions requiring high-precision attitude control. Regular calibration and dynamic compensation can adapt to the changes in sensor performance and the influence of the working environment, enhancing the robustness and reliability of the system. Even if the sensor has a certain degree of performance degradation or environmental change during use, the system can still maintain good performance.

[0071] Furthermore, the zero-drift detection conditions include the acceleration sensor stationary state determination condition and the gyroscope stationary determination condition;

[0072] The determination condition for the accelerometer in the stationary state is , where , and represent the original output values of the three axes of the accelerometer, g represents the local acceleration due to gravity, represents the stationary determination threshold;

[0073] The determination condition for the gyroscope to be delicate satisfies , where , and represent the original output values of the three axes of the gyroscope, represents the stationary determination threshold of the gyroscope.

[0074] In this embodiment, , and respectively represent the original output values of the accelerometer in the X, Y, and Z axial directions. An accelerometer is a sensor that measures the acceleration force. It is usually used for motion detection and device navigation. In the stationary state, the accelerometer is mainly affected by the gravity, so its output value should be related to the local acceleration due to gravity g. g represents the local acceleration due to gravity, which is a constant, and its value depends on the location and altitude on the Earth's surface. In most areas, the value of g is about 9.8 m / s². The stationary determination threshold is a preset threshold used to determine whether the accelerometer is in the stationary state. When the difference between the combined acceleration of the original output values of the three axes of the accelerometer and g is within the stationary determination threshold range, it can be considered that the accelerometer is in the stationary state. By setting the stationary determination threshold, it is possible to accurately determine whether the accelerometer is in the stationary state, thus providing a reliable basis for subsequent zero-drift detection. When the accelerometer is in the stationary state, its output value should mainly reflect the acceleration due to gravity. At this time, zero-drift detection can more accurately measure the zero offset of the sensor.

[0075] , and respectively represent the original output values of the gyroscope in the X, Y, and Z axial directions. A gyroscope is a sensor used to measure the angular velocity. It can detect the angular velocity of an object rotating around a certain axis. The stationary determination threshold of the gyroscope is a preset threshold value used to determine whether the gyroscope is in a stationary state. When the raw output values of the three axes of the gyroscope are all less than the gyroscope stationary determination threshold value, it can be considered that the gyroscope is in a stationary state. By setting the gyroscope stationary determination threshold value, it is possible to accurately determine whether the gyroscope is in a stationary state, thereby avoiding errors caused by zero drift detection in a dynamic environment. When the gyroscope is in a stationary state, its output value should be close to zero. At this time, zero drift detection can more accurately measure the zero offset of the sensor, which helps to improve the measurement accuracy of the sensor.

[0076] In this embodiment, the parameters in the accelerometer stationary state determination condition and the gyroscope stationary determination condition play a crucial role in zero drift detection. By reasonably setting these parameters, it is possible to accurately determine whether the sensor is in a stationary state, thereby providing a reliable basis for subsequent zero drift detection and compensation, which helps to improve the measurement accuracy and stability of the sensor.

[0077] Further, calculating a zero drift reference value based on the raw attitude data of multiple sampling points, and performing zero drift compensation on the zero drift reference value through a dynamic compensation formula and a temperature compensation formula to obtain first zero drift data, specifically including:

[0078] Calculating the zero drift reference values corresponding to the accelerometer, gyroscope, and magnetometer respectively according to the raw attitude data of multiple sampling points, and the zero drift reference values satisfy

[0079]

[0080] where 、 and represent the zero drift reference values of the accelerometer, gyroscope, and magnetometer respectively, 、 and represent the raw attitude data collected by the accelerometer, gyroscope, and magnetometer at the i-th sampling point respectively, and N represents the number of sampling points;

[0081] Using the dynamic compensation formula to perform zero drift compensation on 、 and to obtain second zero drift data, and the dynamic compensation formula satisfies , , ,where 、 and represent the zero drift data corresponding to the accelerometer, gyroscope, and magnetometer in the second attitude data respectively, 、 and respectively represent the original attitude data collected by the accelerometer, gyroscope, and magnetometer;

[0082] Use the temperature compensation formula for , and to perform zero-drift compensation and obtain the first zero-drift data. The temperature compensation formula satisfies

[0083]

[0084]

[0085]

[0086] where T represents the current ambient temperature, represents the calibration temperature, , and respectively represent the zero-drift data corresponding to the accelerometer, gyroscope, and magnetometer in the first zero-drift data, , and respectively represent the corresponding zero-drift reference values of the accelerometer, gyroscope, and magnetometer at the calibration temperature , , and respectively represent the temperature drift coefficients corresponding to the accelerometer, gyroscope, and magnetometer.

[0087] In this embodiment, the zero-drift reference values , and of the accelerometer, gyroscope, and magnetometer are calculated by averaging or other statistical methods on the original attitude data of multiple sampling points, representing the zero-offset of the sensor in a stationary or standard state. These reference values are used for subsequent zero-drift compensation to eliminate the zero-drift error of the sensor.

[0088] The original attitude data , and of the i-th sampling point are collected during the actual measurement of the sensor, including the true attitude information and possible zero-offset. Through the data of multiple sampling points, the zero-drift reference value can be calculated more accurately.

[0089] The number of sampling points N determines the accuracy and stability of the calculation of the zero-drift reference value. The more sampling points, the closer the calculated reference value is to the true value, but the calculation amount will also increase accordingly.

[0090] The zero-drift data , and is used to correct the zero - point offset in the original attitude data.

[0091] The current ambient temperature T is one of the important factors affecting the performance of the sensor. By measuring the current ambient temperature, the zero - point drift amount caused by temperature change can be calculated.

[0092] Calibration temperature is the reference temperature set during the factory production or calibration process of the sensor. At the calibration temperature, the zero - point offset of the sensor is accurately measured and recorded.

[0093] Temperature drift coefficient 、 and represent the rate at which the zero - point drift of the sensor changes with temperature. By measuring the zero - drift reference value at the calibration temperature and the zero - drift data at the current ambient temperature, the temperature drift coefficient can be calculated and used for subsequent temperature compensation.

[0094] In this embodiment, by calculating the zero - drift reference value and performing dynamic and temperature compensation, the zero - point drift error of the sensor can be significantly reduced, thereby improving the measurement accuracy, which is particularly important for application scenarios that require high - precision attitude measurement. Dynamic compensation and temperature compensation can adapt to the changes in sensor performance and the influence of the working environment, enabling the system to maintain good performance under various conditions and contributing to enhancing the robustness and reliability of the system.

[0095] In some embodiments, in the above step S102, the inputting the first attitude data into the time - series prediction algorithm for prediction calculation to obtain attitude change prediction data specifically includes:

[0096] Performing normalization processing on the first attitude data to obtain second attitude data, and the second attitude data satisfies , where represents the second attitude data, represents the first attitude data, represents the mean value of Euler angles within the time window, represents the standard deviation of Euler angles within the time window;

[0097] Inputting the second attitude data into the LSTM prediction model, and performing training and inference through the forward - propagation function to output the attitude change prediction value;

[0098] Fusing the attitude change prediction value with the current attitude value to generate attitude change prediction data, and the attitude change prediction data satisfies , where represents the attitude change prediction value, represents the current attitude value, Represents the predicted estimate of the attitude change at a preset future moment, where t represents the current moment. Represents a preset future moment.

[0099] In this embodiment, the Euler angles are a set of angular values that describe the rotational attitude of an object in three-dimensional space. The mean of the Euler angles within the time window is used to measure the average level of the attitude data over a period of time, which helps to remove the overall offset in the data during the normalization process. The standard deviation of the Euler angles within the time window reflects the degree of dispersion of the data. During the normalization process, the standard deviation can be used to scale the original data into a standard range, usually between 0 and 1, which helps to improve the convergence speed and prediction accuracy of the model. The second attitude data is the data after normalization processing, which has a unified dimension and range and is more suitable as the input of the LSTM prediction model. The attitude change prediction value is the output of the LSTM prediction model, representing the prediction of the attitude change over a future period of time. The current attitude value is the attitude data at the current moment, which is used to fuse with the attitude change prediction value to generate the final attitude change prediction data. The predicted estimate of the attitude change at a preset future moment is the result after fusing the current attitude value and the attitude change prediction value, representing the estimate of the attitude change at a specific future moment. The preset future moment specifies the specific time point for predicting the attitude change.

[0100] In this embodiment, through the normalization process, the original attitude data is converted into data with a unified dimension and range, which helps to accelerate the convergence speed of the LSTM prediction model and improve the training efficiency. The normalization process eliminates the overall offset and dimension differences in the original data, enabling the LSTM prediction model to more accurately capture the long-term dependencies in the sequence data, thereby improving the accuracy of attitude prediction. By fusing the attitude change prediction value with the current attitude value to generate the attitude change prediction data, both the attitude information at the current moment and the prediction of the attitude change over a future period of time are considered, making the generated prediction data more comprehensive and accurate. The LSTM prediction model can capture the long-term dependencies in the sequence data and has good adaptability to attitude change prediction in complex environments. For example, in a dynamic or uncertain environment, the LSTM model can predict future attitude changes based on historical data, providing strong support for tasks such as attitude control and navigation.

[0101] In some embodiments, in the above step S103, the fusion processing of the visual data, ultrasonic data, lidar data, and IMU data of the intelligent handheld device using the Bayesian filtering algorithm specifically includes:

[0102] Set the state vector and the observation vector , where represents the position coordinates of the intelligent handheld device in three-dimensional space, represents the velocity components of the intelligent handheld device in three-dimensional space, represents the attitude quaternion of the intelligent handheld device, respectively represent visual data, ultrasonic data, lidar data, and IMU data;

[0103] Input the state vector into the state transition function to perform state prediction on the IMU drive of the intelligent handheld device, and obtain a predicted state vector, where the predicted state vector satisfies , where represents the predicted state vector, represents the state vector at the previous moment, represents the IMU data at the current moment, represents the state transition function, represents the process noise;

[0104] Fuse the predicted state vector and the observation vector to obtain an updated state vector, where the updated state vector satisfies , , where represents the Kalman gain, represents the observation model for mapping the predicted state vector to the sensor observation space, represents the predicted state covariance matrix, represents the Jacobian matrix of the observation matrix, represents the observation noise covariance matrix.

[0105] In this embodiment, the position coordinates represent the absolute position of the intelligent handheld device in three-dimensional space and are the core parameters of the navigation and positioning system.

[0106] The velocity components describe the motion speed of the intelligent handheld device in three-dimensional space and are crucial for predicting future positions.

[0107] The attitude quaternion is used to represent the spatial attitude of the intelligent handheld device, that is, the rotation angle and direction relative to a certain reference coordinate system. Quaternions are more suitable for interpolation and numerical calculations than Euler angles or rotation matrices because they can avoid problems such as gimbal lock.

[0108] Visual data , ultrasonic data , lidar data , IMU data Coming from different sensors respectively, they provide direct or indirect measurements of the position and attitude of the intelligent handheld device. Visual data may include image feature matching results; ultrasonic data provides distance measurements; lidar data provides a three-dimensional point cloud of the environment; IMU data includes measurements from accelerometers and gyroscopes, providing information about acceleration and angular velocity.

[0109] State transition function Based on the state vector at the previous moment and the IMU data at the current moment, predict the state vector at the current moment. It models the motion of the intelligent handheld device based on physical laws (such as Newton's second law) and the measurement model of the IMU.

[0110] Predicted state vector Represents the state vector at the current moment predicted by the state transition function. It takes into account the direct measurements of the IMU data and the influence of process noise.

[0111] Kalman gain Is a key parameter in Bayesian filtering (especially Kalman filtering), which balances the confidence between the predicted state vector and the observation vector. It is calculated based on the predicted state covariance matrix, the Jacobian matrix of the observation matrix, and the observation noise covariance matrix.

[0112] Observation model Maps the predicted state vector to the sensor observation space for comparison with the observation vector. It takes into account the measurement characteristics and errors of the sensor.

[0113] Predicted state covariance matrix The Jacobian matrix of the observation matrix The observation noise covariance matrix Describe the uncertainty of the predicted state, the linearized approximation of the observation model to the state vector, and the statistical characteristics of the observation noise respectively. They play a key role in calculating the Kalman gain.

[0114] In this embodiment, by fusing data from different sensors, the Bayesian filtering algorithm can make full use of the advantages of each sensor, reduce the errors and uncertainties brought by a single sensor, and thus improve the positioning accuracy of the intelligent handheld device. In a complex or dynamic environment, a single sensor may be interfered with or fail. By fusing data from multiple sensors, the system can tolerate failures or data anomalies of some sensors and maintain overall stability and reliability.

[0115] In some embodiments, in the above step S104, based on the attitude change prediction data, the high-precision positioning information, and the user's historical operation data, using user behavior modeling algorithms and reinforcement learning algorithms, dynamically generate personalized attitude control instructions, specifically including:

[0116] Obtain the user's historical operation data, where the user's historical operation data includes user operation acceleration, user operation angular velocity, game scene type, and environmental complexity;

[0117] Use a convolutional neural network to extract the local spatio-temporal features of the user's historical operation data, use a long short-term memory network to capture the temporal dependence features of the user's historical operation data, and fuse the local spatio-temporal features and the temporal dependence features to form a user behavior feature vector;

[0118] Take the pose change prediction data, the high-precision positioning information, and the user behavior feature vector as the state space, take the proportional coefficient, integral coefficient, and differential coefficient of the controller of the intelligent handheld console and the maximum output torque as the action space, and use minimizing the tracking error, maximizing the operation comfort, and minimizing the energy consumption as the reward function;

[0119] Based on the state space, the action space, and the reward function, use a reinforcement learning algorithm to dynamically generate personalized pose control instructions.

[0120] In this embodiment, through the built-in sensors (such as accelerometers, gyroscopes) of the intelligent handheld console and the game system logs, collect the operation data of the user in different game scenes, including user operation acceleration, user operation angular velocity, game scene type (such as racing, shooting, adventure, etc.) and environmental complexity (such as the number of obstacles, enemy density, etc.). Clean, denoise, and normalize the collected data to ensure the quality and consistency of the data. At the same time, classify and label the data according to the game scene type and environmental complexity.

[0121] A convolutional neural network (CNN) is used to extract the local spatio-temporal features of the user's historical operation data. Taking data such as user operation acceleration and user operation angular velocity as inputs, through multiple convolutional layers and pooling layers, extract the local spatio-temporal patterns in the data, such as the user's operation habits, reaction speed, etc.

[0122] A long short-term memory network (LSTM) is used to capture the temporal dependence features of the user's historical operation data. Taking the feature sequence extracted by the CNN as the input of the LSTM, through the cyclic structure of the LSTM, capture the long-term dependence relationships in the data, such as the user's continuous operation behavior, strategy changes, etc.

[0123] Fuse the local spatio-temporal features extracted by the CNN and the temporal dependence features captured by the LSTM to form a user behavior feature vector. This vector synthesizes information such as the user's operation habits, reaction speed, continuous operation behavior, and strategy changes, and can comprehensively reflect the game behavior characteristics of the user.

[0124] The state space includes attitude change prediction data (such as the attitude angles and angular velocities of the intelligent handheld device), high-precision positioning information (such as GPS coordinates, map information, etc.), and user behavior feature vectors. These state information jointly describe the current state of the intelligent handheld device and the user's behavior characteristics. The action space includes the proportional coefficient, integral coefficient, and differential coefficient of the controller of the intelligent handheld device (i.e., the parameters of the PID controller) and the maximum output torque. These action parameters determine the attitude control instructions of the intelligent handheld device. The reward function includes three parts: minimizing the tracking error (i.e., the difference between the actual attitude and the desired attitude of the intelligent handheld device), maximizing the operation comfort (such as reducing the discomfort caused by sudden acceleration or deceleration), and minimizing the energy consumption (such as reducing the energy consumption of the motor). By adjusting the weights of the reward function, these three goals can be balanced to achieve personalized attitude control.

[0125] The reinforcement learning algorithm (such as the Deep Deterministic Policy Gradient algorithm DDPG) is used to dynamically generate personalized attitude control instructions. The algorithm selects the optimal action space parameters according to the current state space with the goal of maximizing the cumulative reward. Through continuous iteration and optimization, the algorithm can learn the attitude control strategy suitable for the user's behavior characteristics.

[0126] In this embodiment, by extracting the features of the user's historical operation data and performing fusion processing to form the user behavior feature vector, the generation of personalized attitude control instructions is realized. This makes the attitude control of the intelligent handheld device more in line with the user's operation habits and expectations. Through the design of the reward function of minimizing the tracking error, maximizing the operation comfort, and minimizing the energy consumption, the fluency and comfort of the game are improved, enhancing the user's gaming experience. The reinforcement learning algorithm can dynamically adjust the attitude control strategy according to the user's behavior characteristics and the changes in the game scenario, realizing adaptive attitude control.

[0127] In some embodiments, in the above step S105, the deviation correction process of the personalized attitude control instruction according to the AR visual feedback data and tactile feedback data of the intelligent handheld device specifically includes:

[0128] Obtain the AR visual feedback data and the tactile feedback data , where, represents the planar deviation between the operation position of the user in the AR interface and the target position, represents the yaw angle deviation between the user's operation direction and the target direction, represents the vibration intensity of the linear motor, represents the vibration frequency;

[0129] The AR visual feedback data and the tactile feedback data are fused to obtain a comprehensive deviation vector, where the comprehensive deviation vector satisfies

[0130]

[0131]

[0132]

[0133] where represents the comprehensive deviation vector, represents the visual deviation amplitude, represents the visual deviation direction, and represent the tactile intensity coefficient;

[0134] A PID correction coefficient is generated based on the comprehensive deviation vector, and the personalized attitude control instruction is corrected for deviation according to the PID correction coefficient to obtain a corrected personalized attitude control instruction, where the corrected personalized attitude control instruction satisfies

[0135]

[0136]

[0137] where represents the parameter of the corrected personalized attitude control instruction, 、 and represent the proportional coefficient, integral coefficient, and differential coefficient of the controller of the intelligent handheld device respectively, 、 and represent the corrected proportional coefficient, corrected integral coefficient, and corrected differential coefficient in the PID correction coefficient respectively, represents the maximum output torque, 、 and represent the adaptive weight coefficients.

[0138] In this embodiment, the planar deviation between the operation position of the user in the AR interface and the target position reflects the horizontal distance difference between the operation position of the user during the operation in the AR interface and the expected target position. By capturing and analyzing this deviation, the system can understand the accuracy of the user's operation and make adjustments accordingly.

[0139] The yaw angle deviation between the user's operation direction and the target direction represents the angular difference between the user's operation direction and the target direction. It helps the system determine whether the direction of the user's operation is accurate, so as to provide necessary directional adjustments.

[0140] Linear motor vibration intensity Indicates the vibration intensity generated by a haptic feedback device (such as a linear motor). By adjusting the vibration intensity, the system can provide different levels of haptic feedback to the user to simulate different operation effects or alert the user.

[0141] Vibration frequency Determines the rhythm and speed of haptic feedback. Different vibration frequencies can convey different information or effects. For example, high-frequency vibration may indicate urgent or strong feedback, while low-frequency vibration may indicate gentle or continuous reminder.

[0142] Visual deviation amplitude Represents the magnitude of the deviation between the user's operation and the target visually. Visual deviation direction Represents the direction of the deviation between the user's operation and the target visually. Haptic intensity coefficient and Combines haptic feedback data with visual feedback data to form a comprehensive deviation representation. This coefficient reflects the contribution degree of haptic feedback in the comprehensive deviation vector, which helps the system to more comprehensively understand the state of the user's operation.

[0143] Correction proportional coefficient , correction integral coefficient and correction differential coefficient Correspond to the proportional, integral, and differential parts of the PID controller respectively. By adjusting these three coefficients, the system can achieve precise control of the user's operation to reduce deviation and maintain stability.

[0144] Proportional coefficient of the controller of the intelligent handheld device , integral coefficient and differential coefficient Are the basic control parameters of the intelligent handheld device controller, which are used to achieve the preliminary control of the user's operation.

[0145] Maximum output torque Represents the maximum force that the intelligent handheld device can generate when executing control instructions. This parameter helps to ensure that the system has sufficient power to perform necessary adjustment actions.

[0146] Adaptive weight coefficient , and Are used to adjust the contribution degree of the PID correction coefficient in correcting personalized posture control instructions. By adaptively adjusting these coefficients, the system can flexibly adjust the control strategy according to the actual situation to obtain better control effects.

[0147] In this embodiment, by capturing and analyzing the operation position and direction deviation of the user in the AR interface, the system can understand the state of the user's operation in real time and provide necessary adjustment prompts, which helps the user complete the operation task more accurately and improves the operation efficiency. Combining with the haptic feedback data, the system can provide a more intuitive and realistic operation experience to the user. By adjusting the vibration intensity and frequency, the system can simulate different operation effects or remind the user to pay attention, thereby enhancing the user's immersion and satisfaction. Through the deviation correction process of the personalized attitude control instruction by the PID correction coefficient, the system can achieve precise control of the user's operation, which helps to reduce the deviation and maintain stability, and improves the overall control performance of the system.

[0148] Referring to Figure 2 , an embodiment of the present invention provides an attitude control system 2 for an intelligent handheld game console, and the system 2 specifically includes:

[0149] The first attitude control module 201 is configured to collect the original attitude data of the intelligent handheld game console in real time according to the accelerometer, gyroscope, and magnetometer, and perform periodic calibration on the original attitude data through a dynamic zero-drift correction algorithm to obtain corrected attitude data;

[0150] The second attitude control module 202 is configured to perform attitude calculation and time window sampling on the corrected attitude data to form first attitude data, and input the first attitude data into a time series prediction algorithm for prediction calculation to obtain attitude change prediction data;

[0151] The third attitude control module 203 is configured to perform fusion processing on the visual data, ultrasonic data, lidar data, and IMU data of the intelligent handheld game console by using a Bayesian filtering algorithm to generate high-precision positioning information;

[0152] The fourth attitude control module 204 is configured to dynamically generate personalized attitude control instructions based on the attitude change prediction data, the high-precision positioning information, and the user's historical operation data by using a user behavior modeling algorithm and a reinforcement learning algorithm;

[0153] The fifth attitude control module 205 is configured to perform deviation correction processing on the personalized attitude control instruction according to the AR visual feedback data and haptic feedback data of the intelligent handheld game console.

[0154] It can be understood that the content in the embodiment of the attitude control method for the intelligent handheld game console as Figure 1 shown is applicable to the embodiment of the attitude control system for the intelligent handheld game console of the present invention. The functions specifically implemented by the embodiment of the attitude control system for the intelligent handheld game console of the present invention are the same as those in the embodiment of the attitude control method for the intelligent handheld game console as Figure 1 shown, and the beneficial effects achieved are the same as those in the embodiment of the attitude control method for the intelligent handheld game console as Figure 1The beneficial effects achieved by the embodiments of the posture control method of the intelligent handheld device shown are also the same.

[0155] It should be noted that for the content such as information interaction and execution process among the above systems, since they are based on the same concept as the method embodiments of the present invention, for their specific functions and the technical effects brought, reference can be specifically made to the method embodiment part, and details are not described herein again.

[0156] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

[0157] Referring to Figure 3 , an embodiment of the present invention further provides a computer device 3, including: a memory 302, a processor 301, and a computer program 303 stored on the memory 302. When the computer program 303 is executed on the processor 301, the posture control method of the intelligent handheld device as described in any one of the above methods is implemented.

[0158] The computer device 3 may be a computing device such as a desktop computer, a notebook, a handheld computer, and a cloud server. The computer device 3 may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art can understand that Figure 3 merely an example of the computer device 3, which does not constitute a limitation on the computer device 3, and may include more or fewer components than shown in the figure, or combine some components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0159] The so-called processor 301 may be a Central Processing Unit (CPU), and this processor 301 may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.

[0160] In some embodiments, the memory 302 may be an internal storage unit of the computer device 3, such as the hard disk or memory of the computer device 3. In other embodiments, the memory 302 may also be an external storage device of the computer device 3, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. equipped on the computer device 3. Further, the memory 302 may also include both the internal storage unit and the external storage device of the computer device 3. The memory 302 is used to store an operating system, application programs, a Boot Loader, data, and other programs, such as the program code of the computer program, etc. The memory 302 may also be used to temporarily store data that has been output or will be output.

[0161] The embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it implements the attitude control method of the intelligent handheld device as described in any one of the above methods.

[0162] In this embodiment, if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-mentioned embodiment methods of the present application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the photographing device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0163] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0164] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0165] In the embodiments disclosed in the present application, it should be understood that the disclosed device / terminal device and method can be implemented in other ways. For example, the device / terminal device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in an electrical, mechanical or other forms.

[0166] The unit described as a separation component may or may not be physically separated. The component displayed as a unit may or may not be a physical unit, that is, it may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Claims

1. A posture control method for an intelligent handheld game console, characterized in that, The method specifically includes: Collecting the original attitude data of the intelligent handheld device in real time according to the accelerometer, gyroscope, and magnetometer, and performing periodic calibration on the original attitude data through a dynamic zero-drift correction algorithm to obtain corrected attitude data; Performing attitude calculation and time window sampling on the corrected attitude data to form first attitude data, and inputting the first attitude data into a time series prediction algorithm for prediction calculation to obtain attitude change prediction data; Using a Bayesian filtering algorithm to fuse the visual data, ultrasonic data, lidar data, and IMU data of the intelligent handheld device to generate high-precision positioning information; Based on the attitude change prediction data, the high-precision positioning information, and the user's historical operation data, using a user behavior modeling algorithm and a reinforcement learning algorithm to dynamically generate personalized attitude control instructions; Performing deviation correction processing on the personalized attitude control instructions according to the AR visual feedback data and tactile feedback data of the intelligent handheld device; The step of performing periodic calibration on the original attitude data through a dynamic zero-drift correction algorithm to obtain corrected attitude data specifically includes: Setting zero-drift detection conditions and a calibration period; When the zero-drift condition is triggered or in each calibration period, calculating a zero-drift reference value based on the original attitude data of multiple sampling points, and performing zero-drift compensation on the zero-drift reference value through a dynamic compensation formula and a temperature compensation formula to obtain first zero-drift data; Performing attitude calculation on the first zero-drift data to obtain corrected attitude data; The step of calculating a zero-drift reference value based on the original attitude data of multiple sampling points, and performing zero-drift compensation on the zero-drift reference value through a dynamic compensation formula and a temperature compensation formula to obtain first zero-drift data specifically includes: Respectively calculating the zero-drift reference values corresponding to the accelerometer, gyroscope, and magnetometer based on the original attitude data of multiple sampling points, and the zero-drift reference value satisfies Among them, , and respectively represent the zero-drift reference values of the accelerometer, gyroscope, and magnetometer, , and respectively represent the original attitude data collected by the accelerometer, gyroscope, and magnetometer at the i-th sampling point, and N represents the number of sampling points; Zero drift compensation is performed on , and using a dynamic compensation formula to obtain second zero drift data, and the dynamic compensation formula satisfies , , , where , and respectively represent the zero drift data corresponding to the accelerometer, gyroscope, and magnetometer in the second attitude data, , and respectively represent the original attitude data collected by the accelerometer, gyroscope, and magnetometer; Use the temperature compensation formula for , and to perform zero drift compensation and obtain the first zero drift data, and the temperature compensation formula satisfies where T represents the current ambient temperature, represents the calibration temperature, , and respectively represent the zero-drift data corresponding to the accelerometer, gyroscope, and magnetometer in the first zero-drift data, , and respectively represent the zero-drift reference values corresponding to the accelerometer, gyroscope, and magnetometer at the calibration temperature , , and respectively represent the temperature drift coefficients corresponding to the accelerometer, gyroscope, and magnetometer.

2. The method according to claim 1, wherein The zero-drift detection conditions include an accelerometer static state determination condition and a gyroscope static determination condition; The determination condition for the accelerometer in the stationary state is , where , and represent the raw output values of the three axes of the accelerometer, g represents the local acceleration of gravity, represents the stationary determination threshold; The gyroscope stationary determination condition is satisfied , where , and represent the raw output values of the three axes of the gyroscope, represents the gyroscope stationary determination threshold value.

3. The method according to claim 1, wherein The step of inputting the first attitude data into a time series prediction algorithm for prediction calculation to obtain attitude change prediction data specifically includes: Normalize the first attitude data to obtain second attitude data, where the second attitude data satisfies , where represents the second attitude data,[ represents the first attitude data,[ represents the mean of Euler angles within the time window,[ represents the standard deviation of Euler angles within the time window.[ Inputting second attitude data into an LSTM prediction model, and performing training and inference through a forward propagation function to output an attitude change prediction value; Fuse the predicted attitude change value with the current attitude value to generate predicted attitude change data, where the predicted attitude change data satisfies , where represents the predicted attitude change value, represents the current attitude value, represents the predicted estimate of the attitude change at a future preset time, t represents the current time, represents the future preset time.

4. The method according to claim 1, characterized in that, The step of using a Bayesian filtering algorithm to fuse the visual data, ultrasonic data, lidar data, and IMU data of the intelligent handheld device specifically includes: Set the state vector and the observation vector , where represents the position coordinates of the intelligent handheld device in three-dimensional space, represents the velocity components of the intelligent handheld device in three-dimensional space, represents the attitude quaternion of the intelligent handheld device, respectively represent visual data, ultrasonic data, lidar data, and IMU data; Input the state vector into the state transition function to perform state prediction on the IMU drive of the intelligent handheld device, and obtain a predicted state vector, where the predicted state vector satisfies , where represents the predicted state vector,[[]] represents the state vector at the previous moment,[[]] represents the IMU data at the current moment,[[]] represents the state transition function,[[]] represents the process noise.[[]] Fuse the predicted state vector and the observation vector to obtain an updated state vector, where the updated state vector satisfies , , where represents the Kalman gain, represents the observation model for mapping the predicted state vector to the sensor observation space, represents the predicted state covariance matrix, represents the Jacobian matrix of the observation matrix, represents the observation noise covariance matrix.

5. The method according to claim 1, wherein The step of based on the attitude change prediction data, the high-precision positioning information, and the user's historical operation data, using a user behavior modeling algorithm and a reinforcement learning algorithm to dynamically generate personalized attitude control instructions specifically includes: Obtaining the user's historical operation data, where the user's historical operation data includes the user's operation acceleration, user's operation angular velocity, game scene type, and environmental complexity; Using a convolutional neural network to extract the local spatio-temporal features of the user's historical operation data, using a long short-term memory network to capture the temporal dependence features of the user's historical operation data, and fusing the local spatio-temporal features and the temporal dependence features to form a user behavior feature vector; Taking the attitude change prediction data, the high-precision positioning information, and the user behavior feature vector as the state space, and taking the proportional coefficient, integral coefficient, differential coefficient, and maximum output torque of the controller of the intelligent handheld device as the action space, with minimizing the tracking error, maximizing the operation comfort, and minimizing the energy consumption as the reward function; Based on the state space, the action space, and the reward function, a personalized attitude control instruction is dynamically generated using a reinforcement learning algorithm.

6. The method according to claim 1, characterized in that, According to the AR visual feedback data and tactile feedback data of the intelligent handheld device, the deviation correction process for the personalized attitude control instruction specifically includes: Obtain AR visual feedback data and tactile feedback data , where represents the planar deviation between the operation position of the user in the AR interface and the target position, represents the yaw angle deviation between the user's operation direction and the target direction, represents the vibration intensity of the linear motor, represents the vibration frequency; Fuse the AR visual feedback data and the haptic feedback data to obtain a comprehensive deviation vector, where the comprehensive deviation vector satisfies Among them, represents the comprehensive deviation vector, represents the visual deviation amplitude, represents the visual deviation direction, and represents the tactile intensity coefficient; Generating a PID correction coefficient based on the comprehensive deviation vector, and performing deviation correction processing on the personalized attitude control instruction according to the PID correction coefficient to obtain a corrected personalized attitude control instruction, and the corrected personalized attitude control instruction satisfies Among them, represents the parameter for correcting the personalized attitude control instruction, , and respectively represent the proportional coefficient, integral coefficient, and differential coefficient of the controller of the intelligent handheld device, , and respectively represent the correction proportional coefficient, correction integral coefficient, and correction differential coefficient in the PID correction coefficient, represents the maximum output torque, , and represent the adaptive weight coefficient.

7. An attitude control system for an intelligent handheld game console, characterized in that, For implementing the attitude control method of the intelligent handheld device according to any one of claims 1 to 6, the system specifically includes: A first attitude control module, configured to collect the original attitude data of the intelligent handheld device in real time according to an accelerometer, a gyroscope, and a magnetometer, and perform periodic calibration on the original attitude data through a dynamic zero-drift correction algorithm to obtain calibrated attitude data; A second attitude control module, configured to perform attitude calculation and time window sampling on the calibrated attitude data to form first attitude data, and input the first attitude data into a time series prediction algorithm for prediction calculation to obtain attitude change prediction data; A third attitude control module, configured to perform fusion processing on the visual data, ultrasonic data, lidar data, and IMU data of the intelligent handheld device using a Bayesian filtering algorithm to generate high-precision positioning information; A fourth attitude control module, configured to dynamically generate a personalized attitude control instruction based on the attitude change prediction data, the high-precision positioning information, and the user historical operation data using a user behavior modeling algorithm and a reinforcement learning algorithm; A fifth attitude control module, configured to perform deviation correction processing on the personalized attitude control instruction according to the AR visual feedback data and tactile feedback data of the intelligent handheld device.

8. A computer device, characterized in that, Including: A memory, a processor, and a computer program stored on the memory. When the computer program is executed on the processor, the attitude control method of the intelligent handheld device according to any one of claims 1 to 6 is implemented.

Images