Attitude control method and system of intelligent handheld game console

Abstract

The invention discloses an attitude control method and system for an intelligent handheld device, and the method specifically comprises the steps: collecting the original attitude data of the intelligent handheld device in real time, carrying out the periodic calibration of the original attitude data through a dynamic null drift correction algorithm, and obtaining the corrected attitude data; performing attitude calculation and time window sampling on the corrected attitude data to form first attitude data, and inputting the first attitude data into a time sequence prediction algorithm for prediction calculation to obtain attitude change prediction data; performing fusion processing on the visual data, the ultrasonic data, the laser radar data and the IMU data of the intelligent handheld game console to generate high-precision positioning information; and based on the attitude change prediction data, the high-precision positioning information and the user historical operation data, a user behavior modeling algorithm and a reinforcement learning algorithm are adopted to dynamically generate a personalized attitude control instruction. Accurate, dynamic and personalized control over the posture of the intelligent handheld game console is achieved, and user experience and operation precision are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent handheld game consoles, and in particular to a posture control method and system for an intelligent handheld game console. Background Art

[0002] With the rapid popularization of smart devices, handheld devices are increasingly used in gaming, virtual reality (VR), augmented reality (AR) and other fields, providing users with a more immersive and interactive experience. However, the existing handheld gesture control technology still faces a series of challenges, which affects the user experience and device performance.

[0003] First, error accumulation and delay are major challenges in handheld gesture control. When used for a long time or performing rapid and continuous actions, the accuracy of gesture control will be affected due to the inherent errors of the sensor and the system response delay. This error accumulation not only reduces the accuracy of gesture control, but may also cause system response lag, which seriously affects the user experience.

[0004] Secondly, the problem of positioning accuracy in complex environments is also a difficult problem that needs to be solved urgently. In low light, magnetic field interference or complex outdoor environments, the accuracy and stability of traditional sensors and positioning technologies will be greatly reduced. This not only limits the application of handheld game consoles in a variety of environments, but may also lead to inaccurate positioning information, thereby affecting the reliability of posture control.

[0005] Furthermore, the problem of user personalized needs is also a major shortcoming of the existing handheld gesture control algorithm. Different users have different control habits and gesture change patterns, but the existing algorithms are often difficult to adapt to the needs of all users. As a result, in some user scenarios, gesture control may not achieve the best effect, reducing user experience satisfaction.

[0006] In addition, in AR applications, handheld devices need to provide accurate visual feedback to guide user operations, but existing technologies often have difficulty maintaining the accuracy and real-time nature of visual feedback in complex environments or during rapid movement. At the same time, although tactile feedback technology can enhance the user's operational perception, how to effectively combine it with the gesture control algorithm to achieve accurate and comfortable tactile feedback is also an urgent problem to be solved. Summary of the invention

[0007] The purpose of the present invention is to provide a posture control method and system for an intelligent handheld game console, which realizes accurate, dynamic and personalized control of the posture of the intelligent handheld game console, improves user experience and operation accuracy, and solves at least one of the above-mentioned prior art problems.

[0008] In a first aspect, the present invention provides a method for controlling a posture of an intelligent handheld game console, the method specifically comprising: The original posture data of the intelligent handheld game console is collected in real time according to the accelerometer, gyroscope and magnetometer, and the original posture data is periodically calibrated by a dynamic zero drift correction algorithm to obtain corrected posture data; Performing posture calculation and time window sampling on the corrected posture data to form first posture data, and inputting the first posture data into a time series prediction algorithm for prediction calculation to obtain posture change prediction data; The Bayesian filtering algorithm is used to fuse the visual data, ultrasonic data, lidar data and IMU data of the intelligent handheld game console to generate high-precision positioning information; Based on the posture change prediction data, the high-precision positioning information and the user's historical operation data, a user behavior modeling algorithm and a reinforcement learning algorithm are used to dynamically generate personalized posture control instructions; According to the AR visual feedback data and tactile feedback data of the intelligent handheld game console, deviation correction processing is performed on the personalized posture control instruction.

[0009] In a second aspect, the present invention provides a posture control system for an intelligent handheld game console, the system specifically comprising: The first attitude control module is used to collect the original attitude data of the intelligent handheld game console in real time according to the accelerometer, gyroscope and magnetometer, and periodically calibrate the original attitude data through a dynamic zero drift correction algorithm to obtain corrected attitude data; A second posture control module is used to perform posture calculation and time window sampling on the corrected posture data to form first posture data, and input the first posture data into a time series prediction algorithm for prediction calculation to obtain posture change prediction data; The third attitude control module is used to use the Bayesian filtering algorithm to fuse the visual data, ultrasonic data, laser radar data and IMU data of the intelligent handheld game console to generate high-precision positioning information; A fourth posture control module, configured to dynamically generate personalized posture control instructions based on the posture change prediction data, the high-precision positioning information and the user's historical operation data, using a user behavior modeling algorithm and a reinforcement learning algorithm; The fifth posture control module is used to perform deviation correction processing on the personalized posture control instruction according to the AR visual feedback data and tactile feedback data of the intelligent handheld game console.

[0010] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor and a computer program stored in the memory, and when the computer program is executed on the processor, a gesture control method for an intelligent handheld game console as described in any one of the above methods is implemented.

[0011] Fourthly, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it implements the attitude control method of the intelligent handheld device as described in any one of the above methods.

[0012] Compared with the prior art, the present invention has at least one of the following technical effects: 1. By integrating multiple sensor data and combining technologies such as time series prediction, Bayesian filtering, user behavior modeling, and reinforcement learning, the present invention realizes precise, dynamic, and personalized control of the attitude of the intelligent handheld device, improving the user experience and operation accuracy.

[0013] 2. By periodically calibrating the original attitude data, the present invention effectively reduces the influence of sensor zero drift on attitude measurement, improving the accuracy and stability of attitude data.

[0014] 3. By setting the determination conditions for the static state of the accelerometer and the static determination conditions for the gyroscope, the present invention can accurately determine whether the sensor is in a static state, thereby triggering zero drift correction and avoiding inaccurate data caused by mis-triggering correction in a dynamic environment.

[0015] 4. By combining dynamic compensation and temperature compensation to compensate the zero drift reference value, the present invention further improves the accuracy of attitude data. Especially in different environmental temperatures, it can maintain stable measurement performance.

[0016] 5. By performing standardization processing on the first attitude data and training and inferring the LSTM prediction model, the present invention can accurately predict future attitude changes, providing forward-looking data support for attitude control.

[0017] 6. By fusing multiple sensor data through the Bayesian filtering algorithm, the present invention generates high-precision positioning information, improving the spatial positioning accuracy and robustness of the intelligent handheld device.

[0018] 7. By combining user historical operation data, attitude change prediction data, and high-precision positioning information, and adopting user behavior modeling and reinforcement learning algorithms, the present invention can dynamically generate personalized attitude control instructions that conform to user operation habits and game scenarios, improving the comfort and response speed of operations.

[0019] 8. By obtaining AR visual feedback data and tactile feedback data, performing fusion processing and generating a comprehensive deviation vector, and correcting the deviation of the personalized attitude control instruction based on this vector, the present invention further improves the accuracy of attitude control and the user experience. At the same time, the introduction of an adaptive weight coefficient makes the correction process more flexible and accurate. Description of the Drawings

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 It is a flow chart of a posture control method of an intelligent handheld game console provided by an embodiment of the present invention; Figure 2 It is a structural schematic diagram of a posture control system of an intelligent handheld game console provided by an embodiment of the present invention; Figure 3 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0023] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0024] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0025] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0026] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0027] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0028] In the embodiment of the present application, the execution subject of the process includes a terminal device, which includes but is not limited to: a server, a computer, a smart phone, a tablet computer, and other devices capable of executing the method disclosed in the present application. Figure 1 A flow chart of a method for controlling the posture of an intelligent handheld game console disclosed in an embodiment of the present invention is shown, and the details are as follows: S101, collecting original posture data of the intelligent handheld game console in real time according to the accelerometer, gyroscope and magnetometer, and periodically calibrating the original posture data through a dynamic zero drift correction algorithm to obtain corrected posture data.

[0029] In this embodiment, the intelligent handheld game console has built-in accelerometers, gyroscopes and magnetometers. These sensors can collect the posture data of the handheld game console in three-dimensional space in real time, including acceleration, angular velocity and magnetic field direction. Among them, the accelerometer is responsible for collecting the acceleration data of the handheld game console in the X-axis, Y-axis and Z-axis directions, the gyroscope is responsible for collecting the rotation angular velocity data of the handheld game console around the X-axis, Y-axis and Z-axis, and the magnetometer is responsible for collecting the magnetic field direction data of the environment in which the handheld game console is located, which is used to assist in determining the absolute direction of the handheld game console. The dynamic zero drift correction algorithm is applied to periodically calibrate the original posture data. The algorithm is based on the principle of automatic compensation of zero bias voltage, assuming that the zero bias voltage of the sensor is a slowly changing signal, and using the zero drift voltage of the previous moment to offset the zero bias voltage of the adjacent next moment, thereby realizing automatic compensation of zero drift. The frequency of periodic calibration can be adjusted according to the use environment and accuracy requirements of the handheld game console. For example, in a scene with a stable environment and high accuracy requirements, a higher calibration frequency can be set; while in a scene with a complex environment and low accuracy requirements, a lower calibration frequency can be set. The corrected posture data processed by the dynamic zero drift correction algorithm is output to the handheld console's operating system or related applications. These data can be used to achieve more accurate gesture recognition, game control, navigation positioning and other functions.

[0030] In this embodiment, the original gesture data is periodically calibrated by a dynamic zero drift correction algorithm, which can effectively eliminate the influence of the sensor zero drift, thereby improving the accuracy of the gesture data, helping to achieve more accurate gesture recognition and game control functions, and improving the user experience. Periodic calibration can ensure that the sensor can maintain stable performance output in different environments, and help the intelligent handheld game console maintain stable gesture recognition capabilities in various complex scenarios. By reducing the influence of sensor zero drift, the wear and failure rate of the sensor can be reduced, which helps to extend the service life of the intelligent handheld game console and reduce the user's maintenance costs.

[0031] S102, performing posture calculation and time window sampling on the corrected posture data to form first posture data, and inputting the first posture data into a time series prediction algorithm for prediction calculation to obtain posture change prediction data.

[0032] In this embodiment, the corrected posture data is first processed by a posture calculation algorithm. The posture calculation algorithm uses the data fusion technology of the accelerometer, gyroscope and magnetometer, combined with the complementarity between sensors, to accurately calculate the real-time posture of the handheld game console. In the posture calculation process, the noise and error of the sensor, as well as the cross-interference between them, need to be considered. Through appropriate filtering algorithms and fusion algorithms, the accuracy and stability of posture calculation can be effectively improved.

[0033] The data after posture solution is sampled in time window. Time window sampling is a commonly used signal processing technology, which extracts the characteristics of the signal in different time periods by segmenting the continuous signal. In this embodiment, time window sampling is used to extract posture data within a certain time period from the posture solution data to form the first posture data. The size of the time window can be adjusted according to the actual application requirements to balance the prediction accuracy and calculation efficiency.

[0034] The first posture data is input into a time series prediction algorithm for prediction calculation. The time series prediction algorithm is a mathematical model that can use historical data to predict future data. In this embodiment, a suitable time series prediction algorithm can be selected, such as an autoregressive moving average (ARMA) model, an autoregressive integrated moving average (ARIMA) model, a recurrent neural network (RNN), etc. These algorithms can capture the time dependency and trend in the posture data, thereby achieving accurate prediction of future posture changes.

[0035] The time series prediction algorithm outputs posture change prediction data. These data represent the predicted value of the handheld posture in the future. The posture change prediction data can be used in a variety of application scenarios, such as pre-judgment in game control, posture prediction in virtual reality, posture tracking in motion analysis, etc.

[0036] In this embodiment, more accurate and stable posture data can be extracted through posture solution and time window sampling, providing high-quality historical data for the time series prediction algorithm, which helps to improve the accuracy of the prediction algorithm and achieve more accurate prediction of future posture changes. Time window sampling enables the algorithm to process continuously changing posture data and output prediction results in real time, which helps to enhance the real-time performance of the system, allowing the handheld game console to respond quickly to posture changes and improve user experience. By selecting the appropriate time window size and prediction algorithm, while ensuring prediction accuracy, the calculation efficiency can be optimized, which helps to reduce the calculation burden of the system and improve the overall performance of the system.

[0037] S103, using the Bayesian filtering algorithm to fuse the visual data, ultrasonic data, lidar data and IMU data of the intelligent handheld game console to generate high-precision positioning information.

[0038] In this embodiment, the intelligent handheld game console is equipped with visual sensors (such as cameras), ultrasonic sensors, lidar sensors and IMU sensors. These sensors collect information about the environment around the handheld game console in real time, including images, distances, three-dimensional point clouds and acceleration / angular velocity, etc.

[0039] The visual data, ultrasonic data, lidar data and IMU data are input into the Bayesian filtering algorithm. The Bayesian filtering algorithm is based on the Bayesian theorem and combines prior knowledge and observation data to recursively update the posterior probability distribution of the system state. In this embodiment, the system state includes the position and posture of the handheld game console, and the observation data comes from various sensors. The Bayesian filtering algorithm generates more accurate and stable positioning information by fusing data from different sensors. After fusion processing by the Bayesian filtering algorithm, high-precision positioning information is output, which includes the three-dimensional position, posture (such as pitch angle, yaw angle and roll angle) and movement speed of the handheld game console.

[0040] In this embodiment, the Bayesian filtering algorithm can fuse data from different sensors, making full use of the complementarity of various sensors. Through fusion processing, the error and uncertainty of a single sensor can be eliminated, thereby improving positioning accuracy. The Bayesian filtering algorithm has good robustness in processing noise and outliers. In a complex environment, a single sensor may be interfered with or blocked, resulting in inaccurate data. However, the Bayesian filtering algorithm can reduce this impact by fusing data from multiple sensors to ensure the stability of positioning information.

[0041] S104, based on the posture change prediction data, the high-precision positioning information and the user's historical operation data, a user behavior modeling algorithm and a reinforcement learning algorithm are used to dynamically generate personalized posture control instructions.

[0042] In this embodiment, the user's operating habits, preferences, and behavior patterns in common scenarios when using the device in the past are recorded to form user historical operation data. The user's historical operation data is modeled using a hidden Markov model (HMM) or other user behavior modeling algorithms. The user's behavior characteristics in different scenarios are analyzed to identify the user's operating habits and preferences. A user behavior model is established to predict the user's possible behavior in the future.

[0043] Define the state space, action space, and reward function of reinforcement learning. The state space includes information such as the current posture, position, and speed. The action space includes a set of posture control instructions. The reward function is designed based on factors such as the user behavior model and task objectives to evaluate the pros and cons of executing a certain action. Select a suitable reinforcement learning algorithm (such as Q-learning, deep Q network DQN, proximal policy optimization PPO, etc.) for training. Using posture change prediction data and high-precision positioning information as input, the reinforcement learning algorithm continuously optimizes the posture control instructions to make them more in line with user expectations and task objectives. Based on the user behavior model and the output of the reinforcement learning algorithm, dynamically generate posture control instructions that meet user habits and preferences. In actual applications, adjust posture control instructions based on real-time sensor data and user behavior predictions to achieve a more personalized control experience.

[0044] In this embodiment, through user behavior modeling and reinforcement learning algorithms, the generated gesture control instructions are more in line with the user's operating habits and preferences, thereby improving the user experience. The dynamically generated gesture control instructions can be adjusted according to different scenarios and user behaviors, enhancing the adaptability of the device to different environments and users.

[0045] S105, performing deviation correction processing on the personalized gesture control instruction according to the AR visual feedback data and tactile feedback data of the intelligent handheld game console.

[0046] In this embodiment, the intelligent handheld game console is equipped with an AR visual sensor and a tactile sensor, which can obtain the user's visual feedback data and tactile feedback data in real time. The AR visual feedback data is used to detect the visual deviation of the user when interacting with the handheld game console in real time. The difference between the actual action and the expected action of the user in the AR environment is analyzed through image recognition and target tracking technology. According to the size and direction of the visual deviation, the personalized posture control instruction is fine-tuned to more accurately reflect the user's intention. The tactile sensor can sense the force and direction when the user contacts the handheld game console. By analyzing the tactile feedback data, it is determined whether the user encounters resistance or discomfort in the actual operation. If it is detected that the tactile feedback is inconsistent with expectations, such as excessive force or deviation in direction, the personalized posture control instruction is further corrected to ensure the comfort and accuracy of the operation. The results of the correction of the AR visual feedback data and the correction of the tactile feedback data are combined to form the final personalized posture control instruction. The corrected instruction is output to the control system of the handheld game console to achieve more precise and humanized posture control.

[0047] In this embodiment, by combining AR visual feedback data and tactile feedback data, the user's intention and operation deviation can be more accurately identified, thereby improving the accuracy of the operation. The corrected personalized gesture control instructions are more in line with the user's actual needs and operating habits, thereby enhancing the comfort and satisfaction of the user experience in AR applications.

[0048] In some embodiments, in the above step S101, periodically calibrating the original posture data by a dynamic zero drift correction algorithm to obtain corrected posture data specifically includes: Set zero drift detection conditions and calibration cycle; When the zero drift condition is triggered or in each calibration cycle, a zero drift reference value is calculated based on the original attitude data of multiple sampling points, and the zero drift reference value is compensated for by a dynamic compensation formula and a temperature compensation formula to obtain the first zero drift data; The first zero drift data is subjected to attitude calculation to obtain corrected attitude data.

[0049] In this embodiment, a threshold is set, and when the sensor has no signal input (such as when the attitude sensor is stationary), the fluctuation of its output value exceeds the threshold, the zero drift detection condition is triggered. For example, for a six-axis attitude sensor, it can be set that when the raw data of the three axes of the gyroscope fluctuates beyond a certain range (such as ±0.05° / s) in a stationary state, the zero drift detection is triggered.

[0050] According to the characteristics of the sensor and the use environment, set a fixed time interval as the calibration cycle. For example, it can be set to calibrate once a day, or once every 100 working hours. The setting of the calibration cycle should ensure that the sensor maintains accuracy and stability during long-term use.

[0051] When the zero drift detection condition is triggered or in each calibration cycle, the zero drift reference value is calculated by collecting the raw attitude data of multiple sampling points. The number of sampling points can be set as needed, for example, collecting 100 or more sampling point data. The collected raw data is processed, such as removing outliers, calculating the average value, etc., to obtain the zero drift reference value. This reference value represents the offset of the sensor when there is no signal input.

[0052] According to the dynamic characteristics of the sensor, a dynamic compensation formula is designed to correct the zero drift reference value. This formula can be dynamically adjusted based on the historical data of the sensor, the current working environment and other factors. Considering the impact of temperature changes on sensor performance, a temperature compensation formula is designed to further correct the zero drift reference value. This formula can calculate the compensation amount based on the difference between the current temperature and the standard temperature. The results of dynamic compensation and temperature compensation are added to obtain the first zero drift data, which has made a preliminary correction to the zero drift. The first zero drift data is input into the attitude solution algorithm for attitude calculation. This algorithm can calculate the real-time attitude of the object based on the output data of the sensor. In the attitude solution process, the attitude is corrected using the corrected zero drift data to obtain more accurate corrected attitude data.

[0053] In this embodiment, the zero position of the sensor can be tracked and corrected in real time through zero drift detection and compensation technology to ensure the accuracy and stability of the measurement results, which is particularly important for situations requiring high-precision attitude control. Regular calibration and dynamic compensation can adapt to changes in sensor performance and the impact of the working environment, and enhance the robustness and reliability of the system. Even if the sensor has a certain degree of performance degradation or environmental changes during use, the system can maintain good performance.

[0054] Further, the zero drift detection condition includes an accelerometer static state determination condition and a gyroscope refinement determination condition; The accelerometer static state determination condition is: ,in, , and Represents the three-axis original output value of the accelerometer, g represents the local gravity acceleration, represents the stillness determination threshold; The gyroscope refinement determination condition satisfies ,in, , and Represents the three-axis raw output value of the gyroscope, Indicates the gyroscope stillness determination threshold.

[0055] In this embodiment, , and Represents the raw output values ​​of the accelerometer in the X, Y, and Z axes respectively. An accelerometer is a sensor that measures acceleration force. It is usually used for motion detection and device navigation. In a stationary state, the accelerometer is mainly affected by gravity, so its output value should be related to the local gravity acceleration g. g represents the local gravity acceleration, which is a constant and its value depends on the position and altitude of the earth's surface. In most areas, the value of g is about 9.8m / s². Stationary determination threshold It is a preset threshold used to determine whether the accelerometer is in a stationary state. When the difference between the synthetic acceleration of the three-axis original output value of the accelerometer and g is within the stationary determination threshold, the accelerometer can be considered to be in a stationary state. By setting the stationary determination threshold, it is possible to accurately determine whether the accelerometer is in a stationary state, thereby providing a reliable basis for subsequent zero drift detection. When the accelerometer is in a stationary state, its output value should mainly reflect the acceleration of gravity. At this time, performing zero drift detection can more accurately measure the zero offset of the sensor.

[0056] , and Represents the original output values ​​of the gyroscope in the three axes of X, Y, and Z. The gyroscope is a sensor used to measure angular velocity. It can detect the angular velocity of an object around a certain axis. Gyroscope static judgment threshold It is a preset threshold used to determine whether the gyroscope is in a stationary state. When the three-axis raw output values ​​of the gyroscope are all less than the gyroscope stationary determination threshold, the gyroscope can be considered to be in a stationary state. By setting the gyroscope stationary determination threshold, it is possible to accurately determine whether the gyroscope is in a stationary state, thereby avoiding errors caused by zero drift detection in a dynamic environment. When the gyroscope is in a stationary state, its output value should be close to zero. At this time, performing zero drift detection can more accurately measure the zero offset of the sensor, which helps to improve the measurement accuracy of the sensor.

[0057] In this embodiment, the parameters in the accelerometer static state determination condition and the gyroscope static state determination condition play a vital role in zero drift detection. By reasonably setting these parameters, it is possible to accurately determine whether the sensor is in a static state, thereby providing a reliable basis for subsequent zero drift detection and compensation, and helping to improve the measurement accuracy and stability of the sensor.

[0058] Further, the method of calculating a zero drift reference value according to the original posture data of multiple sampling points, and performing zero drift compensation on the zero drift reference value through a dynamic compensation formula and a temperature compensation formula to obtain the first zero drift data specifically includes: The zero drift reference values ​​corresponding to the accelerometer, gyroscope and magnetometer are calculated respectively according to the original attitude data of multiple sampling points, and the zero drift reference values ​​meet in, , and Represent the zero drift reference values ​​of the accelerometer, gyroscope and magnetometer respectively, , and They represent the original attitude data collected by the accelerometer, gyroscope, and magnetometer at the i-th sampling point, respectively, and N represents the number of sampling points; Using dynamic compensation formula , and Perform zero drift compensation to obtain the second zero drift data. The dynamic compensation formula satisfies , , ,in, , and Respectively represent the zero drift data corresponding to the accelerometer, gyroscope and magnetometer in the second attitude data, , and Respectively represent the raw attitude data collected by the accelerometer, gyroscope, and magnetometer; Using the temperature compensation formula , and Perform zero drift compensation to obtain the first zero drift data, and the temperature compensation formula satisfies Where T represents the current ambient temperature, Indicates the calibration temperature, , and Respectively represent the zero drift data corresponding to the accelerometer, gyroscope and magnetometer in the first zero drift data, , and Respectively indicate the calibration temperature The corresponding zero drift reference values ​​of the accelerometer, gyroscope and magnetometer are as follows: , and They represent the temperature drift coefficients of the accelerometer, gyroscope and magnetometer respectively.

[0059] In this embodiment, the zero drift reference values ​​of the accelerometer, gyroscope, and magnetometer are , and It is calculated by averaging or other statistical methods the original attitude data of multiple sampling points, representing the zero offset of the sensor in a static or standard state. These reference values ​​are used for subsequent zero drift compensation to eliminate the zero drift error of the sensor.

[0060] The original posture data of the i-th sampling point , and It is collected by the sensor during the actual measurement process, and contains the real attitude information and possible zero offset. Through the data of multiple sampling points, the zero drift reference value can be calculated more accurately.

[0061] The number of sampling points N determines the accuracy and stability of the zero drift reference value calculation. The more sampling points there are, the closer the calculated reference value is to the true value, but the amount of calculation will also increase accordingly.

[0062] The zero drift data corresponding to the accelerometer, gyroscope and magnetometer in the second attitude data , and Used to correct the zero point offset in the original posture data.

[0063] The current ambient temperature T is one of the important factors affecting the performance of the sensor. By measuring the current ambient temperature, the zero drift caused by temperature changes can be calculated.

[0064] Calibration temperature It is the reference temperature set by the sensor when it leaves the factory or during calibration. At the calibration temperature, the zero offset of the sensor is accurately measured and recorded.

[0065] Temperature drift coefficient , and Indicates the rate at which the sensor zero drift changes with temperature. By measuring the zero drift reference value at the calibration temperature and the zero drift data at the current ambient temperature, the temperature drift coefficient can be calculated and used for subsequent temperature compensation.

[0066] In this embodiment, by calculating the zero drift reference value and performing dynamic and temperature compensation, the zero drift error of the sensor can be significantly reduced, thereby improving the measurement accuracy, which is particularly important for application scenarios that require high-precision attitude measurement. Dynamic compensation and temperature compensation can adapt to changes in sensor performance and the influence of the working environment, so that the system can maintain good performance under various conditions, which helps to enhance the robustness and reliability of the system.

[0067] In some embodiments, in the above step S102, the step of inputting the first posture data into a time series prediction algorithm for prediction calculation to obtain posture change prediction data specifically includes: The first posture data is standardized to obtain second posture data, and the second posture data satisfies ,in, represents the second posture data, represents the first posture data, represents the mean value of the Euler angle in the time window, Represents the standard deviation of the Euler angles within the time window; The second posture data is input into the LSTM prediction model, and training reasoning is performed through the forward propagation function to output the posture change prediction value; The posture change prediction value is merged with the current posture value to generate posture change prediction data, and the posture change prediction data satisfies ,in, represents the predicted value of posture change, Indicates the current attitude value, represents the predicted estimate of the posture change at a preset time in the future, t represents the current time, Indicates a preset time in the future.

[0068] In this embodiment, the Euler angle is a set of angle values ​​that describe the rotation posture of an object in three-dimensional space. It is used to measure the average level of posture data over a period of time, which helps to remove the overall offset in the data during the normalization process. Reflects the degree of discreteness of the data. In the standardization process, the standard deviation can be used to scale the original data to a standard range, usually between 0 and 1, which helps to improve the convergence speed and prediction accuracy of the model. The second posture data is the data after standardization, with a unified dimension and range, and is more suitable as the input of the LSTM prediction model. Posture change prediction value It is the output of the LSTM prediction model, which indicates the prediction of posture changes in the future. That is, the posture data at the current moment is used to merge with the posture change prediction value to generate the final posture change prediction data. The posture change prediction estimate at the future preset moment It is the result of fusing the current posture value and the predicted posture change value, which represents the estimation of the posture change at a specific moment in the future. Specifies the specific time point at which the posture change prediction is performed.

[0069] In this embodiment, the original posture data is converted into data of uniform dimension and range through standardization, which helps to accelerate the convergence speed of the LSTM prediction model and improve the training efficiency. Standardization eliminates the overall offset and dimensional differences in the original data, so that the LSTM prediction model can more accurately capture the long-term dependencies in the sequence data, thereby improving the accuracy of posture prediction. By fusing the posture change prediction value with the current posture value to generate posture change prediction data, both the posture information at the current moment and the prediction of posture changes in the future period of time are considered, making the generated prediction data more comprehensive and accurate. The LSTM prediction model can capture the long-term dependencies in the sequence data and has good adaptability to the posture change prediction in complex environments. For example, in a dynamic or uncertain environment, the LSTM model can predict future posture changes based on historical data, providing strong support for tasks such as posture control and navigation.

[0070] In some embodiments, in the above step S103, the Bayesian filtering algorithm is used to fuse the visual data, ultrasonic data, laser radar data and IMU data of the intelligent handheld game console, specifically including: Setting the state vector and the observation vector ,in, Indicates the position coordinates of the intelligent handheld device in three-dimensional space. Represents the velocity component of the intelligent handheld device in three-dimensional space, The quaternion representing the attitude of the intelligent handheld game console. Represent visual data, ultrasonic data, lidar data and IMU data respectively; The state vector Input into the state transfer function, perform state prediction on the IMU drive of the intelligent handheld game console, and obtain a predicted state vector, which satisfies ,in, represents the predicted state vector, represents the state vector at the previous moment, Represents the IMU data at the current moment, represents the state transition function, represents process noise; The predicted state vector and the observed vector are fused to obtain an updated state vector, which satisfies , ,in, represents the Kalman gain, represents the observation model used to map the predicted state vector to the sensor observation space, represents the predicted state covariance matrix, represents the Jacobian matrix of the observation matrix, represents the observation noise covariance matrix.

[0071] In this embodiment, the position coordinates It indicates the absolute position of the intelligent handheld device in three-dimensional space and is the core parameter of the navigation and positioning system.

[0072] Velocity Component Describing the speed of an intelligent handheld device's movement in three-dimensional space is crucial for predicting its future position.

[0073] Attitude Quaternion Quaternions are used to represent the spatial posture of an intelligent handheld device, that is, the rotation angle and direction relative to a reference coordinate system. Quaternions are more suitable for interpolation and numerical calculations than Euler angles or rotation matrices because they can avoid problems such as universal joint lock.

[0074] Visual Data , Ultrasonic data , LiDAR data , IMU data They come from different sensors and provide direct or indirect measurements of the position and posture of the intelligent handheld device. Visual data may include image feature matching results; ultrasonic data provides distance measurements; lidar data provides a three-dimensional point cloud of the environment; and IMU data includes measurements from accelerometers and gyroscopes, providing information about acceleration and angular velocity.

[0075] State transfer function The current state vector is predicted based on the state vector at the previous moment and the IMU data at the current moment. It models the movement of the intelligent handheld game console based on the laws of physics (such as Newton's second law) and the measurement model of the IMU.

[0076] Predicted state vector Represents the state vector at the current moment predicted by the state transfer function. It takes into account the direct measurement of IMU data and the influence of process noise.

[0077] Kalman Gain It is a key parameter in Bayesian filtering (especially Kalman filtering), which balances the trust between the predicted state vector and the observation vector. It is calculated based on the predicted state covariance matrix, the Jacobian matrix of the observation matrix, and the observation noise covariance matrix.

[0078] Observation Model The predicted state vector is mapped to the sensor observation space for comparison with the observation vector. It takes into account the sensor's measurement characteristics and errors.

[0079] Predicted state covariance matrix , the Jacobian matrix of the observation matrix , observation noise covariance matrix They respectively describe the uncertainty of the predicted state, the linear approximation of the state vector by the observation model, and the statistical characteristics of the observation noise. They play a key role in calculating the Kalman gain.

[0080] In this embodiment, by fusing data from different sensors, the Bayesian filtering algorithm can fully utilize the advantages of each sensor, reduce the error and uncertainty caused by a single sensor, and thus improve the positioning accuracy of the intelligent handheld game console. In a complex or dynamic environment, a single sensor may be interfered with or fail. By fusing data from multiple sensors, the system can tolerate the failure of some sensors or data anomalies, maintaining overall stability and reliability.

[0081] In some embodiments, in the above step S104, the personalized posture control instructions are dynamically generated based on the posture change prediction data, the high-precision positioning information and the user's historical operation data using a user behavior modeling algorithm and a reinforcement learning algorithm, specifically including: Acquire user historical operation data, wherein the user historical operation data includes user operation acceleration, user operation angular velocity, game scene type and environment complexity; A convolutional neural network is used to extract local spatiotemporal features of the user's historical operation data, a long short-term memory network is used to capture the temporal dependency features of the user's historical operation data, and the local spatiotemporal features and the temporal features are fused to form a user behavior feature vector; The posture change prediction data, the high-precision positioning information and the user behavior feature vector are used as the state space, the proportional coefficient, integral coefficient and differential coefficient of the controller of the intelligent handheld game console and the maximum output torque are used as the action space, and minimizing the tracking error, maximizing the operating comfort and minimizing the energy consumption are used as the reward function; Based on the state space, the action space and the reward function, a reinforcement learning algorithm is used to dynamically generate personalized posture control instructions.

[0082] In this embodiment, the built-in sensors (such as accelerometers and gyroscopes) of the intelligent handheld game console and the game system logs are used to collect the user's operation data in different game scenarios, including user operation acceleration, user operation angular velocity, game scene type (such as racing, shooting, adventure, etc.) and environmental complexity (such as the number of obstacles, enemy density, etc.). The collected data is cleaned, denoised and normalized to ensure the quality and consistency of the data. At the same time, the data is classified and marked according to the game scene type and environmental complexity.

[0083] Convolutional neural network (CNN) is used to extract local spatiotemporal features of user historical operation data. It takes user operation acceleration, user operation angular velocity and other data as input, and extracts local spatiotemporal patterns in the data, such as user operation habits, reaction speed, etc., through multiple convolutional layers and pooling layers.

[0084] Long short-term memory (LSTM) is used to capture the temporal dependency features of user historical operation data. The feature sequence extracted by CNN is used as the input of LSTM, and the long-term dependency in the data, such as the user's continuous operation behavior and strategy changes, is captured through the LSTM loop structure.

[0085] The local spatiotemporal features extracted by CNN and the temporal dependency features captured by LSTM are fused to form a user behavior feature vector. This vector integrates information such as the user's operating habits, reaction speed, continuous operating behavior, and strategy changes, and can fully reflect the user's gaming behavior characteristics.

[0086] The state space includes posture change prediction data (such as the posture angle and angular velocity of the intelligent handheld game console), high-precision positioning information (such as GPS coordinates, map information, etc.) and user behavior feature vectors. These state information together describe the current state of the intelligent handheld game console and the user's behavior characteristics. The action space includes the proportional coefficient, integral coefficient and differential coefficient (i.e., the parameters of the PID controller) of the controller of the intelligent handheld game console and the maximum output torque. These action parameters determine the posture control instructions of the intelligent handheld game console. The reward function includes three parts: minimizing the tracking error (i.e., the difference between the actual posture of the intelligent handheld game console and the expected posture), maximizing the operating comfort (such as reducing the discomfort caused by sudden acceleration or deceleration) and minimizing energy consumption (such as reducing the energy consumption of the motor). By adjusting the weight of the reward function, these three goals can be balanced to achieve personalized posture control.

[0087] Reinforcement learning algorithms (such as the deep deterministic policy gradient algorithm DDPG) are used to dynamically generate personalized posture control instructions. The algorithm selects the optimal action space parameters based on the current state space, with the goal of maximizing the cumulative reward. Through continuous iteration and optimization, the algorithm can learn a posture control strategy that suits the user's behavior characteristics.

[0088] In this embodiment, the user behavior feature vector is formed by extracting the features of the user's historical operation data and fusing them, thereby realizing the generation of personalized posture control instructions. This makes the posture control of the intelligent handheld game console more in line with the user's operating habits and expectations. By designing a reward function that minimizes tracking errors, maximizes operating comfort, and minimizes energy consumption, the game's fluency and comfort are improved, and the user's gaming experience is enhanced. The reinforcement learning algorithm can dynamically adjust the posture control strategy according to the user's behavioral characteristics and changes in the game scene, realizing adaptive posture control.

[0089] In some embodiments, in the above step S105, the deviation correction processing of the personalized gesture control instruction according to the AR visual feedback data and tactile feedback data of the intelligent handheld game console specifically includes: Get AR visual feedback data and tactile feedback data ,in, Indicates the plane deviation between the user's operation position and the target position in the AR interface. Indicates the yaw angle deviation between the user's operating direction and the target direction. Indicates the vibration intensity of the linear motor. Indicates vibration frequency; The AR visual feedback data The tactile feedback data is fused and processed to obtain a comprehensive deviation vector, and the comprehensive deviation vector satisfies in, represents the comprehensive deviation vector, represents the visual deviation amplitude, Indicates the direction of visual deviation, and represents the tactile intensity coefficient; A PID correction coefficient is generated based on the comprehensive deviation vector, and a deviation correction process is performed on the personalized posture control instruction according to the PID correction coefficient to obtain a corrected personalized posture control instruction, wherein the corrected personalized posture control instruction satisfies in, Indicates the parameters for correcting the personalized attitude control instructions. , and They represent the proportional coefficient, integral coefficient and differential coefficient of the controller of the intelligent handheld game console respectively. , and They represent the correction proportional coefficient, correction integral coefficient and correction differential coefficient in the PID correction coefficient respectively. Indicates the maximum output torque, , and Represents the adaptive weight coefficient.

[0090] In this embodiment, the plane deviation between the user's operating position in the AR interface and the target position It reflects the horizontal distance difference between the user's operating position and the expected target position when the user operates in the AR interface. By capturing and analyzing this deviation, the system can understand the accuracy of the user's operation and make adjustments accordingly.

[0091] The yaw angle deviation between the user's operating direction and the target direction Indicates the angular difference between the user's operating direction and the target direction. It helps the system determine whether the user's operating direction is accurate, thereby providing necessary directional adjustments.

[0092] Linear motor vibration intensity Indicates the vibration intensity generated by a tactile feedback device (such as a linear motor). By adjusting the vibration intensity, the system can provide different levels of tactile feedback to the user to simulate different operating effects or remind the user to pay attention.

[0093] Vibration frequency Determines the rhythm and speed of tactile feedback. Different vibration frequencies can convey different information or effects. For example, high-frequency vibrations may indicate urgent or strong feedback, while low-frequency vibrations may indicate gentle or continuous reminders.

[0094] Visual deviation amplitude Indicates the visual deviation between the user's operation and the target, and the direction of the visual deviation Indicates the deviation direction between the user's operation and the target visually. and The tactile feedback data is combined with the visual feedback data to form a comprehensive deviation representation. This coefficient reflects the contribution of the tactile feedback in the comprehensive deviation vector, which helps the system to more comprehensively understand the status of the user's operation.

[0095] Correction scale factor , Correction integral coefficient and the modified differential coefficient They correspond to the proportional, integral and differential parts of the PID controller respectively. By adjusting these three coefficients, the system can achieve precise control of user operations to reduce deviations and maintain stability.

[0096] Proportional coefficient of controller of intelligent handheld game console , integral coefficient and the differential coefficient It is the basic control parameter of the intelligent handheld controller and is used to achieve preliminary control of user operations.

[0097] Maximum output torque Indicates the maximum force that the intelligent handheld can generate when executing control commands. This parameter helps ensure that the system has enough power to perform necessary adjustments.

[0098] Adaptive weight coefficient , and It is used to adjust the contribution of PID correction coefficient in correcting personalized attitude control instructions. By adaptively adjusting these coefficients, the system can flexibly adjust the control strategy according to the actual situation to obtain better control effect.

[0099] In this embodiment, by capturing and analyzing the user's operation position and direction deviation in the AR interface, the system can understand the status of the user's operation in real time and provide necessary adjustment prompts, which helps the user to complete the operation task more accurately and improve the operation efficiency. Combined with tactile feedback data, the system can provide users with a more intuitive and realistic operation experience. By adjusting the vibration intensity and frequency, the system can simulate different operation effects or remind users to pay attention, thereby enhancing the user's immersion and satisfaction. By performing deviation correction processing on personalized posture control instructions through PID correction coefficients, the system can achieve precise control of user operations, help reduce deviations and maintain stability, and improve the overall control performance of the system.

[0100] Reference Figure 2 An embodiment of the present invention provides a posture control system 2 for an intelligent handheld game console, and the system 2 specifically includes: The first posture control module 201 is used to collect the original posture data of the intelligent handheld game console in real time according to the accelerometer, gyroscope and magnetometer, and periodically calibrate the original posture data through a dynamic zero drift correction algorithm to obtain corrected posture data; The second posture control module 202 is used to perform posture calculation and time window sampling on the corrected posture data to form first posture data, and input the first posture data into a time series prediction algorithm for prediction calculation to obtain posture change prediction data; The third attitude control module 203 is used to use the Bayesian filtering algorithm to fuse the visual data, ultrasonic data, laser radar data and IMU data of the intelligent handheld device to generate high-precision positioning information; A fourth posture control module 204, configured to dynamically generate personalized posture control instructions based on the posture change prediction data, the high-precision positioning information and the user's historical operation data, using a user behavior modeling algorithm and a reinforcement learning algorithm; The fifth posture control module 205 is used to perform deviation correction processing on the personalized posture control instruction according to the AR visual feedback data and tactile feedback data of the intelligent handheld game console.

[0101] It is understandable that if Figure 1 The contents of the embodiment of the posture control method of the intelligent handheld game console shown in the figure are applicable to the embodiment of the posture control system of the present intelligent handheld game console. The functions specifically implemented by the embodiment of the posture control system of the present intelligent handheld game console are similar to those of the embodiment of the posture control method of the present intelligent handheld game console shown in the figure. Figure 1 The gesture control method embodiment of the intelligent handheld game console shown in the figure is the same as that of the embodiment of the intelligent handheld game console shown in the figure, and the beneficial effects achieved are the same as those of the embodiment of the intelligent handheld game console shown in the figure. Figure 1 The beneficial effects achieved by the embodiment of the gesture control method for the intelligent handheld game console shown are also the same.

[0102] It should be noted that the information interaction, execution process and other contents between the above-mentioned systems are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0103] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0104] Reference Figure 3 The embodiment of the present invention further provides a computer device 3, comprising: a memory 302 and a processor 301 and a computer program 303 stored in the memory 302. When the computer program 303 is executed on the processor 301, the posture control method of the intelligent handheld game console as described in any one of the above methods is implemented.

[0105] The computer device 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 3 may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art will appreciate that Figure 3 It is only an example of computer device 3 and does not constitute a limitation on computer device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0106] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0107] In some embodiments, the memory 302 may be an internal storage unit of the computer device 3, such as a hard disk or memory of the computer device 3. In other embodiments, the memory 302 may also be an external storage device of the computer device 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 3. Further, the memory 302 may also include both an internal storage unit and an external storage device of the computer device 3. The memory 302 is used to store an operating system, an application program, a boot loader, data, and other programs, such as the program code of the computer program. The memory 302 may also be used to temporarily store data that has been output or is to be output.

[0108] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for controlling the posture of an intelligent handheld game console as described in any one of the above methods is implemented.

[0109] In this embodiment, if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above embodiment methods of this application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can at least include: any entity or device capable of carrying the computer program code to the photographing device / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0110] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0111] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0112] In the embodiments disclosed in this application, it should be understood that the disclosed device / terminal device and method can be implemented in other ways. For example, the device / terminal device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0113] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Claims

1. A gesture control method for an intelligent handheld game console, characterized in that: The method specifically comprises: The original posture data of the intelligent handheld game console is collected in real time according to the accelerometer, gyroscope and magnetometer, and the original posture data is periodically calibrated by a dynamic zero drift correction algorithm to obtain corrected posture data; Performing posture calculation and time window sampling on the corrected posture data to form first posture data, and inputting the first posture data into a time series prediction algorithm for prediction calculation to obtain posture change prediction data; The Bayesian filtering algorithm is used to fuse the visual data, ultrasonic data, lidar data and IMU data of the intelligent handheld game console to generate high-precision positioning information; Based on the posture change prediction data, the high-precision positioning information and the user's historical operation data, a user behavior modeling algorithm and a reinforcement learning algorithm are used to dynamically generate personalized posture control instructions; According to the AR visual feedback data and tactile feedback data of the intelligent handheld game console, deviation correction processing is performed on the personalized posture control instruction.

2. The method according to claim 1, characterized in that The periodic calibration of the original posture data by a dynamic zero drift correction algorithm to obtain the corrected posture data specifically includes: Set zero drift detection conditions and calibration cycle; When the zero drift condition is triggered or in each calibration cycle, a zero drift reference value is calculated based on the original attitude data of multiple sampling points, and the zero drift reference value is compensated for by a dynamic compensation formula and a temperature compensation formula to obtain the first zero drift data; The first zero drift data is subjected to attitude calculation to obtain corrected attitude data.

3. The method according to claim 2, characterized in that The zero drift detection conditions include the accelerometer static state determination condition and the gyroscope refinement determination condition; The accelerometer static state determination condition is: ,in, , and Represents the three-axis original output value of the accelerometer, g represents the local gravity acceleration, represents the stillness determination threshold; The gyroscope refinement determination condition satisfies ,in, , and Represents the three-axis raw output value of the gyroscope, Indicates the gyroscope stillness determination threshold.

4. The method according to claim 2, characterized in that: The method of calculating a zero drift reference value according to the original posture data of the plurality of sampling points, and performing zero drift compensation on the zero drift reference value by using a dynamic compensation formula and a temperature compensation formula to obtain the first zero drift data specifically includes: The zero drift reference values ​​corresponding to the accelerometer, gyroscope and magnetometer are calculated respectively according to the original attitude data of multiple sampling points, and the zero drift reference values ​​meet in, , and Represent the zero drift reference values ​​of the accelerometer, gyroscope and magnetometer respectively, , and They represent the original attitude data collected by the accelerometer, gyroscope, and magnetometer at the i-th sampling point, respectively, and N represents the number of sampling points; Using dynamic compensation formula , and Perform zero drift compensation to obtain the second zero drift data. The dynamic compensation formula satisfies , , ,in, , and Respectively represent the zero drift data corresponding to the accelerometer, gyroscope and magnetometer in the second attitude data, , and Respectively represent the raw attitude data collected by the accelerometer, gyroscope, and magnetometer; Using the temperature compensation formula , and Perform zero drift compensation to obtain the first zero drift data, and the temperature compensation formula satisfies Where T represents the current ambient temperature, Indicates the calibration temperature, , and Respectively represent the zero drift data corresponding to the accelerometer, gyroscope and magnetometer in the first zero drift data, , and Respectively indicate the calibration temperature The corresponding zero drift reference values ​​of the accelerometer, gyroscope and magnetometer are as follows: , and They represent the temperature drift coefficients of the accelerometer, gyroscope and magnetometer respectively.

5. The method according to claim 1, characterized in that The step of inputting the first posture data into a time series prediction algorithm for prediction calculation to obtain posture change prediction data specifically includes: The first posture data is standardized to obtain second posture data, and the second posture data satisfies ,in, represents the second posture data, represents the first posture data, represents the mean value of the Euler angle in the time window, Represents the standard deviation of the Euler angles within the time window; The second posture data is input into the LSTM prediction model, and training reasoning is performed through the forward propagation function to output the posture change prediction value; The posture change prediction value is merged with the current posture value to generate posture change prediction data, and the posture change prediction data satisfies ,in, represents the predicted value of posture change, Indicates the current attitude value, represents the predicted estimate of the posture change at a preset time in the future, t represents the current time, Indicates a preset time in the future.

6. The method according to claim 1, characterized in that The Bayesian filtering algorithm is used to fuse the visual data, ultrasonic data, laser radar data and IMU data of the intelligent handheld game console, specifically including: Setting the state vector and the observation vector ,in, Indicates the position coordinates of the intelligent handheld device in three-dimensional space. Represents the velocity component of the intelligent handheld device in three-dimensional space, The quaternion representing the attitude of the intelligent handheld game console. Represent visual data, ultrasonic data, lidar data and IMU data respectively; The state vector Input into the state transfer function, perform state prediction on the IMU drive of the intelligent handheld game console, and obtain a predicted state vector, which satisfies ,in, represents the predicted state vector, represents the state vector at the previous moment, Represents the IMU data at the current moment, represents the state transition function, represents process noise; The predicted state vector and the observed vector are fused to obtain an updated state vector, which satisfies , ,in, represents the Kalman gain, represents the observation model used to map the predicted state vector to the sensor observation space, represents the predicted state covariance matrix, represents the Jacobian matrix of the observation matrix, represents the observation noise covariance matrix.

7. The method according to claim 1, characterized in that The method of dynamically generating personalized posture control instructions based on the posture change prediction data, the high-precision positioning information and the user's historical operation data by using a user behavior modeling algorithm and a reinforcement learning algorithm specifically includes: Acquire user historical operation data, wherein the user historical operation data includes user operation acceleration, user operation angular velocity, game scene type and environment complexity; A convolutional neural network is used to extract local spatiotemporal features of the user's historical operation data, a long short-term memory network is used to capture the temporal dependency features of the user's historical operation data, and the local spatiotemporal features and the temporal features are fused to form a user behavior feature vector; The posture change prediction data, the high-precision positioning information and the user behavior feature vector are used as the state space, the proportional coefficient, integral coefficient and differential coefficient of the controller of the intelligent handheld game console and the maximum output torque are used as the action space, and minimizing the tracking error, maximizing the operating comfort and minimizing the energy consumption are used as the reward function; Based on the state space, the action space and the reward function, a reinforcement learning algorithm is used to dynamically generate personalized posture control instructions.

8. The method according to claim 1, characterized in that: The performing deviation correction processing on the personalized gesture control instruction according to the AR visual feedback data and the tactile feedback data of the intelligent handheld game console specifically includes: Get AR visual feedback data and tactile feedback data ,in, Indicates the plane deviation between the user's operation position and the target position in the AR interface. Indicates the yaw angle deviation between the user's operating direction and the target direction. Indicates the vibration intensity of the linear motor. Indicates vibration frequency; The AR visual feedback data The tactile feedback data is fused and processed to obtain a comprehensive deviation vector, and the comprehensive deviation vector satisfies in, represents the comprehensive deviation vector, represents the visual deviation amplitude, Indicates the direction of visual deviation, and represents the tactile intensity coefficient; A PID correction coefficient is generated based on the comprehensive deviation vector, and a deviation correction process is performed on the personalized posture control instruction according to the PID correction coefficient to obtain a corrected personalized posture control instruction, wherein the corrected personalized posture control instruction satisfies in, Indicates the parameters for correcting the personalized attitude control instructions. , and They represent the proportional coefficient, integral coefficient and differential coefficient of the controller of the intelligent handheld game console respectively. , and They represent the correction proportional coefficient, correction integral coefficient and correction differential coefficient in the PID correction coefficient respectively. Indicates the maximum output torque, , and Represents the adaptive weight coefficient.

9. An intelligent handheld game console posture control system, characterized in that: The system specifically comprises: The first attitude control module is used to collect the original attitude data of the intelligent handheld game console in real time according to the accelerometer, gyroscope and magnetometer, and periodically calibrate the original attitude data through a dynamic zero drift correction algorithm to obtain corrected attitude data; A second posture control module is used to perform posture calculation and time window sampling on the corrected posture data to form first posture data, and input the first posture data into a time series prediction algorithm for prediction calculation to obtain posture change prediction data; The third attitude control module is used to use the Bayesian filtering algorithm to fuse the visual data, ultrasonic data, laser radar data and IMU data of the intelligent handheld game console to generate high-precision positioning information; A fourth posture control module, for dynamically generating personalized posture control instructions based on the posture change prediction data, the high-precision positioning information and the user's historical operation data, using a user behavior modeling algorithm and a reinforcement learning algorithm; The fifth posture control module is used to perform deviation correction processing on the personalized posture control instruction according to the AR visual feedback data and tactile feedback data of the intelligent handheld game console.

10. A computer device, characterized in that: include: A memory, a processor and a computer program stored in the memory, when the computer program is executed on the processor, implements the gesture control method of the intelligent handheld game console as claimed in any one of claims 1 to 8.

Images