Pointing remote control method and system, device, storage medium, and program product
By combining extended Kalman filtering algorithm with IMU and UWB data, and dynamically adjusting yaw angle calculation, the pointing error problem caused by UWB signal loss is solved, achieving a more stable and accurate pointing and remote control effect, and improving the user experience.
Patent Information
- Application Number
- PCT/CN2025/094578
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-05-13
- Publication Date
- 2026-11-19
AI Technical Summary
When the UWB signal is outside its measurement range, the loss of the UWB signal causes the pointing calculation of the remote control to lose its absolute angle in the yaw direction. Relying solely on the IMU will result in pointing errors and reduce accuracy.
By acquiring sensor data from the pointing device, the yaw angle is calculated using the extended Kalman filter algorithm. Combined with IMU and UWB data, the yaw angle error is constrained by the difference in yaw angles between adjacent time points. The yaw angle calculation method is dynamically adjusted to improve positioning accuracy and stability.
It achieves more stable and smooth pointing in different scenarios, improves the user experience, and enhances the accuracy and responsiveness of pointing remote control.
Smart Images

Figure CN2025094578_19112026_PF_FP_ABST
Abstract
Description
Pointing remote control methods, systems, devices, storage media and software products Technical Field
[0001] This disclosure relates to the field of communication technology, and in particular to a pointing remote control method, system, device, storage medium, and program product. Background Technology
[0002] Pointing remote control involves sensor technology and spatial positioning technology. Inertial measurement units (IMUs) can be used to detect the movement trajectory, angle and direction of the remote control to achieve real-time capture of pointing actions. Ultra-wideband (UWB) technology can also be used to determine the absolute position of the remote control and the target device in space.
[0003] Currently, when the UWB signal is lost outside the measurement range, the pointing calculation of the remote control loses its absolute angle in the yaw direction. Relying solely on the IMU will produce pointing errors, ultimately reducing the accuracy of pointing. Summary of the Invention
[0004] This disclosure provides a pointing remote control method, system, device, storage medium, and program product to improve positioning accuracy and stability, and enhance the user experience.
[0005] In a first aspect, embodiments of this disclosure provide a pointing remote control method, the method comprising:
[0006] Acquire sensor data sent by the pointing device, wherein the sensor data is collected using different types of sensors;
[0007] The first yaw angle is determined based on the sensor data and the filtering algorithm;
[0008] The first difference between the yaw angles collected at adjacent times in the sensor data is used as the observation in the filtering algorithm to constrain the error of the first yaw angle and obtain the second yaw angle.
[0009] Based on the second yaw angle and the sensor data, determine the pointing position corresponding to the sensor data on the display screen.
[0010] Secondly, the present disclosure provides a pointing remote control system, including a display and a remote control, wherein:
[0011] The remote control is configured to collect data and transmit it to the display.
[0012] The display includes a display unit and a controller. The display unit is used to display content, and the controller is configured to perform the following steps:
[0013] Acquire sensor data sent by the pointing device, wherein the sensor data is collected using different types of sensors;
[0014] The first yaw angle is determined based on the sensor data and the filtering algorithm;
[0015] The first difference between the yaw angles collected at adjacent times in the sensor data is used as the observation in the filtering algorithm to constrain the error of the first yaw angle and obtain the second yaw angle.
[0016] Based on the second yaw angle and the sensor data, determine the pointing position corresponding to the sensor data on the display screen.
[0017] Thirdly, embodiments of this disclosure also provide a remote control device, including a control unit and a remote control unit, wherein:
[0018] The remote control unit is configured to collect data and transmit it to the display.
[0019] The control unit includes a processor and a memory, the memory being used to store programs executable by the processor, and the processor being used to read the programs from the memory and perform the following steps:
[0020] Acquire sensor data sent by the pointing device, wherein the sensor data is collected using different types of sensors;
[0021] The first yaw angle is determined based on the sensor data and the filtering algorithm;
[0022] The first difference between the yaw angles collected at adjacent times in the sensor data is used as the observation in the filtering algorithm to constrain the error of the first yaw angle and obtain the second yaw angle.
[0023] Based on the second yaw angle and the sensor data, determine the pointing position corresponding to the sensor data on the display screen.
[0024] Fourthly, embodiments of this disclosure also provide an electronic device, which includes a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and perform the following steps:
[0025] Acquire sensor data sent by the pointing device, wherein the sensor data is collected using different types of sensors;
[0026] The first yaw angle is determined based on the sensor data and the filtering algorithm;
[0027] The first difference between the yaw angles collected at adjacent times in the sensor data is used as the observation in the filtering algorithm to constrain the error of the first yaw angle and obtain the second yaw angle.
[0028] Based on the second yaw angle and the sensor data, determine the pointing position corresponding to the sensor data on the display screen.
[0029] Fifthly, embodiments of this disclosure also provide a pointing remote control device, the device comprising:
[0030] The data acquisition unit is used to acquire sensor data sent by the pointing device, wherein the sensor data is collected using different types of sensors;
[0031] The first calculation unit is used to determine the first yaw angle based on the sensor data and the filtering algorithm;
[0032] The second calculation unit is used to take the first difference of the yaw angle collected at adjacent time moments in the sensing data as the observation in the filtering algorithm, constrain the error of the first yaw angle, and obtain the second yaw angle.
[0033] The position determination unit is used to determine the pointing position corresponding to the sensing data on the display screen based on the second yaw angle and the sensing data.
[0034] In a sixth aspect, embodiments of this disclosure also provide a computer storage medium having a computer program stored thereon, which, when executed by a processor, is used to implement the steps of the method described in the first aspect above.
[0035] In a seventh aspect, this disclosure provides a computer program product comprising: computer program code, which, when run on a computer, causes the computer to perform the method described in the first aspect.
[0036] These or other aspects of this disclosure will become more apparent in the following description of embodiments. Attached Figure Description
[0037] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0038] Figure 1 is a schematic diagram of a scenario for executing a remote control system according to an embodiment of this disclosure;
[0039] Figure 2 is a schematic diagram of pointing position calculation in a pointing remote control scenario provided by an embodiment of this disclosure;
[0040] Figure 3 is a flowchart of a pointing remote control method provided in an embodiment of this disclosure;
[0041] Figure 4 is a flowchart illustrating an implementation of dynamic error adjustment according to an embodiment of this disclosure;
[0042] Figure 5 is a schematic diagram of a UWB signal measurement range provided in an embodiment of this disclosure;
[0043] Figure 6 is a flowchart of a reporting implementation method provided in an embodiment of this disclosure;
[0044] Figure 7 is a schematic diagram of infrared signal interaction between the display screen and the pointing device provided in an embodiment of this disclosure;
[0045] Figure 8 is a flowchart illustrating an implementation of infrared data pointing method according to an embodiment of this disclosure;
[0046] Figure 9 is a schematic diagram of a pointing system provided in an embodiment of this disclosure;
[0047] Figure 10 is a flowchart illustrating a specific implementation of a pointing remote control according to an embodiment of this disclosure;
[0048] Figure 11 is a schematic diagram of a pointing remote control system provided in an embodiment of this disclosure;
[0049] Figure 12 is a schematic diagram of a remote control device provided in an embodiment of this disclosure;
[0050] Figure 13 is a schematic diagram of an electronic device provided in an embodiment of this disclosure;
[0051] Figure 14 is a schematic diagram of a pointing remote control device provided in an embodiment of this disclosure. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of this disclosure clearer, the disclosure will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.
[0053] In this disclosure, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0054] The application scenarios described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided in this disclosure. Those skilled in the art will understand that with the emergence of new application scenarios, the technical solutions provided in this disclosure are also applicable to similar technical problems. In the description of this disclosure, unless otherwise stated, "multiple" means two or more.
[0055] Before introducing the pointing remote control method provided in the embodiments of this disclosure, the technical background of the embodiments of this disclosure will be described in detail below for ease of understanding.
[0056] Pointing remote control is an interactive technology that achieves control by physically pointing at a target device. Its core lies in translating the user's pointing actions into control commands. Users simply need to point the remote control at the target device or a specific area on the screen to operate it. Pointing remote control involves sensor technology and spatial positioning technology. It uses inertial measurement units (IMUs) to detect the remote control's trajectory, angle, and direction, achieving real-time capture of pointing actions. An IMU typically includes a gyroscope (detecting angular velocity), an accelerometer (detecting linear acceleration), and a magnetometer (detecting direction). By fusing gyroscope and accelerometer data, it calculates the remote control's attitude changes (such as tilting and rotating) in real time. It can also combine the initial position to estimate the screen coordinates or device position pointed at by the user. Pointing remote control typically uses ultra-wideband (UWB) to determine the absolute position of the remote control and the target device in space. UWB achieves high-precision ranging and positioning by emitting extremely short pulse (nanosecond-level) broadband signals. Specifically, it determines the absolute position and pointing angle of the remote control and the target device by measuring the signal's time of flight (ToF) or phase difference.
[0057] The key to pointing remote control lies in accurate pointing. Currently, due to the limitations of UWB chips and antennas, UWB signals are directional. When the UWB signal measurement is out of range, the UWB signal will be unavailable. Losing the UWB signal will cause the entire pointing remote control to lose the absolute angle in the yaw direction. Relying solely on the IMU, over time, yaw angle errors and position errors will gradually accumulate, causing errors in pointing calculations.
[0058] To address the aforementioned technical issues, the pointing remote control method provided in this embodiment is based on the core idea of using the first difference between two yaw angle measurements to constrain the first yaw angle calculated from sensor data and a filtering algorithm, thus obtaining a second yaw angle. This second yaw angle, along with the sensor data, is then used to determine the pointing position. This solves the problem of significant errors in the calculated first yaw angle due to environmental influences, thereby achieving a smoother following response to user pointing and improving the timeliness and smoothness of the response. By providing different yaw angle calculation methods, it adapts to yaw angle error calibration under different conditions, improving the stable and smooth following effect in various scenarios and effectively enhancing the user experience.
[0059] First, a brief introduction to the basic principle of pointing remote control will be given, taking a pointing remote control system including a remote controller and a display screen as an example:
[0060] Referring to Figure 1, this embodiment provides a schematic diagram of a scenario for executing a remote control system, including a remote controller 100 and a display screen 101. The basic principle of pointing remote control is that the position and orientation of the remote controller can determine the pointing position. Position represents the coordinates of the remote controller relative to the display screen, and orientation represents the specific direction the remote controller is pointing. The remote controller has a UWB array antenna and an IMU module, and the display screen has a UWB array antenna. The UWB array antennas are generally orthogonal, thereby defining a coordinate system in space. Through a pair of UWB array antennas, the coordinates of the remote controller relative to the display screen and the coordinates of the display screen relative to the remote controller can be obtained. These two coordinates have a rotational relationship, and this rotational relationship is the orientation of the remote controller.
[0061] Referring to Figure 2, this embodiment provides a schematic diagram of pointing position calculation in a pointing remote control scenario, where point 200 represents the position of the remote control. It is assumed that the coordinates of the remote control in the display screen coordinate system are (p...). x p y p z Point 201 represents the specific point the remote control is pointing to. The coordinates of this point consist of two parts: one is the projection of the remote control's position onto the display screen, and the other is the position offset caused by the remote control's attitude. Using the UWB array antenna, the remote control's coordinates P2 are obtained by positioning it in the display screen's coordinate system, and the display screen's coordinates P1 are obtained by positioning it in the remote control's coordinate system.
[0062] The attitude of a remote control is usually represented by quaternions or Euler angles, and the change in attitude can be represented by a rotation matrix. For a fixed point (such as a large screen), the coordinates of the fixed point in the remote control coordinate system at a certain moment are P1. After the attitude of the remote control changes, the coordinates of the fixed point are P2. The rotation matrix for this change is R. For example, if R is a 3×3 matrix, then P2 = R×P1. The specific value of R can be calculated using a filtering algorithm.
[0063] The Extended Kalman Filter (EKF) is built upon the classic Kalman filter algorithm. KF inherits the prediction-update framework of the Kalman filter but introduces a linearization operation. The core idea is that, for general nonlinear systems, it first focuses on the filter value... The nonlinear functions f(*) and h(*) are expanded into Taylor series, but only the first-order and lower-order terms are retained (discarding second-order and higher-order terms), resulting in an approximate linearized model. Then, the Kalman filter algorithm is used to perform filtering and estimation of the target. The basic formula of this filtering algorithm is shown below:
[0064] a. The steps in the prediction stage (linearization of the nonlinear model) are as follows:
[0065] Step a1) State prediction:
[0066] In formula (1), Let f(*) represent the predicted state, and f(*) represent the nonlinear state transition function (i.e., the state prediction function), corresponding to the prediction process in system modeling. k-1 This represents system control variables (such as acceleration, angular velocity, etc.).
[0067] Step a2) Covariance prediction:
[0068] In formula (2), Q represents the covariance of the predicted state. k A represents the noise covariance of the prediction process (i.e., the error in the prediction process). k It is the Jacobian matrix of the state prediction function f(*), and its calculation formula is:
[0069] In formula (3), A k Indicates f(*) in The partial derivative at point .
[0070] b. The update phase (linearization of the nonlinear observation model) steps are as follows:
[0071] Step b1) Kalman gain calculation:
[0072] In formula (4), K k Indicates Kalman gain, H represents the covariance of the predicted state. k Let h(*) be the Jacobian matrix, where h(*) represents the nonlinear observation function (e.g., sensor model), corresponding to the observation process in system modeling. R k This represents the observation noise covariance (the error during the observation process).
[0073] Step b2) Status update:
[0074] In formula (5), K represents the updated predicted state. k Let h(*) represent the Kalman gain, h(*) represent the nonlinear observation function, and z represent the Kalman gain. k This represents the actual observed value, corresponding to the actual measured value of the system.
[0075] Step b3) Covariance update:
[0076] In formula (6), P k Let I represent the updated covariance, and K represent the identity matrix. k Indicates Kalman gain, H represents the covariance of the predicted state. k It is the Jacobian matrix of the measurement function h(*), and the calculation formula is:
[0077] Quaternions are a way to represent attitude, and attitude angles (Euler angles) are also a way to represent attitude; they can be converted to each other. Rotation matrices are a way to express rotation. Quaternion differential equations are used in quaternion prediction. The conversion process between quaternions and angles is shown below:
[0078] (1) The rotation matrix R in quaternion representation:
[0079] In formula (8), q1, q2, q3, and q4 represent quaternions.
[0080] (2) Quaternion to attitude angle conversion:
[0081] In formula (9), roll represents the roll angle about the y-axis, pitch represents the pitch angle about the x-axis, yaw represents the yaw angle about the z-axis, and q1, q2, q3, and q4 represent quaternions.
[0082] (3) Attitude angles are converted to quaternions:
[0083] (4) Quaternion differential equation:
[0084] In formula (11), ω x ω y ω z It represents angular velocity.
[0085] As shown in Figure 3, this embodiment provides a pointing remote control method, which can be applied to display devices or electronic devices and establish a communication connection with the display device. This embodiment does not impose too many limitations on it. The specific implementation process of this method is as follows:
[0086] Step 300: Obtain the sensing data sent by the pointing device, wherein the sensing data is collected using different types of sensors;
[0087] Optionally, the pointing device includes, but is not limited to, devices that collect sensor data, such as remote controls and game controllers. Optionally, the sensor data includes, but is not limited to, at least one of attitude data, position data, motion data, and infrared data. Optionally, the attitude data includes, but is not limited to, at least one of roll angle, pitch angle, and yaw angle. Optionally, the position data includes, but is not limited to, the coordinates of the pointing device in the display coordinate system and the coordinates of the display in the pointing device coordinate system. Optionally, the motion data includes, but is not limited to, at least one of angular velocity and acceleration.
[0088] Optionally, the sensing data in this embodiment includes various types, acquired using different types of sensors, including but not limited to: at least one of IMU, UWB, and infrared sensors. In practice, the IMU is used to acquire attitude data, motion data, etc.; the UWB is used to acquire position data; and the infrared sensor is used to acquire infrared data.
[0089] Step 301: Determine the first yaw angle based on the sensor data and filtering algorithm;
[0090] Optionally, the filtering algorithm in this embodiment includes, but is not limited to, any of the following: Kalman filtering, extended Kalman filtering, or other variations of Kalman filtering.
[0091] In some embodiments, the sensing data includes attitude data, position data, and motion data; determining the first yaw angle based on the sensing data and a filtering algorithm includes:
[0092] The process of predicting state variables and obtaining observation variables from state variables is established based on the filtering algorithm and control variables, wherein the control variables represent motion data, the state variables represent attitude data, and the observation variables represent position data; the first yaw angle is determined based on the iterative updates in the prediction process and the observation process.
[0093] In implementation, based on the extended Kalman filter, this embodiment establishes a first filtering model to calculate the first yaw angle. The model establishment process is as follows:
[0094] (1) State variables: Where q1, q2, q3, and q4 represent quaternions;
[0095] (2) Observations: Where x′, y′, z′ represent the inverted coordinates P2 of the pointing device (such as a remote control) in the display screen coordinate system; x, y, z represent the coordinates P1 of the display screen in the remote control coordinate system. P1 and P2 satisfy the following relationship: P2 = R × P1, where R is the rotation matrix.
[0096] (3) Control quantity: Where, ω x ω y ω z This represents the angular velocities along the x, y, and z axes in the remote control coordinate system.
[0097] (4) Prediction process:
[0098] (5) Observation process (obtaining the observed quantity from the state quantity):
[0099] In practice, the extended Kalman filter algorithm is used in the above manner to calculate the quaternion based on the sensor data collected by the IMU and UWB. The first yaw angle is obtained based on the conversion relationship between the quaternion and the attitude data.
[0100] Step 302: Take the first difference of the yaw angles collected at adjacent times in the sensing data as the observation in the filtering algorithm, constrain the error of the first yaw angle, and obtain the second yaw angle;
[0101] In some embodiments, the second yaw angle is determined as follows:
[0102] The second yaw angle is determined based on the sensor data, the first difference, and the filtering algorithm.
[0103] In implementation, based on the extended Kalman filter, this embodiment establishes a second filtering model to calculate the second yaw angle. The model establishment process is as follows:
[0104] (1) State variables: Where q1, q2, q3, and q4 represent quaternions;
[0105] (2) Observations:
[0106] Where x′, y′, z′ represent the inverted coordinates P2 of the pointing device (such as a remote control) in the display coordinate system; a x a y a z The x, y, and z axes represent the accelerations in the remote control coordinate system, and dyaw represents the first difference in yaw angles collected at adjacent moments.
[0107] (3) Control quantity:
[0108] Where, ω x ω y ω z Let x, y, and z represent the angular velocities along the x, y, and z axes in the remote control coordinate system. x, y, and z represent the coordinates P1 of the display screen in the remote control coordinate system. P1 and P2 satisfy the following relationship: P2 = R × P1, where R is the rotation matrix.
[0109] (4) Prediction process:
[0110] (5) Observation process (obtaining the observed quantity from the state quantity):
[0111] The difference in yaw angles collected by the IMU at adjacent times can be used as the first difference.
[0112] In practice, the extended Kalman filter algorithm is used in the above manner to calculate the quaternion based on the sensor data collected by the IMU and UWB and the difference in yaw angle. The second yaw angle is obtained based on the conversion relationship between the quaternion and the attitude data.
[0113] Step 303: Determine the pointing position corresponding to the sensing data on the display screen based on the second yaw angle and the sensing data.
[0114] In some embodiments, the pointing position corresponding to the sensing data on the display screen is determined by the following method:
[0115] Based on the second yaw angle and the attitude data in the sensing data, the pointing position on the display screen corresponding to the sensing data is determined.
[0116] In practice, the pointing position on the display screen is determined based on the second yaw angle and the attitude data (roll angle and pitch angle) in the sensor data. The reporting data is determined based on the pointing position, and the display screen responds to the pointing device based on the reporting data, thereby realizing the function of controlling the display screen display through the pointing device.
[0117] In some embodiments, the pointing position on the display screen corresponding to the sensing data is determined in the following manner:
[0118] A second difference between a first yaw angle and a second yaw angle is determined; in response to the second difference being less than or equal to a threshold, the pointing position on the display screen corresponding to the sensing data is determined based on the second yaw angle and the attitude data in the sensing data.
[0119] In implementation, the absolute value of the difference between the first and second yaw angles can be used as the second difference. If the second difference is less than or equal to a threshold, it indicates that the attitude data of the pointing device is relatively stable and the user's gesture movement is relatively smooth. The pointing position can be directly calculated using the second yaw angle and the roll and pitch angles in the attitude data. Then, the reporting data is calculated based on the pointing position. After receiving the reporting data, the display device responds, for example, by displaying a mouse that moves according to the movement of the reporting data. For the user, when operating the pointing device, the movement of the mouse on the display device can follow the user's movement trajectory. Since the second yaw angle is obtained by constraining the directly calculated first yaw angle based on the first difference of the yaw angle, the relative difference of the yaw angle collected by the IMU is introduced when calculating the second yaw angle. Because the accuracy of the yaw angle collected by the IMU is much higher than the accuracy of the calculated first yaw angle, the accuracy of the second yaw angle is higher. This allows for more precise positioning and following of the user's pointing, effectively improving the user experience.
[0120] In some embodiments, this embodiment may also provide the following determination method, wherein the sensing data includes angular velocity; used to determine whether a stable mode has been entered, and in the stable mode, the pointing position is directly determined using the second yaw angle and the attitude data in the sensing data, the specific steps of which are as follows:
[0121] The total number of angular velocities exceeding a speed threshold is counted; in response to the total number exceeding a first set value, the pointing position on the display screen corresponding to the sensing data is determined based on the second yaw angle and the attitude data in the sensing data.
[0122] In practice, the sensor data also includes angular velocity, such as the angular velocity detected by the gyroscope of the IMU. Each time an angular velocity is detected, it is sent to the display device. This can be added to the sensor data. By counting multiple consecutive angular velocities, when it is determined that the angular velocity is greater than the velocity threshold for h consecutive times, and h is greater than the first set value (e.g., h>3), it indicates that the current state has entered a stable state. The pointing position can be directly calculated using the second yaw angle and the roll and pitch angles from the attitude data. Then, the reporting data is calculated based on the pointing position and sent to the display device. h is an integer greater than 0.
[0123] In some embodiments, this embodiment also provides a processing flow in a non-stationary mode to enter a fast convergence mode, the specific flow of which is as follows:
[0124] In response to the second difference being greater than a threshold, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data until the second difference between the first yaw angle and the updated second yaw angle is less than or equal to the threshold, at which point the update stops.
[0125] In practice, when the second difference is greater than a threshold (i.e., the second difference between the first and second yaw angles is greater than the threshold), it indicates a large error in the yaw angle. A rapid correction of the error between the first and second yaw angles is needed. This can be achieved by recalculating the quaternion using the first yaw angle and the roll and pitch angles from the attitude data. This quaternion is then used to iteratively update the second yaw angle until the error is less than or equal to the threshold. For example, in the first iteration, the first second yaw angle is obtained by updating based on the first yaw angle and attitude data. The second difference between the first yaw angle and the second yaw angle from the first iteration is calculated. If the second difference is greater than the threshold, the next iteration continues. In the i-th iteration, the second difference is calculated based on the i-th second yaw angle obtained in the i-th iteration. If the second difference is greater than the threshold, the (i+1)-th iteration continues until the second difference is less than or equal to the threshold, at which point the iteration stops. Here, i is an integer greater than 1.
[0126] In some embodiments, the second yaw angle is iteratively updated in the following manner:
[0127] An updated quaternion is generated based on the first yaw angle and the attitude data; the updated quaternion is used to update the prediction and observation processes of the filtering algorithm to determine the updated second yaw angle.
[0128] In practice, when the second difference is greater than the threshold, it indicates that the error of the second yaw angle is large. Therefore, the quaternion can be recalculated using the first yaw angle and the current roll and pitch angles measured by the IMU. This quaternion is then used to update the state variables in the extended Kalman filter algorithm. Based on the state variables, observations, and control variables, the equations for the prediction and observation processes are re-obtained, thus obtaining the updated second yaw angle.
[0129] In some embodiments, this embodiment also provides a method for determining whether to enter a fast convergence mode. When there is a large error in the current second yaw angle, the error of the second yaw angle is quickly corrected through the following process to enter the fast convergence mode, as shown below:
[0130] The second difference between the first yaw angle and the second yaw angle is determined based on multiple collected sensor data.
[0131] Count the total number of second differences that are consecutively greater than the threshold;
[0132] In response to the total number being greater than a second set value, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data.
[0133] In practice, to improve the accuracy of error judgment, multiple sets of sensor data can be recorded. The second difference between the first and second yaw angles is calculated using each set of sensor data. This second difference is then statistically analyzed over multiple calculations. If m consecutive second differences are greater than a threshold and m is greater than a second set value, it indicates that the yaw angle error is large and needs correction. In this case, the second yaw angle can be iteratively updated using the first yaw angle and the attitude data. m is an integer greater than 0.
[0134] This embodiment provides two different modes. In fast convergence mode, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data. The iteration stops when the calculated second difference after the update is less than or equal to a threshold, or when the number of iterations reaches a threshold. Then, the pointing position on the display screen corresponding to the sensing data is determined using the second yaw angle and the attitude data in the sensing data. In stable mode, the pointing position on the display screen corresponding to the sensing data is determined directly based on the second yaw angle and the attitude data in the sensing data.
[0135] In fast convergence mode, the rotation matrix R in the Extended Kalman Filter (EKF) algorithm has a smaller error related to positioning P1 compared to the stationary mode. However, the error of the first difference in yaw angles collected at adjacent time points in the sensor data is larger than the error of the first difference dyaw in the stationary mode. Similarly, in stationary mode, the rotation matrix R in the EKF algorithm has a larger error related to positioning P1 compared to the fast convergence mode. However, the error of the first difference in yaw angles collected at adjacent time points in the sensor data is smaller than the error of the first difference in the stationary mode. For example, in fast convergence mode, the error of P1 is 10 and the error of dyaw is 1, while in stationary mode, the error of P1 is 150 and the error of dyaw is 0.0001.
[0136] As shown in Figure 4, this embodiment provides a flowchart for dynamic error adjustment, and the specific process is as follows:
[0137] Step 400: Initial power-on;
[0138] Step 401: Enter fast convergence mode;
[0139] In the fast convergence mode, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data. The process stops when the second difference calculated after the iterative update is less than or equal to a threshold, or when the number of iterations reaches a threshold. Then, the pointing position on the display screen corresponding to the sensing data is determined using the second yaw angle and the attitude data in the sensing data.
[0140] Step 402: Determine whether the current angular velocity of the gyroscope is greater than the velocity threshold. If yes, proceed to step 403; otherwise, return to step 401.
[0141] Step 403: Accumulate the first count value;
[0142] Step 404: Determine whether the first count value is greater than the first set value. If yes, proceed to step 405; otherwise, return to step 401.
[0143] Step 405: Switch to stable mode;
[0144] In stable mode, the pointing position on the display screen corresponding to the sensing data is determined directly based on the second yaw angle and the attitude data in the sensing data.
[0145] Step 406: Determine the second difference between the first yaw value and the second yaw value;
[0146] Step 407: Determine whether the second difference is greater than the threshold. If yes, proceed to step 408; otherwise, proceed to step 411.
[0147] Step 408: Accumulate the second count value;
[0148] Step 409: Determine whether the second count value is greater than the second set value. If so, proceed to step 410.
[0149] Step 410: Clear the first count value to zero and return to step 401;
[0150] Step 411: Clear the second count value and return to step 405.
[0151] During implementation, upon initial power-on, it enters fast convergence mode. When the gyroscope's angular velocity exceeds a velocity threshold (equivalent to significant motion), it begins counting. If the count value exceeds a first set value, it switches to steady mode (EKF algorithm; movement helps with result convergence). In steady mode, if the error between the first and second yaw angles (i.e., the second difference) exceeds a threshold, and this error continues to exceed the threshold multiple times, it enters fast convergence mode again. Optionally, it can also automatically switch to fast convergence mode via manual command, or switch between fast convergence and steady modes when the remote control direction is detected to have changed from reverse to forward.
[0152] This embodiment provides switching between different convergence parameters. Due to the characteristics of the IMU, the IMU can obtain the pitch and roll angles relative to the ground. Under normal circumstances, the pitch and roll angles relative to the ground can be considered as the pitch and roll angles of the device (such as a remote control) (equivalent to assuming that the large screen is placed perpendicular to the ground, ignoring other special cases). However, the IMU cannot determine the absolute value of the yaw angle pointing to the remote control. The yaw angle needs to be calculated using the values of the UWB dual-sided array antennas. However, UWB data is greatly affected by the environment and is prone to large errors during use. Therefore, in this embodiment, during the UWB yaw angle calculation process, the difference in relative yaw angles calculated by the IMU (dyaw) is used as the observation in the UWB calculation process to constrain the error of the UWB yaw angle calculation. During this process, when the dyaw error is large and the UWB error is small, the yaw angle calculation can converge quickly, thus rapidly correcting the error. Conversely, when the dyaw error is small and the UWB error is large, the yaw angle calculation is relatively stable and has better tracking performance. When an error occurs, because the entire calculation process is biased towards the IMU's dyaw, the yaw angle error cannot be corrected by the UWB in a timely manner. Therefore, this embodiment provides a dynamic switching process between a fast convergence mode and a stable mode, allowing the errors of UWB and dyaw to be dynamically adjusted. This enables rapid calibration when an absolute error occurs in the yaw angle, while ensuring that the yaw angle remains stable and does not fluctuate during normal use.
[0153] Based on the UWB PDOA (Phase-Difference-of-Arrival) positioning principle, the UWB array antenna has a certain directionality (e.g., 60 degrees). When the remote control position exceeds the measurement range of the UWB signal, the UWB information will be unstable or even completely lost. See Figure 5, which is a schematic diagram of the UWB signal measurement range provided in this embodiment. The measurement range of the UWB array antenna is [-60°, 60°]. When the range exceeds 60°, the UWB signal is easily lost. In actual use, the pointing device cannot be guaranteed to be within a 60-degree range. When the remote control is outside the range, the cumulative error of the algorithm and IMU yaw angle increases the yaw angle error, making observation impossible. This increases the error in the pointing position of the pointing device on the display screen. When the remote control returns to the UWB measurement range, the cumulative error of the yaw angle is corrected, and the remote control coordinates in the display screen coordinate system are also corrected. At this time, the pointing position is prone to sudden changes. In order to improve the user experience, this embodiment provides a reporting method that can be corrected imperceptibly during reporting.
[0154] Optionally, this embodiment can use HID (Human Interface Device) reporting. HID reporting refers to the process of converting physical actions (such as pointing, clicking, moving, etc.) of a pointing device (such as a remote control) into a standardized data report format according to the HID protocol specification, and transmitting it to the host (such as a computer, smart TV, etc.). Its core lies in using the report descriptor defined by the HID protocol to accurately describe the data format and purpose, thereby achieving efficient interaction between the device and the host. The core of HID reporting is the report descriptor, which defines the specific meaning of each byte or bit sent by the device. For example, in pointing remote control, the descriptor needs to specify: which bits represent the X / Y axis offset (corresponding to pointing actions); which bits represent button states (such as click, long press); and whether it includes other functions (such as scroll wheel, gyroscope data). In point-and-control scenarios, HID (Hyper-Indexed Reporting) is primarily used for coordinate mapping of pointing actions. Pointing angles and displacements are collected via inertial measurement units (IMUs) or optical sensors, converted into relative X / Y coordinates, and encapsulated as HID reports. The host computer adjusts the cursor position or interface focus based on these coordinates to achieve an interactive effect that follows user input. This requires fusing data from multiple sources, such as IMUs and optical sensors, into a unified HID report. For example, Kalman filtering can be used to eliminate gyroscope noise before converting the data to screen coordinates.
[0155] In some embodiments, this embodiment also provides a reporting method, the specific implementation process of which is as follows:
[0156] In response to an update of the current pointing position, and at least one of the previous n pointing positions being outside the display screen range, the reporting data is determined based on the current pointing position.
[0157] In practice, when the system detects that the current pointing position has been updated, it counts whether there is a pointing position that exceeds the range of the display screen among the previous n pointing positions. If so, it is assumed that the user may have just started to perform a pointing operation. Therefore, the reporting data can be directly determined based on the current position, converted into screen coordinates, and sent to the display screen for response.
[0158] In some embodiments, this embodiment provides a method for curve reporting, the specific implementation process of which is as follows:
[0159] In response to an update of the current pointing position, and provided that the pointing positions in the previous n updates did not exceed the display screen range, the reporting data is determined based on the current pointing position, the previous pointing position, and the current angular velocity.
[0160] During implementation, if an update to the current pointing position is detected, it indicates that the user has made a new pointing operation. At this point, the previous n pointing positions are counted. If none of the previous n positions exceeded the display screen's range, it means the user has been pointing on the screen for a period of time. To improve the responsiveness of user actions and make the pointing trajectory smoother and more responsive, the current pointing position can be updated based on the previous pointing position and the current angular velocity, thus determining more accurate reporting data. Here, n is an integer greater than 0.
[0161] In some embodiments, the current pointing position is updated to obtain new reporting data using the following method:
[0162] The current reporting data is determined based on the current pointing position, the previous reporting data is determined based on the previous pointing position, and the compensation value is determined based on the current angular velocity.
[0163] The current reporting data, the previous reporting data, and the compensation value are weighted and summed according to their respective compensation coefficients to obtain the reporting data.
[0164] In practice, the current reporting data, the previous reporting data, and the compensation value determined by the current angular velocity can be weighted and summed to obtain reporting data that changes more smoothly.
[0165] In some embodiments, the respective compensation coefficients can also be determined in the following ways:
[0166] Based on the difference between the current reporting data and the previous reporting data, and the current angular velocity, determine the compensation coefficients for the current reporting data, the previous reporting data, and the compensation value.
[0167] In practice, different compensation coefficients can be set according to the relationship between the reporting difference and the current angular velocity. Optionally, the sum of the compensation coefficients of the current reporting data and the previous reporting data is 1.
[0168] In some embodiments, the compensation coefficients for each data point are adjusted in the following manner:
[0169] If the reported point difference changes in a direct proportional manner with the current angular velocity, then the compensation coefficient of the current reported point data is increased.
[0170] If the reported point difference changes inversely with the current angular velocity, and the degree of change in the reported point data is greater than that of the current angular velocity, then the compensation coefficient of the previous reported point data is increased.
[0171] If the reported point difference changes inversely with the current angular velocity, and the degree of change in the reported point data is less than that of the current angular velocity, then the compensation coefficient of the compensation value is increased.
[0172] Optionally, the degree of change in the reported data can be determined by the difference between the current reported value and the previous reported value.
[0173] In implementation, in the curve reporting scheme, the current pointing position, the previous reporting value, and the current gyroscope data are comprehensively considered. Under normal circumstances, the difference between the current reporting value and the previous reporting value of the current pointing position should be proportional to the gyroscope angular velocity data. If the difference between the current reporting value and the previous reporting value is large and the w value is small, the compensation coefficient should be biased towards the previous reporting value, that is, the compensation coefficient of the previous reporting value should be increased. If the difference between the current reporting value and the previous reporting value is small and the w value is large, the compensation coefficient of the compensation value should be increased. In other cases, the compensation coefficient of the current reporting value should be increased. Finally, the comprehensive reporting value is calculated, and then the reporting is performed and saved. The formula for calculating the comprehensive reporting value is as follows: x=k1×x1+(1-k1)×x2+k2×x3 Formula (12);
[0174] In formula (12), x1 represents the previous reported value, x2 represents the current reported value, x3 represents the compensation value (the compensation value is related to the angular velocity and reflects the current actual movement speed of the human hand), k1 represents the compensation coefficient of the previous reported value, 1-k1 represents the compensation coefficient of the current reported value, and k2 represents the compensation coefficient of the compensation value.
[0175] As shown in Figure 6, this embodiment also provides a flowchart for the reporting implementation process, and the specific implementation process is as follows:
[0176] Step 600: An update to the currently pointed-to position has been detected;
[0177] Step 601: Determine whether the positions pointed to in the first 3 times did not exceed the range of the display screen. If so, proceed to step 602; otherwise, proceed to step 609.
[0178] Step 602: Determine the reporting difference between the current reporting data and the previous reporting data;
[0179] Step 603: Determine whether the reported difference and the current angular velocity change proportionally. If yes, proceed to step 604; otherwise, proceed to step 605.
[0180] Step 604: Increase the compensation coefficient of the current reporting data;
[0181] Step 605: Determine whether the change in the reported data is greater than the current angular velocity. If yes, proceed to step 606; otherwise, proceed to step 607.
[0182] Step 606: Increase the compensation coefficient of the previous reported data;
[0183] Step 607: Increase the compensation coefficient of the compensation value;
[0184] Step 608: According to the compensation coefficients of the current reporting data, the previous reporting data, and the compensation value, perform a weighted sum of the current reporting data, the previous reporting data, and the compensation value to obtain the reporting data;
[0185] Step 609: Determine the reporting data based on the current pointing position;
[0186] Step 610: Send the reported data to the display screen so that the display screen can respond and display according to the reported data.
[0187] During implementation, when the pointing position and gyroscope data are updated, it is first determined whether to use curve reporting. If the pointing position goes out from one place on the screen and then returns from another, curve reporting is not performed, and the reporting value is calculated directly based on the pointing position. Otherwise, curve reporting is enabled. During curve reporting, the compensation coefficients of the current reporting value, the previous reporting value, and the compensation value are determined based on the relationship between the difference between the current reporting value and the previous reporting value and the current angular velocity. Then, using the respective compensation coefficients of the current reporting value, the previous reporting value, and the compensation value, a weighted sum is performed on the current reporting value, the previous reporting value, and the compensation value to calculate the final reporting value. HID reporting is then performed, and the reporting data is updated.
[0188] In some embodiments, the sensing data in this embodiment also includes infrared data; referring to Figure 7, a schematic diagram of infrared signal interaction between the display screen and the pointing device provided in this embodiment, wherein both the display screen and the pointing device have infrared modules. The display screen can only receive infrared signals when the remote control is facing the display screen. When the remote control is facing the display screen in reverse, the display screen cannot receive infrared signals. During normal use of the remote control, the remote control will periodically send coded signals to the infrared module. If the display screen can receive the infrared signal normally, it means that the remote control is facing the right way; otherwise, the remote control is facing the wrong way.
[0189] In some embodiments, this embodiment also uses infrared data to determine the orientation of the pointing device, thereby correcting the currently obtained yaw angle in a timely manner. The specific implementation process is as follows:
[0190] Procedure a: Determine the direction of the pointing device based on the infrared data, and convert the first yaw angle into a direction consistent with the stated direction;
[0191] In some embodiments, this embodiment can also correct the pointing of the pointing device by repeatedly judging the pointing of the infrared data, so as to improve the accuracy of the correction. Specifically, the positive and negative directions of the pointing device are determined in the following way:
[0192] In response to receiving infrared data multiple times in succession, the pointing device is determined to be pointing in the positive direction;
[0193] If no infrared data is received at least once in a series of consecutive tests, the pointing device is determined to be pointing in the opposite direction.
[0194] During implementation, the number of times infrared data is received consecutively can be counted. If infrared data is received multiple times (e.g., 3 times), it indicates that the pointing device is pointing in the right direction, meaning the user wants to control the display screen through the pointing device. If infrared data is not received at least once, it indicates that the pointing device may be pointing in the wrong direction, and the yaw angle needs to be corrected.
[0195] Optionally, after determining the direction of the pointing device, if the direction of the pointing device and the direction of the first yaw angle are inconsistent, the first yaw angle is converted to a direction consistent with the direction. This can be done by inverting the first yaw angle. The range of the inverted first yaw angle is between -180° and 180°, the range of the forward pointing is between 90° and 180°, and the range of the reverse pointing is between -90° and -180°.
[0196] Procedure b: Update the second yaw angle based on the converted first yaw angle;
[0197] In some embodiments, after converting the first yaw angle, the second yaw angle is updated through the following steps:
[0198] Based on the converted first yaw angle and the attitude data in the current sensor data, an updated quaternion is generated; the updated quaternion is used to update the prediction and observation processes of the filtering algorithm to determine the updated second yaw angle.
[0199] In practice, after the first yaw angle is converted, a new quaternion is calculated using the converted first yaw angle, roll angle, and pitch angle. This new quaternion is used as the state variable of the second filtering model (in the extended Kalman filter algorithm). Position data and angular velocity are used as observations, and the prediction and observation processes are re-executed to obtain the updated second yaw angle.
[0200] Step c: Based on the updated second yaw angle and the sensor data, determine the pointing position corresponding to the sensor data on the display screen.
[0201] Since the UWB dual-sided array antenna and IMU have difficulty distinguishing between the forward and reverse orientation of the remote control, and both UWB and IMU can obtain normal data even when the orientation is reversed, this embodiment can use infrared data to calibrate the forward and reverse orientation of the pointing device (such as the remote control). In practice, the infrared data is used to determine whether the current pointing device is pointing forward or backward at the display screen. If the infrared data and the calculated first yaw angle are inconsistent, the first yaw angle can be reversed first, and then the second yaw angle can be updated using the reversed first yaw angle. Finally, the pointing position of the remote control is calculated using the updated second yaw angle, roll angle, and pitch angle.
[0202] As shown in Figure 8, this embodiment provides an implementation process for determining the direction using infrared data, as detailed below:
[0203] Step 800: Wait for Bluetooth connection to succeed upon initial power-on;
[0204] Step 801: Receive sensor data from the remote control;
[0205] Step 802: Determine whether infrared data is continuously received. If yes, proceed to step 803; otherwise, proceed to step 804.
[0206] Step 803: Determine if the first yaw angle is positive. If yes, return to step 801; otherwise, proceed to step 805.
[0207] Step 804: Determine if the first yaw angle is reversed. If yes, return to step 801; otherwise, proceed to step 805.
[0208] Step 805: Reverse the first yaw angle;
[0209] Step 806: Update the second yaw angle using the converted first yaw angle, and determine the pointing position using the updated second yaw angle, roll angle, and pitch angle.
[0210] The roll and pitch angles are obtained from attitude data collected by the IMU in the sensor data.
[0211] As shown in Figure 9, this embodiment provides a schematic diagram of a pointing system, including a controller and a remote control connected to the display screen. Both the controller and remote control have a Bluetooth BLE module and a Bluetooth antenna, a UWB and a UWB array antenna, and an infrared module. In use, the Bluetooth modules pair with each other for communication and data transmission, and the UWB modules pair with each other for UWB positioning data acquisition. The remote control collects sensor data from the IMU and UWB and reports it to the controller. The controller then processes and fuses the sensor data and sends HID reporting to the display screen via a USB interface, enabling mouse pointing, button, and keyboard functions.
[0212] As shown in Figure 10, this embodiment provides a specific implementation process for pointing remote control, as follows:
[0213] Step 1000: Receive Bluetooth data sent by the remote control, and execute steps 1001 and 1002d;
[0214] Optionally, Bluetooth data may include any one or more of the following: sensor data, button presses, control commands, etc.
[0215] Step 1001: Confirm sensor data update, then execute steps 1002a, 1002b, and 1002c;
[0216] Step 1002a: Update the quaternion based on the IMU's sensor data, and then execute steps 1003a and 1003b;
[0217] Step 1003a: Output pitch and roll angles;
[0218] Step 1003b: Output the first difference of the yaw angles collected at adjacent time points;
[0219] Step 1002b: Update the quaternion in the second filtering model based on the first difference and the sensor data, and then proceed to step 1004;
[0220] Step 1004: Output the second yaw angle;
[0221] Step 1002c: Update the quaternions in the first filtering model based on the sensor data, and then proceed to step 1005;
[0222] Step 1005: Output the first yaw angle;
[0223] Step 1006: Determine whether the second difference between the first yaw angle and the second yaw angle is greater than the threshold. If yes, proceed to step 1007; otherwise, proceed to step 1008.
[0224] Step 1007: Update the quaternion using pitch angle, roll angle and first yaw angle, and execute step 1002b;
[0225] Step 1008: Determine the pointing position based on the pitch angle, roll angle, and second yaw angle;
[0226] Step 1002d: Determine the direction of the pointing device based on the infrared data;
[0227] Step 1009: Determine if the first yaw angle and the direction are opposite. If yes, proceed to step 1010; otherwise, return to step 1002d.
[0228] Step 1010: Rotate the first yaw angle by 180 degrees;
[0229] Step 1011: Update the quaternion based on the pitch angle, roll angle, and the first yaw angle after rotation, and then proceed to step 1002c.
[0230] During implementation, after the system powers on, it waits for Bluetooth connection. Once the Bluetooth connection is successful, the remote controller begins reporting Bluetooth data to the controller on the display screen. Simultaneously, the controller begins infrared detection and uses the IMU and UWB sensor data reported via Bluetooth to maintain the algorithm. The quaternions under the IMU are updated using IMU data, and then the attitude angle and the first difference are obtained. The quaternions under the first filtering model are updated using UWB and IMU sensor data, outputting the current first yaw angle. Then, the quaternions under the second filtering model are updated using the first difference obtained from the IMU and the IMU and UWB sensor data. The yaw angles obtained from the quaternions under the two filtering models are compared (i.e., comparing the first yaw angle and the second yaw angle). If the yaw angle error is too large, a new quaternion is calculated using the first yaw angle under the first filtering model and the pitch and roll angles under the IMU, and this new quaternion is updated under the second filtering model. If the remote control's forward / reverse state determination, infrared determination, and first yaw angle determination are different, then the first yaw angle under the first filter model is deflected by 180 degrees, and then the quaternion is calculated with the pitch angle and roll angle under the IMU, and updated to the quaternion under the first filter model and the second filter model.
[0231] This embodiment enables seamless HID switching when the remote control loses and regains the UWB signal. It also allows for parameter switching based on the current situation during signal transitions, enabling rapid algorithm convergence at different stages. When using the IMU for yaw angle calculation, the relative yaw angle of the IMU is used as the UWB observation, improving the accuracy and stability of UWB yaw angle calculation. Furthermore, the use of an infrared module allows the entire system to quickly distinguish the forward and reverse directions of the remote control, thus enhancing the overall user experience of the directional remote control.
[0232] Based on the same inventive concept, this disclosure also provides a pointing remote control system. Since this system is the same as the system in the method of this disclosure, and the principle of solving the problem by this system is similar to that of this method, the implementation of this system can refer to the implementation of the method, and the repeated parts will not be described again.
[0233] As shown in Figure 11, the system includes a display 1100 and a remote control 1101, wherein:
[0234] The remote control 1101 is configured to collect data and transmit it to the display.
[0235] The display 1100 includes a display unit and a controller. The display unit is used to display content, and the controller is configured to perform the following steps:
[0236] Acquire sensor data sent by the pointing device, wherein the sensor data is collected using different types of sensors;
[0237] The first yaw angle is determined based on the sensor data and the filtering algorithm;
[0238] The first difference between the yaw angles collected at adjacent times in the sensor data is used as the observation in the filtering algorithm to constrain the error of the first yaw angle and obtain the second yaw angle.
[0239] Based on the second yaw angle and the sensor data, determine the pointing position corresponding to the sensor data on the display screen.
[0240] As an optional implementation, the sensing data includes attitude data, position data, and motion data; the controller is specifically configured to execute:
[0241] The process of predicting state variables is established based on the filtering algorithm and control variables, and the process of obtaining observation variables from state variables is established, wherein the control variables represent motion data, the state variables represent attitude data, and the observation variables represent position data.
[0242] The first yaw angle is determined based on the iterative updates during the prediction and observation processes.
[0243] As an optional implementation, the controller is specifically configured to determine the second yaw angle in the following manner:
[0244] The second yaw angle is determined based on the sensor data, the first difference, and the filtering algorithm.
[0245] As an optional implementation, the controller is specifically configured to perform:
[0246] Based on the second yaw angle and the attitude data in the sensing data, the pointing position on the display screen corresponding to the sensing data is determined.
[0247] As an optional implementation, the controller is specifically configured to perform:
[0248] Determine the second difference between the first yaw angle and the second yaw angle;
[0249] In response to the second difference being less than or equal to a threshold, the pointing position on the display screen corresponding to the sensing data is determined based on the second yaw angle and the attitude data in the sensing data.
[0250] As an optional implementation, the sensing data includes angular velocity; the controller is further configured to perform:
[0251] Count the total number of consecutive angular velocities that exceed the velocity threshold;
[0252] In response to the total number being greater than a first set value, the pointing position on the display screen corresponding to the sensing data is determined based on the second yaw angle and the attitude data in the sensing data.
[0253] As an optional implementation, the controller is further configured to perform:
[0254] In response to the second difference being greater than a threshold, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data until the second difference between the first yaw angle and the updated second yaw angle is less than or equal to the threshold, at which point the update stops.
[0255] As an optional implementation, the controller is specifically configured to perform:
[0256] An updated quaternion is generated based on the first yaw angle and the attitude data;
[0257] The prediction and observation processes of the filtering algorithm are updated using the updated quaternions to determine the updated second yaw angle.
[0258] As an optional implementation, the controller is specifically configured to perform:
[0259] The second difference between the first yaw angle and the second yaw angle is determined based on multiple collected sensor data.
[0260] Count the total number of second differences that are consecutively greater than the threshold;
[0261] In response to the total number being greater than a second set value, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data.
[0262] As an optional implementation, the controller is further configured to perform:
[0263] In response to an update of the current pointing position, and at least one of the previous n pointing positions being outside the display screen range, the reporting data is determined based on the current pointing position, where n is an integer greater than 0.
[0264] As an optional implementation, the controller is further configured to perform:
[0265] In response to an update of the current pointing position, and provided that the pointing position in the previous n updates did not exceed the display screen range, the reporting data is determined based on the current pointing position, the previous pointing position, and the current angular velocity, where n is an integer greater than 0.
[0266] As an optional implementation, the controller is specifically configured to perform:
[0267] The current reporting data is determined based on the current pointing position, the previous reporting data is determined based on the previous pointing position, and the compensation value is determined based on the current angular velocity.
[0268] The current reporting data, the previous reporting data, and the compensation value are weighted and summed according to their respective compensation coefficients to obtain the reporting data.
[0269] As an optional implementation, the controller is further configured to perform:
[0270] Based on the difference between the current reporting data and the previous reporting data, and the current angular velocity, determine the compensation coefficients for the current reporting data, the previous reporting data, and the compensation value.
[0271] As an optional implementation, the controller is further configured to perform:
[0272] If the reported point difference changes in a direct proportional manner with the current angular velocity, then the compensation coefficient of the current reported point data is increased.
[0273] If the reported point difference changes inversely with the current angular velocity, and the degree of change in the reported point data is greater than that of the current angular velocity, then the compensation coefficient of the previous reported point data is increased.
[0274] If the reported point difference changes inversely with the current angular velocity, and the degree of change in the reported point data is less than that of the current angular velocity, then the compensation coefficient of the compensation value is increased.
[0275] As an optional implementation, the sensing data includes infrared data; the controller is further configured to perform:
[0276] The direction of the pointing device is determined based on infrared data, and the first yaw angle is converted into a direction consistent with the stated direction.
[0277] The second yaw angle is updated based on the converted first yaw angle;
[0278] Based on the updated second yaw angle and the sensor data, the pointing position corresponding to the sensor data on the display screen is determined.
[0279] As an optional implementation, the controller is specifically configured to perform:
[0280] In response to receiving infrared data multiple times in succession, the pointing device is determined to be pointing in the positive direction;
[0281] If no infrared data is received at least once in a series of consecutive tests, the pointing device is determined to be pointing in the opposite direction.
[0282] As an optional implementation, the controller is specifically configured to perform:
[0283] Based on the converted first yaw angle and the attitude data in the current sensor data, an updated quaternion is generated;
[0284] The prediction and observation processes of the filtering algorithm are updated using the updated quaternions to determine the updated second yaw angle.
[0285] Based on the same inventive concept, this disclosure also provides a remote control device. Since this remote control device is the same as the remote control device in the method of this disclosure, and the principle of solving the problem by the remote control device is similar to that of the method, the implementation of the remote control device can refer to the implementation of the method, and the repeated parts will not be described again.
[0286] As shown in Figure 12, the remote control device includes a control unit 1200 and a remote control unit 1201, wherein:
[0287] The remote control unit 1201 is configured to collect data and transmit it to the display.
[0288] The control unit 1200 includes a processor and a memory. The memory stores programs executable by the processor, and the processor reads the programs from the memory and performs the following steps:
[0289] Acquire sensor data sent by the pointing device, wherein the sensor data is collected using different types of sensors;
[0290] The first yaw angle is determined based on the sensor data and the filtering algorithm;
[0291] The first difference between the yaw angles collected at adjacent times in the sensor data is used as the observation in the filtering algorithm to constrain the error of the first yaw angle and obtain the second yaw angle.
[0292] Based on the second yaw angle and the sensor data, determine the pointing position corresponding to the sensor data on the display screen.
[0293] As an optional implementation, the sensing data includes attitude data, position data, and motion data; the processor is specifically configured to execute:
[0294] The process of predicting state variables is established based on the filtering algorithm and control variables, and the process of obtaining observation variables from state variables is established, wherein the control variables represent motion data, the state variables represent attitude data, and the observation variables represent position data.
[0295] The first yaw angle is determined based on the iterative updates during the prediction and observation processes.
[0296] As an optional implementation, the processor is specifically configured to determine the second yaw angle in the following manner:
[0297] The second yaw angle is determined based on the sensor data, the first difference, and the filtering algorithm.
[0298] As an optional implementation, the processor is specifically configured to execute:
[0299] Based on the second yaw angle and the attitude data in the sensing data, the pointing position on the display screen corresponding to the sensing data is determined.
[0300] As an optional implementation, the processor is specifically configured to execute:
[0301] Determine the second difference between the first yaw angle and the second yaw angle;
[0302] In response to the second difference being less than or equal to a threshold, the pointing position on the display screen corresponding to the sensing data is determined based on the second yaw angle and the attitude data in the sensing data.
[0303] As an optional implementation, the sensing data includes angular velocity; the processor is further configured to execute:
[0304] Count the total number of consecutive angular velocities that exceed the velocity threshold;
[0305] In response to the total number being greater than a first set value, the pointing position on the display screen corresponding to the sensing data is determined based on the second yaw angle and the attitude data in the sensing data.
[0306] As an optional implementation, the processor is further configured to execute:
[0307] In response to the second difference being greater than a threshold, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data until the second difference between the first yaw angle and the updated second yaw angle is less than or equal to the threshold, at which point the update stops.
[0308] As an optional implementation, the processor is specifically configured to execute:
[0309] An updated quaternion is generated based on the first yaw angle and the attitude data;
[0310] The prediction and observation processes of the filtering algorithm are updated using the updated quaternions to determine the updated second yaw angle.
[0311] As an optional implementation, the processor is specifically configured to execute:
[0312] The second difference between the first yaw angle and the second yaw angle is determined based on multiple collected sensor data.
[0313] Count the total number of second differences that are consecutively greater than the threshold;
[0314] In response to the total number being greater than a second set value, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data.
[0315] As an optional implementation, the processor is further configured to execute:
[0316] In response to an update of the current pointing position, and at least one of the previous n pointing positions being outside the display screen range, the reporting data is determined based on the current pointing position, where n is an integer greater than 0.
[0317] As an optional implementation, the processor is further configured to execute:
[0318] In response to an update of the current pointing position, and provided that the pointing position in the previous n updates did not exceed the display screen range, the reporting data is determined based on the current pointing position, the previous pointing position, and the current angular velocity, where n is an integer greater than 0.
[0319] As an optional implementation, the processor is specifically configured to execute:
[0320] The current reporting data is determined based on the current pointing position, the previous reporting data is determined based on the previous pointing position, and the compensation value is determined based on the current angular velocity.
[0321] The current reporting data, the previous reporting data, and the compensation value are weighted and summed according to their respective compensation coefficients to obtain the reporting data.
[0322] As an optional implementation, the processor is further configured to execute:
[0323] Based on the difference between the current reporting data and the previous reporting data, and the current angular velocity, determine the compensation coefficients for the current reporting data, the previous reporting data, and the compensation value.
[0324] As an optional implementation, the processor is further configured to execute:
[0325] If the reported point difference changes in a direct proportional manner with the current angular velocity, then the compensation coefficient of the current reported point data is increased.
[0326] If the reported point difference changes inversely with the current angular velocity, and the degree of change in the reported point data is greater than that of the current angular velocity, then the compensation coefficient of the previous reported point data is increased.
[0327] If the reported point difference changes inversely with the current angular velocity, and the degree of change in the reported point data is less than that of the current angular velocity, then the compensation coefficient of the compensation value is increased.
[0328] As an optional implementation, the sensing data includes infrared data; the processor is further configured to execute:
[0329] The direction of the pointing device is determined based on infrared data, and the first yaw angle is converted into a direction consistent with the stated direction.
[0330] The second yaw angle is updated based on the converted first yaw angle;
[0331] Based on the updated second yaw angle and the sensor data, the pointing position corresponding to the sensor data on the display screen is determined.
[0332] As an optional implementation, the processor is specifically configured to execute:
[0333] In response to receiving infrared data multiple times in succession, the pointing device is determined to be pointing in the positive direction;
[0334] If no infrared data is received at least once in a series of consecutive tests, the pointing device is determined to be pointing in the opposite direction.
[0335] As an optional implementation, the processor is specifically configured to execute:
[0336] Based on the converted first yaw angle and the attitude data in the current sensor data, an updated quaternion is generated;
[0337] The prediction and observation processes of the filtering algorithm are updated using the updated quaternions to determine the updated second yaw angle.
[0338] Based on the same inventive concept, this disclosure also provides an electronic device. Since this electronic device is the same as the electronic device in the method of this disclosure, and the principle of solving the problem by this electronic device is similar to that of this method, the implementation of this electronic device can refer to the implementation of the method, and the repeated parts will not be described again.
[0339] As shown in Figure 13, the electronic device includes a processor 1300 and a memory 1301. The memory 1301 is used to store programs executable by the processor 1300. The processor 1300 is used to read the programs in the memory 1301 and perform the following steps:
[0340] Acquire sensor data sent by the pointing device, wherein the sensor data is collected using different types of sensors;
[0341] The first yaw angle is determined based on the sensor data and the filtering algorithm;
[0342] The first difference between the yaw angles collected at adjacent times in the sensor data is used as the observation in the filtering algorithm to constrain the error of the first yaw angle and obtain the second yaw angle.
[0343] Based on the second yaw angle and the sensor data, determine the pointing position corresponding to the sensor data on the display screen.
[0344] As an optional implementation, the sensing data includes attitude data, position data, and motion data; the processor 1300 is specifically configured to execute:
[0345] The process of predicting state variables is established based on the filtering algorithm and control variables, and the process of obtaining observation variables from state variables is established, wherein the control variables represent motion data, the state variables represent attitude data, and the observation variables represent position data.
[0346] The first yaw angle is determined based on the iterative updates during the prediction and observation processes.
[0347] As an optional implementation, the processor 1300 is specifically configured to determine the second yaw angle in the following manner:
[0348] The second yaw angle is determined based on the sensor data, the first difference, and the filtering algorithm.
[0349] As an optional implementation, the processor 1300 is specifically configured to execute:
[0350] Based on the second yaw angle and the attitude data in the sensing data, the pointing position on the display screen corresponding to the sensing data is determined.
[0351] As an optional implementation, the processor 1300 is specifically configured to execute:
[0352] Determine the second difference between the first yaw angle and the second yaw angle;
[0353] In response to the second difference being less than or equal to a threshold, the pointing position on the display screen corresponding to the sensing data is determined based on the second yaw angle and the attitude data in the sensing data.
[0354] As an optional implementation, the sensing data includes angular velocity; the processor 1300 is further configured to execute:
[0355] Count the total number of consecutive angular velocities that exceed the velocity threshold;
[0356] In response to the total number being greater than a first set value, the pointing position on the display screen corresponding to the sensing data is determined based on the second yaw angle and the attitude data in the sensing data.
[0357] As an optional implementation, the processor 1300 is further configured to perform:
[0358] In response to the second difference being greater than a threshold, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data until the second difference between the first yaw angle and the updated second yaw angle is less than or equal to the threshold, at which point the update stops.
[0359] As an optional implementation, the processor 1300 is specifically configured to execute:
[0360] An updated quaternion is generated based on the first yaw angle and the attitude data;
[0361] The prediction and observation processes of the filtering algorithm are updated using the updated quaternions to determine the updated second yaw angle.
[0362] As an optional implementation, the processor 1300 is specifically configured to execute:
[0363] The second difference between the first yaw angle and the second yaw angle is determined based on multiple collected sensor data.
[0364] Count the total number of second differences that are consecutively greater than the threshold;
[0365] In response to the total number being greater than a second set value, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data.
[0366] As an optional implementation, the processor 1300 is further configured to perform:
[0367] In response to an update of the current pointing position, and at least one of the previous n pointing positions being outside the display screen range, the reporting data is determined based on the current pointing position, where n is an integer greater than 0.
[0368] As an optional implementation, the processor 1300 is further configured to perform:
[0369] In response to an update of the current pointing position, and provided that the pointing position in the previous n updates did not exceed the display screen range, the reporting data is determined based on the current pointing position, the previous pointing position, and the current angular velocity, where n is an integer greater than 0.
[0370] As an optional implementation, the processor 1300 is specifically configured to execute:
[0371] The current reporting data is determined based on the current pointing position, the previous reporting data is determined based on the previous pointing position, and the compensation value is determined based on the current angular velocity.
[0372] The current reporting data, the previous reporting data, and the compensation value are weighted and summed according to their respective compensation coefficients to obtain the reporting data.
[0373] As an optional implementation, the processor 1300 is further configured to perform:
[0374] Based on the difference between the current reporting data and the previous reporting data, and the current angular velocity, determine the compensation coefficients for the current reporting data, the previous reporting data, and the compensation value.
[0375] As an optional implementation, the processor 1300 is further configured to perform:
[0376] If the reported point difference changes in a direct proportional manner with the current angular velocity, then the compensation coefficient of the current reported point data is increased.
[0377] If the reported point difference changes inversely with the current angular velocity, and the degree of change in the reported point data is greater than that of the current angular velocity, then the compensation coefficient of the previous reported point data is increased.
[0378] If the reported point difference changes inversely with the current angular velocity, and the degree of change in the reported point data is less than that of the current angular velocity, then the compensation coefficient of the compensation value is increased.
[0379] As an optional implementation, the sensing data includes infrared data; the processor 1300 is further configured to execute:
[0380] The direction of the pointing device is determined based on infrared data, and the first yaw angle is converted into a direction consistent with the stated direction.
[0381] The second yaw angle is updated based on the converted first yaw angle;
[0382] Based on the updated second yaw angle and the sensor data, the pointing position corresponding to the sensor data on the display screen is determined.
[0383] As an optional implementation, the processor 1300 is specifically configured to execute:
[0384] In response to receiving infrared data multiple times in succession, the pointing device is determined to be pointing in the positive direction;
[0385] If no infrared data is received at least once in a series of consecutive tests, the pointing device is determined to be pointing in the opposite direction.
[0386] As an optional implementation, the processor 1300 is specifically configured to execute:
[0387] Based on the converted first yaw angle and the attitude data in the current sensor data, an updated quaternion is generated;
[0388] The prediction and observation processes of the filtering algorithm are updated using the updated quaternions to determine the updated second yaw angle.
[0389] Based on the same inventive concept, this disclosure also provides a pointing remote control device. Since this device is the same as the device in the method of this disclosure, and the principle of the device in solving the problem is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.
[0390] As shown in Figure 14, the device includes:
[0391] The data acquisition unit 1400 is used to acquire sensor data sent by the pointing device, wherein the sensor data is collected using different types of sensors;
[0392] The first calculation unit 1401 is used to determine the first yaw angle based on the sensor data and the filtering algorithm;
[0393] The second calculation unit 1402 is used to take the first difference of the yaw angle collected at adjacent times in the sensing data as the observation in the filtering algorithm, constrain the error of the first yaw angle, and obtain the second yaw angle.
[0394] The position determination unit 1403 is used to determine the pointing position corresponding to the sensing data on the display screen based on the second yaw angle and the sensing data.
[0395] As an optional implementation, the sensing data includes attitude data, position data, and motion data; the first calculation unit 1401 is specifically used for:
[0396] The process of predicting state variables is established based on the filtering algorithm and control variables, and the process of obtaining observation variables from state variables is established, wherein the control variables represent motion data, the state variables represent attitude data, and the observation variables represent position data.
[0397] The first yaw angle is determined based on the iterative updates during the prediction and observation processes.
[0398] As an optional implementation, the second calculation unit 1402 is specifically used to determine the second yaw angle in the following manner:
[0399] The second yaw angle is determined based on the sensor data, the first difference, and the filtering algorithm.
[0400] As an optional implementation, the position determination unit 1403 is specifically used for:
[0401] Based on the second yaw angle and the attitude data in the sensing data, the pointing position on the display screen corresponding to the sensing data is determined.
[0402] As an optional implementation, the position determination unit 1403 is specifically used for:
[0403] Determine the second difference between the first yaw angle and the second yaw angle;
[0404] In response to the second difference being less than or equal to a threshold, the pointing position on the display screen corresponding to the sensing data is determined based on the second yaw angle and the attitude data in the sensing data.
[0405] As an optional implementation, the sensing data includes angular velocity; the position determination unit 1403 is further configured to:
[0406] Count the total number of consecutive angular velocities that exceed the velocity threshold;
[0407] In response to the total number being greater than a first set value, the pointing position on the display screen corresponding to the sensing data is determined based on the second yaw angle and the attitude data in the sensing data.
[0408] As an optional implementation, the position determination unit 1403 is further configured to:
[0409] In response to the second difference being greater than a threshold, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data until the second difference between the first yaw angle and the updated second yaw angle is less than or equal to the threshold, at which point the update stops.
[0410] As an optional implementation, the position determination unit 1403 is specifically used for:
[0411] An updated quaternion is generated based on the first yaw angle and the attitude data;
[0412] The prediction and observation processes of the filtering algorithm are updated using the updated quaternions to determine the updated second yaw angle.
[0413] As an optional implementation, the position determination unit 1403 is specifically used for:
[0414] The second difference between the first yaw angle and the second yaw angle is determined based on multiple collected sensor data.
[0415] Count the total number of second differences that are consecutively greater than the threshold;
[0416] In response to the total number being greater than a second set value, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data.
[0417] As an optional implementation, the position determination unit 1403 is further configured to:
[0418] In response to an update of the current pointing position, and at least one of the previous n pointing positions being outside the display screen range, the reporting data is determined based on the current pointing position, where n is an integer greater than 0.
[0419] As an optional implementation, the position determination unit 1403 is further configured to:
[0420] In response to an update of the current pointing position, and provided that the pointing position in the previous n updates did not exceed the display screen range, the reporting data is determined based on the current pointing position, the previous pointing position, and the current angular velocity, where n is an integer greater than 0.
[0421] As an optional implementation, the position determination unit 1403 is specifically used for:
[0422] The current reporting data is determined based on the current pointing position, the previous reporting data is determined based on the previous pointing position, and the compensation value is determined based on the current angular velocity.
[0423] The current reporting data, the previous reporting data, and the compensation value are weighted and summed according to their respective compensation coefficients to obtain the reporting data.
[0424] As an optional implementation, a coefficient determination unit is also included, specifically used for:
[0425] Based on the difference between the current reporting data and the previous reporting data, and the current angular velocity, determine the compensation coefficients for the current reporting data, the previous reporting data, and the compensation value.
[0426] As an optional implementation, the coefficient determining unit is further configured to:
[0427] If the reported point difference changes in a direct proportional manner with the current angular velocity, then the compensation coefficient of the current reported point data is increased.
[0428] If the reported point difference changes inversely with the current angular velocity, and the degree of change in the reported point data is greater than that of the current angular velocity, then the compensation coefficient of the previous reported point data is increased.
[0429] If the reported point difference changes inversely with the current angular velocity, and the degree of change in the reported point data is less than that of the current angular velocity, then the compensation coefficient of the compensation value is increased.
[0430] As an optional implementation, the sensing data includes infrared data; the position determination unit 1403 is further configured to:
[0431] The direction of the pointing device is determined based on infrared data, and the first yaw angle is converted into a direction consistent with the stated direction.
[0432] The second yaw angle is updated based on the converted first yaw angle;
[0433] Based on the updated second yaw angle and the sensor data, the pointing position corresponding to the sensor data on the display screen is determined.
[0434] As an optional implementation, the position determination unit 1403 is specifically used for:
[0435] In response to receiving infrared data multiple times in succession, the pointing device is determined to be pointing in the positive direction;
[0436] If no infrared data is received at least once in a series of consecutive tests, the pointing device is determined to be pointing in the opposite direction.
[0437] As an optional implementation, the position determination unit 1403 is specifically used for:
[0438] Based on the converted first yaw angle and the attitude data in the current sensor data, an updated quaternion is generated;
[0439] The prediction and observation processes of the filtering algorithm are updated using the updated quaternions to determine the updated second yaw angle.
[0440] Based on the same inventive concept, this disclosure provides a computer storage medium comprising: computer program code, which, when executed on a computer, causes the computer to perform any of the pointing remote control methods described above. Since the principle by which the computer storage medium solves the problem is similar to that of the pointing remote control method, the implementation of the computer storage medium can be referred to the implementation of the method, and repeated details will not be elaborated further.
[0441] In specific implementation, computer storage media can include: Universal Serial Bus Flash Drive (USB), portable hard drive, Read-Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, and other storage media that can store program code.
[0442] Based on the same inventive concept, this disclosure also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the pointing remote control methods discussed above. Since the principle by which the above-described computer program product solves the problem is similar to that of the pointing remote control method, the implementation of the above-described computer program product can be referred to the implementation of the method, and repeated details will not be described again.
[0443] Computer program products may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0444] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.
[0445] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.
[0446] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction device that implements the functions specified in one or more flowcharts and / or one or more block diagrams.
[0447] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.
[0448] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.
Claims
1. A pointing remote control method, wherein, The method includes: Acquire sensor data sent by the pointing device, wherein the sensor data is collected using different types of sensors; The first yaw angle is determined based on the sensor data and the filtering algorithm; The first difference between the yaw angles collected at adjacent times in the sensor data is used as the observation in the filtering algorithm to constrain the error of the first yaw angle and obtain the second yaw angle. Based on the second yaw angle and the sensor data, determine the pointing position corresponding to the sensor data on the display screen.
2. The method of claim 1, wherein, The sensing data includes attitude data, position data, and motion data; determining the first yaw angle based on the sensing data and a filtering algorithm includes: The process of predicting state variables is established based on the filtering algorithm and control variables, and the process of obtaining observation variables from state variables is established, wherein the control variables represent motion data, the state variables represent attitude data, and the observation variables represent position data. The first yaw angle is determined based on the iterative updates during the prediction and observation processes.
3. The method of claim 1, wherein, The second yaw angle is determined as follows: The second yaw angle is determined based on the sensor data, the first difference, and the filtering algorithm.
4. The method of claim 1, wherein, The step of determining the pointing position corresponding to the sensing data on the display screen based on the second yaw angle and the sensing data includes: Based on the second yaw angle and the attitude data in the sensing data, the pointing position on the display screen corresponding to the sensing data is determined.
5. The method of claim 4, wherein, The step of determining the pointing position on the display screen corresponding to the sensing data based on the second yaw angle and the attitude data in the sensing data includes: Determine the second difference between the first yaw angle and the second yaw angle; In response to the second difference being less than or equal to a threshold, the pointing position on the display screen corresponding to the sensing data is determined based on the second yaw angle and the attitude data in the sensing data.
6. The method of claim 5, wherein, The sensing data includes angular velocity; the method further includes: Count the total number of consecutive angular velocities that exceed the velocity threshold; In response to the total number being greater than a first set value, the pointing position on the display screen corresponding to the sensing data is determined based on the second yaw angle and the attitude data in the sensing data.
7. The method of claim 5, wherein, The method also includes: In response to the second difference being greater than a threshold, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data until the second difference between the first yaw angle and the updated second yaw angle is less than or equal to the threshold, at which point the update stops.
8. The method of claim 7, wherein, The step of iteratively updating the second yaw angle based on the first yaw angle and the attitude data includes: An updated quaternion is generated based on the first yaw angle and the attitude data; The prediction and observation processes of the filtering algorithm are updated using the updated quaternions to determine the updated second yaw angle.
9. The method of claim 7, wherein, In response to the second difference being greater than a threshold, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data, including: The second difference between the first yaw angle and the second yaw angle is determined based on multiple collected sensor data. Count the total number of second differences that are consecutively greater than the threshold; In response to the total number being greater than a second set value, the second yaw angle is iteratively updated based on the first yaw angle and the attitude data.
10. The method of claim 1, wherein, The method also includes: In response to an update of the current pointing position, and at least one of the previous n pointing positions being outside the display screen range, the reporting data is determined based on the current pointing position, where n is an integer greater than 0.
11. The method of claim 1, wherein, The method also includes: In response to an update of the current pointing position, and provided that the pointing position in the previous n updates did not exceed the display screen range, the reporting data is determined based on the current pointing position, the previous pointing position, and the current angular velocity, where n is an integer greater than 0.
12. The method of claim 11, wherein, The process of determining the reporting data based on the current pointing position, the previous pointing position, and the current angular velocity includes: The current reporting data is determined based on the current pointing position, the previous reporting data is determined based on the previous pointing position, and the compensation value is determined based on the current angular velocity. The current reporting data, the previous reporting data, and the compensation value are weighted and summed according to their respective compensation coefficients to obtain the reporting data.
13. The method of claim 12, wherein, The method also includes: Based on the difference between the current reporting data and the previous reporting data, and the current angular velocity, determine the compensation coefficients for the current reporting data, the previous reporting data, and the compensation value.
14. The method of claim 13, wherein, The method also includes: If the reported point difference changes in a direct proportional manner with the current angular velocity, then the compensation coefficient of the current reported point data is increased. If the reported point difference changes inversely with the current angular velocity, and the degree of change in the reported point data is greater than that of the current angular velocity, then the compensation coefficient of the previous reported point data is increased. If the reported point difference changes inversely to the current angular velocity, and the change in the reported point data is less than that of the current angular velocity, then the compensation coefficient of the compensation value is increased.
15. The method of claim 1, wherein, The sensing data includes infrared data; the method further includes: The direction of the pointing device is determined based on infrared data, and the first yaw angle is converted into a direction consistent with the stated direction. The second yaw angle is updated based on the converted first yaw angle; Based on the updated second yaw angle and the sensor data, the pointing position corresponding to the sensor data on the display screen is determined.
16. The method of claim 15, wherein, The step of determining the direction of the pointing device based on infrared data includes: In response to receiving infrared data multiple times in succession, the pointing device is determined to be pointing in the positive direction; If no infrared data is received at least once in a series of consecutive tests, the pointing device is determined to be pointing in the opposite direction.
17. The method of claim 15, wherein, The step of updating the second yaw angle based on the converted first yaw angle includes: Based on the converted first yaw angle and the attitude data in the current sensor data, an updated quaternion is generated; The prediction and observation processes of the filtering algorithm are updated using the updated quaternions to determine the updated second yaw angle.
18. A pointing remote control system, wherein, Includes a monitor and a remote control, wherein: The remote control is configured to collect data and transmit it to the display. The display includes a display unit and a controller, the display unit being used to display content, and the controller being configured to perform the steps of the method according to any one of claims 1 to 17.
19. A remote control device, wherein, Includes a control unit and a remote control unit, wherein: The remote control unit is configured to collect data and transmit it to the display. The control unit includes a processor and a memory, the memory being used to store a program executable by the processor, and the processor being used to read the program in the memory and execute the steps of the method according to any one of claims 1 to 17.
20. An electronic device, comprising: It includes a processor and a memory, the memory being used to store a program executable by the processor, and the processor being used to read the program in the memory and execute the steps of the method according to any one of claims 1 to 17.
21. A computer storage medium having stored thereon a computer program, wherein, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1 to 17.
22. A computer program product, wherein, The computer program product includes: computer program code, which, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 17.
