Rapid trapezoidal adjustment method and device based on IMU and TOF and medium thereof
By combining the data of IMU and TOF sensors, real-time detection of the attitude and yaw angle of the projection equipment, the problem of insufficient trapezoidal adjustment speed and accuracy in traditional methods is solved, and fast and accurate trapezoidal correction is achieved, which improves the quality of projection display and User experience.
Patent Information
- Application Number
- CN202411572012.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-06
- Publication Date
- 2024-12-03
AI Technical Summary
The traditional trapezoidal adjustment method has shortcomings in processing speed and accuracy. IMU has limitations in estimating yaw angles. The data rate of the TOF sensor is low, making it difficult to meet the needs of real-time picture follow-up.
Combined with the accelerometer and gyroscope built in the IMU module, the attitude changes of the projection equipment are detected in real time, the yaw angle is estimated through complementary filtering algorithms, the TOF sensor is used for accurate ranging, and the data of the IMU and TOF modules are fused to real-time application in trapezoids. Correction function.
It realizes fast and accurate trapezoidal adjustment, improves the quality and user experience of projection display, and has the advantages of low cost and fast and accurate follow-up.
Smart Images

Figure CN119071466A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a fast trapezoidal adjustment method, a device and a medium thereof based on IMU and TOF. Background Art
[0002] In projection display devices, keystone distortion is a common problem, which is mainly caused by the deviation of the projection angle or the unevenness of the projection surface. Traditional keystone adjustment methods mainly rely on image processing algorithms, IMU gravity sensors or TOF sensors to correct keystone distortion. Although these methods can correct keystone distortion to a certain extent, there are certain problems in processing speed and accuracy: 1. Traditional trapezoidal adjustment methods mainly rely on image processing algorithms, which are usually highly computationally complex, resulting in slow processing speeds and unable to meet the needs of real-time adjustment. At the same time, due to the limitations of the hardware performance of the projection equipment and the algorithm itself, the accuracy of image processing is often difficult to achieve the desired effect; 2. The gravity sensor built into the IMU (Inertial Measurement Unit) can accurately and quickly provide the attitude information of the projection device, including pitch and roll angles. However, the IMU has great limitations in estimating the yaw angle. The yaw angle describes the angle of rotation of the device around the vertical axis. Since the IMU does not have a sensor that directly measures the yaw angle, the yaw angle can only be estimated by integrating the gyroscope data. This method not only requires an initial reference point to set the initial yaw angle, but the error generated during the integration process will accumulate over time, causing the estimation accuracy of the yaw angle to gradually decrease; 3. The TOF (time of flight) sensor estimates the distance by measuring the time from the emission to the reception of light. Combining the data of multiple TOF sensors, the yaw angle of the device can be calculated. The TOF sensor has high accuracy and stability in providing yaw angle information, but its data rate is low, usually around 10Hz. This means that in the same amount of time, the TOF sensor collects fewer data points, which is difficult to meet the needs of real-time picture tracking; Therefore, a fast trapezoidal adjustment method, device and medium based on IMU and TOF are proposed. Summary of the invention
[0003] In view of this, the embodiments of the present invention hope to provide a fast trapezoidal adjustment method, device and medium based on IMU and TOF to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial option.
[0004] In order to solve the above technical problems, a technical solution adopted in this application is: a fast trapezoidal adjustment method based on IMU and TOF, comprising the following steps: Step 1: Use the built-in accelerometer and gyroscope of the IMU module to detect the posture changes of the projection device in real time, including the pitch angle and roll angle; Step 2: Calculate the pitch angle and roll angle according to the direction of gravity acceleration through accelerometer data to sense the posture of the projection device; Step 3: Use the gyroscope to measure the angular velocity of the device in each axis, integrate these angular velocities to obtain the rotation angle of the device in each direction, and use the complementary filtering algorithm to estimate the yaw angle; Step 4: Use the two areas in the horizontal direction of the TOF sensor to measure the distance, and combine the FOV information of the two areas to obtain two direction vectors; Step 5: Obtain the pitch angle and roll angle of the device according to the IMU, as well as the ranging information of the two areas in the horizontal direction of TOF and the FOV angle information of the two areas, and calculate the precise yaw angle; Step 6: Synchronize the precise yaw angle obtained by the stabilized TOF to the IMU following module, and fuse the attitude data and yaw angle data obtained by the IMU and TOF modules; Step 7: Apply the fused data to the keystone correction function in real time, and adjust the projected image by software or hardware to make it present a standard rectangular shape.
[0005] As a further preferred embodiment of the present technical solution, in step 2, the pitch angle pitch is calculated by measuring the components of gravity acceleration on the Y-axis and Z-axis of the device.
[0006] The calculation formula for the pitch angle is: ; in, and are the gravitational acceleration components measured by the accelerometer on the Y-axis and Z-axis respectively.
[0007] The roll angle roll is calculated by measuring the components of gravity acceleration on the X-axis and Z-axis of the device.
[0008] The calculation formula of the roll angle roll is: ; in, and are the gravitational acceleration components measured by the accelerometer on the X-axis and Z-axis respectively.
[0009] As a further preferred embodiment of the present technical solution, in step three, the gyroscope measurement values gx, gy, gz, and error terms ex, ey, ez and their integral terms exInt, eyInt, ezInt are defined, the quaternions q0, q1, q2, q3 are initialized, and the noise parameters Ki of the gyroscope and accelerometer are set.
[0010] Using the current quaternions q0, q1, q2, q3, the formula for estimating the gravity directions vc, vy, and vz is: ; ; ; The formula for integrating the error term based on the time step dt, the error term, and the integral gain Ki is: ; ; ; Using the gyroscope measurement and the integral term of the error term, the formula for adjusting the gyroscope measurement is: ; ; ; ; The formula for calculating the yaw angle using the updated quaternion is: ; in, , , , is the updated quaternion.
[0011] As a further preferred embodiment of the present technical solution, in step 5, the TOF two-region ranging model is defined, and the FOV angle , D1 and D2 correspond to the distances between the two TOF regions, and the direction vectors of D1 and D2 are vectors and .
[0012] Rotate according to the Euler angle yaw→pitch→roll order, and you can get the rotated vectors are and ,in: Rotate around the z axis The rotation matrix of (yaw) is: ; Rotate around the y axis The rotation matrix of (roll) is: ; Rotate around the x-axis The rotation matrix of (pitch) is: ; Calculate the rotated vector according to the rotation formula and The calculation formula is as follows: ; ; According to the rotation of D1 and D2, and and Using analytical geometry to infer the yaw angle yaw ( ); Specifically: Let the vector corresponding to D1 after rotation be ; The vector corresponding to D2 ; According to the dot product formula of vectors: ; in, is the angle between two vectors, for vector and ,have: ; Because the distances between D1 and D2 and their corresponding FOV angle information are known, we can get The expression of yaw angle yaw is established (here yaw is reflected in the rotation matrix ) equation; For example, if the distance between D1 and D2 is known to be , the FOV angle corresponding to D1 is , the FOV angle corresponding to D2 is , through some geometric derivation (the complicated derivation process is omitted here because it involves detailed analysis based on specific geometric figures and known conditions), we can get: ; in, and It can be further deduced based on the distance and angle relationship between D1 and D2. The formula here is just a schematic one.
[0013] Change the formula: ; ; Substituting into the formula: ; Get an equation about the yaw angle, and then solve the equation by numerical calculation method or further algebraic simplification to get the value of the yaw angle; For example, if we simplify the equation: ; Here A, B, and C are coefficients obtained based on the previous calculations and known conditions. The auxiliary angle formula can be used: ; Transform it into:
[0014] in, ; Then solve for The value of is the value of the yaw angle yaw.
[0015] As a further preferred embodiment of the present technical solution, in step one, the IMU module also includes a temperature sensor for detecting the temperature of the projection device in real time, and performing temperature compensation on the measurement data of the accelerometer and gyroscope of the IMU module according to the temperature information, so as to improve the accuracy of posture detection.
[0016] As a further preferred embodiment of the present technical solution, in step seven, the trapezoidal correction function is implemented using a built-in processor of the projection device or an externally connected computer, and a linear interpolation or bilinear interpolation algorithm is used during the correction process to correct the smoothness and accuracy of the image.
[0017] As a further preferred embodiment of the present technical solution, in step four, the TOF sensor is also used to detect the distance between the projection device and the projection surface, and the distance information is used to assist in calculating the degree of trapezoidal distortion of the projected image.
[0018] As a further preferred embodiment of the present technical solution, in step six, when fusing the attitude data and yaw angle data obtained by the IMU and TOF modules, a Kalman filter or an extended Kalman filter is used for fusion.
[0019] To solve the above technical problems, another technical solution adopted by the present application is: a computer device, The computer device includes: a processor, and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, and when the instructions are executed by the processor, the processor executes the steps of the IMU and TOF-based fast trapezoidal adjustment method as described above.
[0020] In order to solve the above technical problems, another technical solution adopted in the present application is: a computer-readable storage medium storing program instructions capable of implementing the fast trapezoidal adjustment method based on IMU and TOF as described above.
[0021] The embodiment of the present invention has the following advantages due to the adoption of the above technical solution: The present invention combines the built-in accelerometer and gyroscope of the IMU module and the TOF sensor to detect the posture changes of the projection device in real time, including the pitch angle, roll angle and yaw angle, so as to quickly and accurately adjust the trapezoidal distortion of the projected image. At the same time, the pitch angle and roll angle are calculated using the accelerometer data, the yaw angle is estimated by combining the gyroscope data with the complementary filtering algorithm, and the yaw angle is further accurately calculated in combination with the TOF sensor data, thereby improving the accuracy of posture perception, realizing fast and accurate trapezoidal adjustment, and significantly improving the quality of projection display and user experience. The present invention has low requirements for TOF and only needs to support two-point ranging to accurately calculate the yaw angle yaw, which has the advantages of low cost, rapid and accurate following, and good user experience.
[0022] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0024] Figure 1 It is a flow chart of the fast trapezoidal adjustment method based on IMU and TOF of the present invention; Figure 2 Schematic diagram of the distance between two TOF regions corresponding to D1 and D2 of the present invention; Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0025] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.
[0026] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.
[0027] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.
[0028] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.
[0029] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.
[0030] Figure 1 FIG. 1 is a flow chart of a fast trapezoidal adjustment method based on IMU and TOF according to an embodiment of the present invention. It should be noted that if there are substantially the same results, the method of the present application is not based on Figure 1 The process sequence shown is limited. Figure 1-Figure 2 As shown: A fast trapezoidal adjustment method based on IMU and TOF includes the following steps: Step 1: Use the built-in accelerometer and gyroscope of the IMU module to detect the posture changes of the projection device in real time, including the pitch angle and roll angle. The pitch angle is the angle of rotation of the device around the Y axis, which can be obtained by calculating arctan(Ay / Az) (assuming that the Y axis is in the vertical direction and Az is the component of gravity acceleration on the Z axis); The roll angle is the angle of rotation of the device around the X-axis, which can be obtained by calculating arctan(Ax / sqrt(Ay2 + Az2)) (Ax is the component of gravity acceleration on the X-axis); Step 2: Calculate the pitch angle and roll angle according to the direction of gravity acceleration through the accelerometer data to sense the posture of the projection device. The acceleration data of the accelerometer on three axes (X, Y, Z), i.e. gravity acceleration, is read in real time through the interface of the IMU module. The pitch angle and roll angle can describe the tilt degree of the device in space. Step 3: Use the gyroscope to measure the angular velocity of the device in each axis, integrate these angular velocities, obtain the rotation angle of the device in each direction, and use the complementary filtering algorithm to estimate the yaw angle yaw. In the process of posture perception of the projection device, in addition to the pitch angle pitch and roll angle roll, the yaw angle describes the angle of rotation of the device around the vertical axis (usually the Z axis). Since the gyroscope can measure angular velocity, the change of the yaw angle can be calculated by integrating the gyroscope data. Due to the drift phenomenon of the gyroscope, long-term integration will lead to error accumulation. Therefore, the complementary filtering algorithm is used to estimate the yaw angle yaw to improve the accuracy and stability of the yaw angle estimation; Step 4: Use two areas in the horizontal direction of the TOF sensor to measure the distance. Combine the FOV information of these two areas to obtain two direction vectors. The TOF sensor, i.e., the time-of-flight sensor, is mainly used to measure the flight time of the signal between two asynchronous transceivers, so as to calculate the distance between the nodes. Select two different areas in the horizontal direction of the TOF sensor for distance measurement. For example, assuming that the TOF sensor has a field of view (FOV) of 90°, the 45° areas on its left and right sides can be selected as the distance measurement areas. Since the TOF sensor can directly measure the flight time of light, the distance can be calculated based on the flight time and speed of light. It is known that the FOV of the TOF sensor is 90°, and the 45° areas on its left and right sides are selected for distance measurement. Since these two areas correspond to two different directions in the horizontal direction, it can be considered that the distance measurement results in these two directions represent two direction vectors respectively. For example, assuming that the distance measurement result of the 45° area on the left is d1, and the distance measurement result of the 45° area on the right is d2. Then, the two direction vectors can be expressed as V1 (left, size d1) and V2 (right, size d2); Step 5. Obtain the pitch angle pitch and roll angle roll of the device according to the IMU, as well as the ranging information of the two areas in the horizontal direction of the TOF and the FOV angle information of the two areas, and calculate the precise yaw angle yaw. By combining the pitch angle pitch and roll angle roll provided by the IMU, as well as the ranging information and FOV angle information of the two areas in the horizontal direction of the TOF sensor, a more accurate calculation can be performed. Step 6. Synchronize the precise yaw angle obtained by the stabilized TOF to the IMU following module, and fuse the attitude data and yaw angle data obtained by the IMU and TOF modules. In the fusion process, different weights or trusts can be set according to the characteristics and application scenarios of the IMU and TOF data. For example, when the TOF sensor data is reliable, it can be given a higher weight; when the TOF sensor is interfered with or cannot obtain valid data, it can rely more on the IMU data. By fusing the data of the IMU and TOF modules, more stable and accurate attitude and yaw angle information can be obtained, which is of great significance for the trapezoidal correction and automatic adjustment functions of the projection device, and can improve the performance and user experience of the device; Step 7. Apply the fused data to the trapezoidal correction function in real time, and adjust the projected image by software or hardware to make it present a standard rectangular shape. By fusing the data of the IMU and TOF modules and applying them to the trapezoidal correction function in real time, the trapezoidal distortion problem of the projected image can be effectively solved, and the projected image can present a standard rectangular shape.
[0031] In one embodiment, specifically, in step 2, the pitch angle pitch is calculated by measuring the components of gravity acceleration on the Y axis and Z axis of the device.
[0032] The calculation formula for the pitch angle is: ; in, and are the gravitational acceleration components measured by the accelerometer on the Y-axis and Z-axis respectively.
[0033] The pitch angle is the angle of rotation of the device around the Y axis, indicating the degree of inclination of the device head relative to the horizontal plane. The pitch angle can be calculated by measuring the components of gravity acceleration on the Y and Z axes of the device.
[0034] The roll angle roll is calculated by measuring the components of gravity acceleration on the device's X and Z axes.
[0035] The calculation formula of the roll angle roll is: ; in, and are the gravitational acceleration components measured by the accelerometer on the X-axis and Z-axis respectively.
[0036] The roll angle is the angle of rotation of the device around the X-axis, indicating the degree of rotation of the device around its longitudinal axis. The roll angle can be calculated by measuring the components of gravity acceleration on the X-axis and Z-axis of the device.
[0037] In one embodiment, specifically, in step three, the gyroscope measurement values gx, gy, gz, and the error terms ex, ey, ez and their integral terms exInt, eyInt, ezInt are defined, the quaternions q0, q1, q2, q3 are initialized, and the noise parameters Ki of the gyroscope and accelerometer are set.
[0038] Using the current quaternions q0, q1, q2, q3, the formula for estimating the gravity directions vc, vy, and vz is: ; ; ; The formula for integrating the error term based on the time step dt, the error term, and the integral gain Ki is: ; ; ; Using the gyroscope measurement and the integral term of the error term, the formula for adjusting the gyroscope measurement is: ; ; ; ; The formula for calculating the yaw angle using the updated quaternion is: ; in, , , , is the updated quaternion.
[0039] According to the updated quaternion, the atan2 function is used to calculate the yaw angle yaw. The yaw angle describes the angle of rotation of an object around the vertical axis on the horizontal plane. It is very important for applications such as navigation and positioning. Through the combination of quaternions, the tangent value of the yaw angle is obtained and converted into an angle form. Since the atan2 function returns a radian value, it needs to be multiplied by 180 / π (ie 57.3) to convert it into degrees.
[0040] In one embodiment, specifically, in step five, a two-region TOF ranging model is defined, and the FOV angle , D1 and D2 correspond to the distances between the two TOF regions, and the direction vectors of D1 and D2 are vectors and .
[0041] Rotate according to the Euler angle yaw→pitch→roll order, and you can get the rotated vectors are and ,in: Rotate around the z axis The rotation matrix of (yaw) is: ; Rotate around the y axis The rotation matrix of (roll) is: ; Rotate around the x-axis The rotation matrix of (pitch) is: ; These rotation matrix formulas are well known in the theory of rigid body rotation. They are derived from the basic definitions of Euler angles and rotation matrices. Euler angles are a way to describe the rotation of a rigid body in three-dimensional space, usually in a certain order (such as the yaw→pitch→roll order mentioned here) to determine the final posture of the rigid body. Using these well-known rotation matrix formulas, the rotated vector is calculated based on the direction vectors of the two regions of the TOF sensor and the device attitude angles (pitch, roll, yaw) obtained by the IMU. and ; Calculate the rotated vector according to the rotation formula and The calculation formula is as follows: ; ; According to the rotation of D1 and D2, and and Using analytical geometry to infer the yaw angle yaw ( );Specifically: Let the vector corresponding to D1 after rotation be ; The vector corresponding to D2 ; According to the dot product formula of vectors: ; in, is the angle between two vectors, for vector and ,have: ; Because the distances between D1 and D2 and their corresponding FOV angle information are known, we can get The expression of yaw angle yaw is established (here yaw is reflected in the rotation matrix ) equation; For example, if the distance between D1 and D2 is known to be , the FOV angle corresponding to D1 is , the FOV angle corresponding to D2 is , through some geometric derivation (the complicated derivation process is omitted here because it involves detailed analysis based on specific geometric figures and known conditions), we can get: ; in, and It can be further deduced based on the distance and angle relationship between D1 and D2. The formula here is just a schematic one.
[0042] Change the formula: ; ; Substituting into the formula: ; Get an equation about the yaw angle, and then solve the equation by numerical calculation method or further algebraic simplification to get the value of the yaw angle; For example, if we simplify the equation: ; Here A, B, and C are coefficients obtained based on the previous calculations and known conditions. The auxiliary angle formula can be used: ; Transform it into:
[0043] in, ; Then solve for The value of is the value of the yaw angle yaw.
[0044] When using a TOF (time of flight) sensor in conjunction with IMU (inertial measurement unit) data, the ranging information of the two regions (D1 and D2) provided by the TOF sensor and their respective FOV (field of view) can be used to assist in calculating the yaw angle (yaw). Here we assume that we already have accurate estimates of the pitch and roll angles provided by the IMU. First, we define the ranging model for the two regions of TOF, where D1 and D2 correspond to the measured distances of the two regions of TOF, and the direction vectors are and (in the device coordinate system). These two direction vectors are usually known or can be determined by the physical characteristics and installation method of the TOF sensor. Then, we need to rotate these two direction vectors into the world coordinate system (or earth coordinate system) through the attitude data (Euler angles yaw, pitch, roll) provided by the IMU. However, please note that the rotation order of the Euler angles is usually yaw (rotation around the Z axis), pitch (rotation around the Y axis), roll (rotation around the X axis), but when rotating the vectors, we need to apply these rotations in reverse, that is, apply pitch first, then roll, and finally yaw.
[0045] In one embodiment, specifically, in step one, the IMU module also includes a temperature sensor for detecting the temperature of the projection device in real time, and performing temperature compensation on the measurement data of the accelerometer and gyroscope of the IMU module according to the temperature information, so as to improve the accuracy of posture detection, wherein the temperature sensor in the IMU module is used to detect the temperature of the projection device in real time. Temperature is one of the important factors affecting the performance of the IMU sensor, which may cause deviations in the sensor readings. In order to improve the performance of the IMU, a temperature compensation algorithm is required to correct the errors caused by temperature. The temperature compensation algorithm infers the measurement errors of other sensors by using the readings of the temperature sensor, and performs corresponding compensation. By accurately estimating the deviations caused by the temperature of the sensor, the IMU's posture, acceleration, angular velocity and other measurement values can be more accurately calculated.
[0046] In one embodiment, specifically, in step seven, the trapezoidal correction function is implemented using a built-in processor of the projection device or an externally connected computer. In the correction process, a linear interpolation or bilinear interpolation algorithm is used to correct the smoothness and accuracy of the image, wherein linear interpolation is a simple interpolation method, and its interpolation function is a first-order polynomial. In the trapezoidal correction process, linear interpolation can be used to approximate the mapping relationship between the original image and the corrected image. By selecting key points in the original image and using a linear interpolation algorithm to calculate the positions of these key points in the corrected image, a smooth transformation of the image can be achieved; bilinear interpolation is also called bilinear interpolation, which is an extension of linear interpolation in two-dimensional space. In trapezoidal correction, bilinear interpolation can be used to process pixel points in the image to improve the accuracy and smoothness of the correction. Bilinear interpolation calculates the value of the point by considering four neighboring points around the pixel point (forming a 2x2 pixel grid). First, linear interpolation is performed twice in the horizontal direction, and then linear interpolation is performed once in the vertical direction to obtain the final value of the pixel point.
[0047] In one embodiment, specifically, in step four, the TOF sensor is also used to detect the distance between the projection device and the projection surface, and the distance information is used to assist in calculating the degree of trapezoidal distortion of the projected image, wherein the trapezoidal distortion is usually caused by the non-perpendicular angle between the projection device and the projection surface. When the projection device is projected at a non-perpendicular angle, the projected image will appear as a trapezoid on the projection surface instead of a standard rectangle. In order to correct this distortion, it is necessary to know the exact distance and angle information between the projection device and the projection surface. The TOF sensor determines the distance between the projection device and the projection surface by measuring the flight time. It emits a beam of light pulses or radio waves and measures the time required for these signals to travel from the projection device to the projection surface and back. Since the speed of light is known, the distance can be determined by measuring the time.
[0048] In one embodiment, specifically, in step six, when the posture data and yaw angle data obtained by the IMU and TOF modules are fused, a Kalman filter or an extended Kalman filter is used for fusion, wherein the data of the IMU and TOF modules are fused using the Kalman filter or the extended Kalman filter. Specifically, the posture data of the IMU and the yaw angle data of the TOF can be used as inputs of the filter, and the fused posture information is obtained through the prediction and update steps of the filter. In this process, the filter estimates the optimal value of the posture based on the data characteristics of the IMU and TOF and the dynamic model of the system, and considers the noise and uncertainty of the data at the same time. After fusion processing by the Kalman filter or the extended Kalman filter, more accurate and stable posture information can be obtained, and this information can be used in application scenarios such as trapezoidal correction and posture control of projection equipment.
[0049] In summary, this embodiment combines the data of IMU and TOF sensors to achieve fast and accurate perception of the projection device posture and precise calculation of the yaw angle. Based on these data, real-time trapezoidal correction can be performed on the projected image, thereby effectively improving the trapezoidal distortion problem. This method has the advantages of high real-time performance, high precision, good stability, etc., and is suitable for various projection display devices and application scenarios.
[0050] like Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure is shown, which is a schematic diagram of the structure of a computer device suitable for implementing the embodiment of the present disclosure. Figure 3 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0051] like Figure 3 As shown, the computer device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer device are also stored. The processor, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.
[0052] Typically, the following devices can be connected to the I / O interface: input devices such as sensors or visual information acquisition devices; output devices such as display screens; storage devices such as tapes, hard disks, etc.; and communication devices. The communication device can allow the computer device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Figure 3 A computer device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.
[0053] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the fast trapezoidal adjustment method based on IMU and TOF of the embodiment of the present disclosure are executed.
[0054] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.
[0055] According to the computer-readable storage medium of the embodiment of the present disclosure, non-transitory computer-readable instructions are stored thereon. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the IMU- and TOF-based fast trapezoidal adjustment method of each embodiment of the present disclosure are executed.
[0056] The above-mentioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or mobile hard disk), media with built-in rewritable non-volatile memory (e.g., memory card) and media with built-in ROM (e.g., ROM box).
[0057] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.
[0058] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.
[0059] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.
[0060] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.
[0061] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.
[0062] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.
[0063] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
[0064] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.
Claims
1. A fast trapezoidal adjustment method based on IMU and TOF, characterized in that: The following steps are involved: Step 1: Use the built-in accelerometer and gyroscope of the IMU module to detect the posture changes of the projection device in real time, including the pitch angle and roll angle; Step 2: Calculate the pitch angle and roll angle according to the direction of gravity acceleration through accelerometer data to sense the posture of the projection device; Step 3: Use the gyroscope to measure the angular velocity of the device in each axis, integrate these angular velocities to obtain the rotation angle of the device in each direction, and use the complementary filtering algorithm to estimate the yaw angle; Step 4: Use the two areas in the horizontal direction of the TOF sensor to measure the distance, and combine the FOV information of the two areas to obtain two direction vectors; Step 5: Obtain the pitch angle and roll angle of the device according to the IMU, as well as the ranging information of the two areas in the horizontal direction of TOF and the FOV angle information of the two areas, and calculate the yaw angle; Step 6: Synchronize the precise yaw angle obtained by the stabilized TOF to the IMU following module, and fuse the attitude data and yaw angle data obtained by the IMU and TOF modules; Step 7: Apply the fused data to the keystone correction function in real time, and adjust the projected image by software or hardware to make it present a standard rectangular shape.
2. The fast trapezoidal adjustment method based on IMU and TOF according to claim 1, characterized in that: In step 2, the pitch angle pitch is calculated by measuring the components of gravity acceleration on the Y axis and Z axis of the device; The calculation formula for the pitch angle is: ; in, and They are the gravitational acceleration components measured by the accelerometer on the Y-axis and Z-axis respectively; The roll angle roll is calculated by measuring the components of gravity acceleration on the X-axis and Z-axis of the device; The calculation formula of the roll angle roll is: ; in, and are the gravitational acceleration components measured by the accelerometer on the X-axis and Z-axis respectively.
3. The fast trapezoidal adjustment method based on IMU and TOF according to claim 1, characterized in that: In step 3, the gyroscope measurement values gx, gy, gz, and the error terms ex, ey, ez and their integral terms exInt, eyInt, ezInt are defined, the quaternions q0, q1, q2, q3 are initialized, and the noise parameters Ki of the gyroscope and accelerometer are set; using the current quaternions q0, q1, q2, q3, the formula for estimating the gravity direction vc, vy, vz is: ; ; ; The formula for integrating the error term based on the time step dt, the error term, and the integral gain Ki is: ; ; ; Using the gyroscope measurement and the integral term of the error term, the formula for adjusting the gyroscope measurement is: ; ; ; ; The formula for calculating the yaw angle using the updated quaternion is: ; in, , , , is the updated quaternion.
4. The fast trapezoidal adjustment method based on IMU and TOF according to claim 1, characterized in that: In step 5, the TOF two-area ranging model is defined, and the FOV angle , D1 and D2 correspond to the distances between the two TOF regions, and the direction vectors of D1 and D2 are vectors and ; Rotate according to the Euler angle yaw→pitch→roll order, and you can get the rotated vectors are and ,in: Rotate around the z axis The rotation matrix of (yaw) is: ; Rotate around the y axis The rotation matrix of (roll) is: ; Rotate around the x-axis The rotation matrix of (pitch) is: ; Calculate the rotated vector according to the rotation formula and The calculation formula is as follows: ; ; According to the rotation of D1 and D2, and and Using analytical geometry to infer the yaw angle yaw ( ).
5. The fast trapezoidal adjustment method based on IMU and TOF according to claim 1, characterized in that: In step one, the IMU module further includes a temperature sensor for detecting the temperature of the projection device in real time, and performing temperature compensation on the measurement data of the accelerometer and gyroscope of the IMU module according to the temperature information.
6. The fast trapezoidal adjustment method based on IMU and TOF according to claim 5, characterized in that: In step seven, the trapezoidal correction function is implemented using a built-in processor of the projection device or an externally connected computer, and a linear interpolation or bilinear interpolation algorithm is used during the correction process to correct the smoothness and accuracy of the image.
7. The fast trapezoidal adjustment method based on IMU and TOF according to claim 1, characterized in that: In step four, the TOF sensor is also used to detect the distance between the projection device and the projection surface, and the distance information is used to assist in calculating the degree of trapezoidal distortion of the projected image.
8. The fast trapezoidal adjustment method based on IMU and TOF according to claim 1, characterized in that: In step six, when fusing the attitude data and yaw angle data obtained by the IMU and TOF modules, a Kalman filter or an extended Kalman filter is used for fusion.
9. A computer device, characterized in that: The computer device comprises: a processor, and a memory connected to the processor in communication; wherein, The memory stores instructions that can be executed by the processor, and the instructions are executed by the processor so that the processor can execute the IMU and TOF-based fast trapezoidal adjustment method described in any one of claims 1-8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the IMU- and TOF-based fast trapezoidal adjustment method described in any one of claims 1-8.
Citation Information
Patent Citations
Keystone correction method and device, projector and computer readable storage medium
CN113824942A
Projector single TOF keystone correction method and projector
CN114745529A
Attitude fusion algorithm based on Kalman filtering and explicit complementary filtering
CN115164875A
Projection equipment attitude correction method and device, electronic equipment and readable storage medium
CN118474310A
Projection equipment correction method, projection equipment and storage medium
CN118625860A
