Pose estimation method and device and control equipment

Abstract

The embodiment of the invention discloses a pose estimation method and device and control equipment. The method comprises the following steps: acquiring vehicle motion information and first IMU information of control equipment; converting the first IMU information into second IMU information under a vehicle coordinate system based on the vehicle motion information; and obtaining pose information of the control equipment relative to the vehicle based on the second IMU information. Through the above mode, the first IMU information can be converted into the second IMU information under the vehicle coordinate system based on the vehicle motion information, so that the first IMU information and the vehicle motion information can be effectively fused to obtain the accurate second IMU information, and the pose information of the control device on the vehicle is obtained based on the second IMU information. And the accuracy of the pose information is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a posture estimation method, apparatus, and control device. Background Art

[0002] As people's second living space, the status of vehicles in daily life is constantly improving, which has led to an increasing demand for the use of MR (Mixed Reality) devices, VR (Virtual Reality) devices, XR (Extended Reality) devices, AR (Virtual Reality) devices, etc. in the cabin, and in order to improve the user experience, the above devices can also be configured with control devices (such as handles, etc.). In a related method, the position and posture information of the control device can be determined based on the VIO (Visual-Inertial Odometry) algorithm, but the related method still has the problem of inaccurate position and posture information obtained. Summary of the invention

[0003] In view of the above problems, the present application proposes a posture estimation method, apparatus and control device to improve the above problems.

[0004] In a first aspect, the present application provides a posture estimation method, the method comprising: obtaining vehicle motion information and first IMU information of a control device; converting the first IMU information into second IMU information in a vehicle coordinate system based on the vehicle motion information; and obtaining the posture information of the control device relative to the vehicle based on the second IMU information.

[0005] In the second aspect, the present application provides a posture estimation device, which includes: an IMU information acquisition unit, used to obtain vehicle motion information and first IMU information of a control device; based on the vehicle motion information, converting the first IMU information into second IMU information in a vehicle coordinate system; a posture information acquisition unit, used to obtain the posture information of the control device relative to the vehicle based on the second IMU information.

[0006] In a third aspect, the present application provides a control device comprising one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0007] In a fourth aspect, the present application provides a computer-readable storage medium, in which program code is stored, wherein the above method is executed when the program code is run.

[0008] The present application provides a method, device, control device and storage medium for posture estimation. After obtaining the vehicle motion information and the first IMU information of the control device, the first IMU information is converted into the second IMU information in the vehicle coordinate system based on the vehicle motion information, and the posture information of the control device relative to the vehicle is obtained based on the second IMU information. Through the above method, the first IMU information can be converted into the second IMU information in the vehicle coordinate system based on the vehicle motion information, so that the first IMU information and the vehicle motion information can be effectively fused to obtain accurate second IMU information, and then the posture information of the control device on the vehicle is obtained based on the second IMU information, thereby improving the accuracy of the posture information. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A flowchart of a posture estimation method proposed in an embodiment of the present application;

[0011] Figure 2 A schematic diagram showing a time difference between a vehicle and a control device proposed in the present application is shown;

[0012] Figure 3 A flow chart of a posture estimation method proposed in another embodiment of the present application is shown;

[0013] Figure 4 A structural block diagram of a posture estimation device proposed in an embodiment of the present application is shown;

[0014] Figure 5 A structural block diagram of a control device proposed in this application is shown. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0016] In an embodiment of the present application, the inventor proposes a method, apparatus and control device for posture estimation. After obtaining the vehicle motion information and the first IMU information of the control device, the first IMU information is converted into the second IMU information in the vehicle coordinate system based on the vehicle motion information, and the posture information of the control device relative to the vehicle is obtained based on the second IMU information. Through the above method, the first IMU information can be converted into the second IMU information in the vehicle coordinate system based on the vehicle motion information, so that the first IMU information and the vehicle motion information can be effectively fused to obtain accurate second IMU information, and then the posture information of the control device on the vehicle is obtained based on the second IMU information, thereby improving the accuracy of the posture information.

[0017] See also Figure 1 , an embodiment of the present application provides a posture estimation method, the method comprising:

[0018] S110: Acquire vehicle motion information and first IMU information of the control device.

[0019] Among them, vehicle motion information may refer to information used to describe the motion of a vehicle in a world coordinate system (such as a UTM (Universal Transverse Mercator Grid System) coordinate system). Vehicle motion information may include vehicle position information, vehicle speed information, and vehicle rotation information. Vehicle rotation information may be used to characterize the orientation of the vehicle, that is, its posture in the world coordinate system. First IMU information may refer to information used to describe the motion of a control device in a world coordinate system. The first IMU information may include first position information and first rotation information. The first rotation information may characterize the posture of the control device in the world coordinate system.

[0020] In the embodiments of the present application, the control device may refer to a physical handle that is compatible with MR (Mixed Reality) devices, VR (Virtual Reality) devices, XR (Extended Reality) devices, AR (Virtual Reality) devices, etc. The control device may also refer to an electronic device (such as a mobile phone, tablet, etc.) equipped with a virtual handle, and the virtual handle may refer to an application installed on the control device, which can present images of all physical handle buttons on the screen of the control device, so that the user can simulate the operation of a real physical handle by operating the button images on the touch screen.

[0021] The control device can establish a network connection with the vehicle through technologies such as Bluetooth and WiFi (Wireless-Fidelity), so as to obtain vehicle motion information; and the control device can be equipped with an IMU (Inertial Measurement Unit), which can be used to detect and measure the acceleration and rotational motion of the control device in the world coordinate system.

[0022] As a method, the control device can obtain the vehicle motion information and the first IMU information of the control device in real time.

[0023] Optionally, when acquiring vehicle motion information and the first IMU information, the vehicle motion information acquired by the control device can be synchronized with the collection time of the first IMU information through the NTP (Network Time Protocol) service provided by the vehicle, that is, the vehicle motion information and the first IMU information at the same collection time are acquired.

[0024] Optionally, when the vehicle does not provide NTP service, the time delay between the vehicle motion information and the first IMU information received by the control device at the same collection time can be calculated by means of average error, so that the vehicle motion information and the first IMU information can be time-aligned through the calculated time delay, and the vehicle motion information and the first IMU information at the same collection time can be obtained.

[0025] For example, Figure 2 As shown, the time used by the vehicle and the control device are different, and the delay can be calculated by the data transmission rate between the two, the amount of data transmitted, etc.

[0026] In an embodiment of the present application, the vehicle motion information acquired by the control device is synchronized with the collection time of the first IMU information through NTP service or average error, which can improve the accuracy and reliability of the posture information.

[0027] S120: Convert the first IMU information into second IMU information in a vehicle coordinate system based on the vehicle motion information.

[0028] As a method, the acceleration function in the second IMU information can be obtained based on the vehicle position information, the vehicle speed information, and the first position information. The acceleration function can characterize the relationship between the acceleration of the control device in the vehicle coordinate system and the vehicle acceleration; the angular velocity function in the second IMU information can be obtained based on the vehicle rotation information and the first rotation information. The angular velocity function can characterize the relationship between the angular velocity of the control device in the vehicle coordinate system and the vehicle angular velocity.

[0029] Optionally, a position function can be obtained based on the vehicle position information and the first position information, and the position function can characterize the relationship between the position information of the control device in the vehicle coordinate system and the vehicle position information and the first position information; an acceleration function can be obtained based on the position function and the vehicle speed information.

[0030] Among them, the position function can be:

[0031] P car,imu =P car，world +R car，world P world,imu

[0032] Among them, P car，imu It can represent the position information of the control device in the vehicle coordinate system; P car，world Can be P world，car The inverse of P car，world and P world，car Can be a matrix, P world，car Can represent vehicle location information, P car，world It can represent the position information of the world in the vehicle coordinate system; R car，world It can represent R world，car The inverse of R car，world and R world，car Can be a matrix, R world，car Can represent vehicle rotation information, R car，world It can represent the rotation information of the world in the vehicle coordinate system; P world,imu The first position information may be indicated.

[0033] Optionally, the process of obtaining the acceleration function based on the position function and the vehicle speed information may be: firstly taking partial derivative of the position function to obtain the speed function; and obtaining the acceleration function based on the speed function and the vehicle speed information.

[0034] Among them, the vehicle motion information may also include angular velocity information, angular acceleration information, etc., and the first IMU information may also include first velocity information, first acceleration information, etc.

[0035] The speed function can be:

[0036]

[0037] Among them, v car，imu It can represent the speed information of the control device in the vehicle coordinate system; P car，imu It can represent the position information of the control device in the vehicle coordinate system; P car，world Can be P world，car The inverse of P world，car Can represent vehicle location information; R car，world It can represent R world，carThe inverse of R world，car Can represent vehicle rotation information; P world,imu Can represent the first position information; v world，car Can indicate vehicle speed information; v world,min It can represent the speed information of the control device in the world coordinate system, that is, the first speed information in the first IMU information; car，world Can be w world，car The inverse of W world，car It can represent the angular velocity information of the vehicle in the world coordinate system, that is, the angular velocity information in the vehicle motion information.

[0038] The acceleration function can be:

[0039]

[0040] Among them, a car，imu It can represent the speed information of the control device in the vehicle coordinate system; car，imu It can represent the speed information of the control device in the vehicle coordinate system; R car，world It can represent R world，car The inverse of R world，car Can represent vehicle rotation information; v world,min It can represent the speed information of the control device in the world coordinate system, that is, the first speed information in the first IMU information; car，world Can be w world，car The inverse of w world，car It can represent the angular velocity information of the vehicle in the world coordinate system, that is, the angular velocity information in the vehicle motion information; a world,min It can represent the acceleration information of the control device in the vehicle coordinate system, that is, the first acceleration information in the first IMU information; w′ car，world It can be expressed as w′ world，car The inverse of w′ world，car It can represent the acceleration information of the vehicle's angular velocity in the world coordinate system, that is, the angular acceleration information in the vehicle's motion information.

[0041] Optionally, the process of obtaining the angular velocity function in the second IMU information based on the vehicle rotation information and the first rotation information is as follows:

[0042]

[0043]

[0044] Among them, R car，imu It can represent the speed information of the control device in the vehicle coordinate system; R world，car Can represent vehicle rotation information; R world,mminIt can represent the rotation information of the control device in the world coordinate system, that is, the first rotation information in the first IMU information.

[0045] S130: Based on the second IMU information, obtain the position information of the control device relative to the vehicle.

[0046] The position and posture information of the control device relative to the vehicle can be understood as the position and posture information of the control device in the vehicle coordinate system.

[0047] As a method, the position information of the control device relative to the vehicle can be obtained based on the second IMU information and the EKF (Extended Kalman Filter) algorithm.

[0048] This embodiment provides a method for posture estimation, after obtaining vehicle motion information and first IMU information of a control device, the first IMU information is converted into second IMU information in a vehicle coordinate system based on the vehicle motion information, and the posture information of the control device relative to the vehicle is obtained based on the second IMU information. Through the above method, the first IMU information can be converted into the second IMU information in the vehicle coordinate system based on the vehicle motion information, so that the first IMU information and the vehicle motion information can be effectively fused to obtain accurate second IMU information, and then the posture information of the control device on the vehicle is obtained based on the second IMU information, thereby improving the accuracy of the posture information.

[0049] See also Figure 3 , an embodiment of the present application provides a posture estimation method, the method comprising:

[0050] S210: Acquire vehicle motion information and first IMU information of the control device.

[0051] S220: Convert the first IMU information into second IMU information in a vehicle coordinate system based on the vehicle motion information.

[0052] S230: Based on the multiple images captured by the multi-eye camera in the control device, a photometric error function is obtained, wherein the photometric error function characterizes the photometric errors between multiple feature point pairs in the multiple images, and the two feature points in each feature point pair come from different images.

[0053] Among them, the photometric error can represent the brightness error between two feature points.

[0054] As a method, a feature point pair position relationship function can be obtained based on multiple images and a direct method. The feature point pair position relationship function can characterize the position relationship between two feature points in each feature point pair; based on the feature point pair position relationship function and a photometric calibration algorithm, a photometric error function is obtained.

[0055] Among them, a multi-eye camera may refer to a camera including at least two cameras, and all cameras in the multi-eye camera may continuously acquire images at the same acquisition frequency. At the same acquisition time, each camera of the multi-eye camera may acquire an image. Therefore, a feature point may be selected from two images acquired by the same camera at adjacent acquisition times (adjacent frames) according to the direct method to form a feature point pair; a feature point may also be selected from two images acquired by different cameras at the same acquisition time according to the direct method to form a feature point pair, and each feature point is equivalent to a pixel point in the corresponding image. The direct method may refer to an algorithm for calculating the motion of feature points.

[0056] In an embodiment of the present application, by extracting feature point pairs from images captured by adjacent frames of the same camera, and extracting feature point pairs from images captured by different cameras at the same capture time, it is possible to track the movement changes of the same camera in the vehicle, and to track the real-time posture relationship between different cameras, so that the obtained multiple feature point pairs can represent at least two features related to the posture information of the control device, thereby improving the density and effectiveness of the feature information in the multiple feature point pairs, and further improving the accuracy of the posture information finally obtained.

[0057] Optionally, in order to reduce the amount of calculation, a sparse direct method (Sparse Direct Method) in the direct method can be used to select pixel points in the image whose gradient changes exceed a preset value as feature points.

[0058] Taking a feature point pair as an example, the position relationship between the two feature points in the feature point pair obtained based on the sparse direct method is:

[0059]

[0060] Among them, (u1,v1) can represent the position information of one feature point (feature point 1) in the feature point pair in the corresponding image (image 1), (u2,2) can represent the position information of the other feature point (feature point 2) in the feature point pair in the corresponding image (image 2), ∏1(.) can represent the projection function of the camera model (camera) corresponding to image 1, and ∏2(.) can represent the projection function of the camera model (camera) corresponding to image 2. It can represent the relative position and posture of image 1 and image 2.

[0061] Optionally, the photometric error function can be:

[0062]

[0063] Among them, I1(u1,v1) can represent the pixel value of feature point 1 in image 1, I2(u2,v2) can represent the pixel value of the feature point pair (feature point 2) in image 2, G(.) can represent the response function in the photometric calibration algorithm, which is an inherent property of fresh eggs, and V(.) can represent the attenuation factor in the photometric calibration algorithm, which can be used to describe the vignetting phenomenon (Vignetting).

[0064] In the embodiments of the present application, by introducing photometric calibration, the pose estimation algorithm proposed in the present application can better cope with the huge changes in light and shadow inside the cabin, thereby improving the accuracy and robustness of pose estimation.

[0065] S240: Based on the second IMU information, the photometric error function and the extended Kalman filter algorithm, obtain the position information of the control device relative to the vehicle.

[0066] The position information of the control device relative to the vehicle may be:

[0067]

[0068] Among them, P car,imu It can represent the position information of the control device in the vehicle coordinate system (relative to the vehicle); car,imu It can represent the speed of the control device in the vehicle coordinate system; R car,imu Can represent the rotation information of the control device in the vehicle coordinate system; bias acc It can represent the bias of the acceleration measured by the IMU in the control device; bias acc It can represent the internal reference bias corresponding to the gyroscope in the IMU of the control device; d i It can represent the depth value of the i-th feature point.

[0069] It should be noted that the above x can be used as the state of the system corresponding to the extended Kalman filter algorithm. When the posture information is obtained by the extended Kalman filter algorithm, an initial value containing the same information as x can be input. The posture information of the control device relative to the vehicle can be 6DoF (Degree of Freedom) information.

[0070] As a method, the predicted state information and the predicted covariance matrix can be obtained based on the second IMU information, the state prediction equation and the covariance matrix prediction equation; based on the predicted state information, the predicted covariance matrix and the photometric error function, the position information of the control device relative to the vehicle can be obtained.

[0071] Among them, the state prediction equation can be used to make a priori estimation of the position and posture information of the control device relative to the vehicle, that is, to obtain the predicted state information. The prior estimation result can then be corrected by the predicted state information, the predicted covariance matrix and the photometric error function to obtain the position and posture information of the control device relative to the vehicle.

[0072] Among them, the state prediction equation can be:

[0073]

[0074] Among them, f(.) can represent the second IMU information, X k|k It can represent the position information of the control device relative to the vehicle at the last moment. Can represent an XOR operation.

[0075] The prediction covariance matrix can be:

[0076]

[0077] Among them, F can represent the first-order linear matrix of f(.); Q k It can represent the error of system update, which is determined by the internal parameters of IMU and obtained by calibrating IMU; P k|k It can represent the covariance matrix of the previous moment.

[0078] After obtaining the forecast state information and forecast covariance matrix, the observation can be updated by the following formula:

[0079]

[0080] Among them, r(.) can represent residual, that is, the photometric error function in this application, which can usually be represented by a Gaussian error matrix, j can represent the number of iterations of the algorithm, v(.) can represent the first-order derivative of the system state x, D(.) represents the linearization matrix of the first-order derivative of the h(.) function relative to v(.), and H(.) represents the linearization matrix of the first-order derivative of the h(.) function relative to the system state x.

[0081] After the observations are updated, the prediction state information and the prediction covariance matrix may be corrected based on the updated observations, so that iteration may continue based on the corrected prediction covariance matrix.

[0082]

[0083] Among them, I, J, L can represent the unit matrix, K can represent the Kalman gain matrix, X k+1|k It can represent the position information of the control device relative to the vehicle.

[0084] Optionally, during the iteration process of the extended Kalman filter algorithm, a conjugate gradient method may be used to reduce the running time of the system on hardware and increase the accuracy of the system with the same number of iterations.

[0085] The present embodiment provides a method for posture estimation, which can convert the first IMU information into the second IMU information in the vehicle coordinate system based on the vehicle motion information through the above-mentioned method, so that the first IMU information and the vehicle motion information can be effectively fused to obtain accurate second IMU information, and then obtain the posture information of the control device on the vehicle based on the second IMU information, thereby improving the accuracy of the posture information. In addition, in the present embodiment, the accuracy of the observation results of the multi-eye camera of the control device is improved by direct method, photometric calibration method, etc., and the image data and multi-sensor data (vehicle motion information, first IMU information, etc.) acquired by the multi-eye camera are fused by Kalman filtering algorithm, thereby improving the accuracy and stability of the posture information of the control device in the cabin environment, and then when the user uses MR equipment, VR equipment, XR equipment, AR equipment, it can be free from the interference of light changes or vehicle motion, and can more accurately perceive and operate the virtual environment, and more smoothly perform interactive operations, thereby improving the realism of the virtual reality experience. In addition, in this application, through direct methods, data fusion, etc., it is possible to improve the accuracy and stability of the acquired control device posture information without adding additional hardware, thereby reducing the use cost of MR devices, VR devices, XR devices, AR devices, etc., and attracting more users.

[0086] See also Figure 4 The present application provides a posture estimation device 400, the device 400 comprising:

[0087] The IMU information acquisition unit 410 is used to acquire vehicle motion information and first IMU information of the control device; based on the vehicle motion information, the first IMU information is converted into second IMU information in the vehicle coordinate system;

[0088] A posture information acquisition unit 420 is used to obtain the posture information of the control device relative to the vehicle based on the second IMU information.

[0089] As a method, the vehicle motion information includes vehicle position information, vehicle speed information and vehicle rotation information. The IMU information acquisition unit 410 is specifically used to obtain an acceleration function in the second IMU information based on the vehicle position information, vehicle speed information and first position information, and the acceleration function represents the relationship between the acceleration of the control device in the vehicle coordinate system and the vehicle acceleration; based on the vehicle rotation information and the first rotation information, an angular velocity function in the second IMU information is obtained, and the angular velocity function represents the relationship between the angular velocity of the control device in the vehicle coordinate system and the angular velocity of the vehicle.

[0090] Optionally, the IMU information acquisition unit 410 is specifically used to obtain a position function based on the vehicle position information and the first position information, wherein the position function characterizes the relationship between the position information of the control device in the vehicle coordinate system and the vehicle position information and the first position information; and obtain the acceleration function based on the position function and the vehicle speed information.

[0091] As a method, the posture information acquisition unit 420 is specifically used to obtain a photometric error function based on multiple images captured by the multi-eye camera in the control device, wherein the photometric error function characterizes the photometric error between multiple feature point pairs in the multiple images, and the two feature points in each feature point pair come from different images; based on the second IMU information, the photometric error function and the extended Kalman filter algorithm, the posture information of the control device relative to the vehicle is obtained.

[0092] Optionally, the posture information acquisition unit 420 is specifically used to obtain a feature point pair position relationship function based on the multiple images and the direct method, wherein the feature point pair position relationship function can characterize the position relationship between two feature points in each feature point pair; and obtain the photometric error function based on the feature point pair position relationship function and the photometric calibration algorithm.

[0093] Optionally, the posture information acquisition unit 420 is specifically used to obtain predicted state information and a predicted covariance matrix based on the second IMU information, the state prediction equation and the covariance matrix prediction equation; and obtain the posture information of the control device relative to the vehicle based on the predicted state information, the predicted covariance matrix and the photometric error function.

[0094] The following will be combined Figure 5 A control device provided in the present application is described.

[0095] See also Figure 5Based on the above-mentioned posture estimation method and apparatus, the embodiment of the present application also provides another control device 100 that can execute the above-mentioned posture estimation method. The control device 100 includes a processor 102, a memory 104, a data acquisition module 106 and a network module 108. Among them, the memory 104 stores a program that can execute the content of the above-mentioned embodiment, and the backup processor 102 can execute the program stored in the memory 104.

[0096] Among them, the processor 102 may include one or more processing cores. The processor 102 uses various interfaces and lines to connect various parts within the entire control device 100, and executes various functions and processes data of the control device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 104, and calling data stored in the memory 104. Optionally, the processor 102 can be implemented in at least one hardware form of a network processor (Neural network Processing Unit, NPU), a digital signal processing (Digital Signal Processing, DSP), a field programmable gate array (Field-Programmable Gate Array, FPGA), and a programmable logic array (Programmable Logic Array, PLA). The processor 102 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU), a network processor (Neural network Processing Unit, NPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; the NPU is responsible for processing multimedia data such as video and images; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 102, but may be implemented separately through a communication chip.

[0097] The memory 104 may include a random access memory (RAM), and may also include a read-only memory (RAM) and a double data rate synchronous dynamic random access memory (DDR). The memory 104 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 104 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the control device 100 during use.

[0098] The data acquisition module 106 may include but is not limited to: IMU (Inertial Measurement Unit), multi-eye camera, level, light sensor, motion sensor, pressure sensor, infrared thermal sensor, distance sensor, acceleration sensor, and other sensors. Among them, the IMU can obtain the first IMU information of the control device 100. The multi-eye camera can collect environmental information around the control device 100, such as the seat and window of the vehicle.

[0099] Among them, the pressure sensor may be a sensor that detects the pressure generated by pressing on the control device 100. That is, the pressure sensor detects the pressure generated by contact or pressing between the user and the control device 100, such as the pressure generated by contact or pressing between the user's hand and the control device 100. Therefore, the pressure sensor can be used to determine whether contact or pressing occurs between the user and the control device 100, and the magnitude of the pressure.

[0100] Among them, the acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, and can be used for applications that identify the posture of the control device 100 (such as magnetometer posture calibration), vibration recognition related functions (such as pedometer, knocking), etc. In addition, the control device 100 can also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, etc., which will not be repeated here.

[0101] The network module 108 can be used to realize information exchange between the control device 100 and other devices through the network, for example, transmission of device control instructions, operation request instructions, and status information acquisition instructions, etc. When the other devices are different devices, the corresponding network modules 108 may be different.

[0102] An embodiment of the present application provides a computer-readable storage medium in which a program code is stored, and the program code can be called by a processor to execute the method described in the above method embodiment.

[0103] The computer readable storage medium may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium includes a non-transitory computer-readable storage medium. The computer readable storage medium has storage space for program codes that execute any of the method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code can be compressed, for example, in an appropriate form.

[0104] In summary, the present application provides a method, device and control device for posture estimation. After acquiring the vehicle motion information and the first IMU information of the control device, the first IMU information is converted into the second IMU information in the vehicle coordinate system based on the vehicle motion information, and the posture information of the control device relative to the vehicle is obtained based on the second IMU information. Through the above method, the first IMU information can be converted into the second IMU information in the vehicle coordinate system based on the vehicle motion information, so that the first IMU information and the vehicle motion information can be effectively fused to obtain accurate second IMU information, and then the posture information of the control device on the vehicle is obtained based on the second IMU information, thereby improving the accuracy of the posture information.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A pose estimation method, characterized in that: The method comprises: Obtain vehicle motion information and first IMU information of the control device; Converting the first IMU information into second IMU information in a vehicle coordinate system based on the vehicle motion information; Based on the second IMU information, the position information of the control device relative to the vehicle is obtained.

2. The method according to claim 1, characterized in that: The vehicle motion information includes vehicle position information, vehicle speed information and vehicle rotation information, the first IMU information includes first position information and first rotation information, and the converting the first IMU information into second IMU information in a vehicle coordinate system based on the vehicle motion information includes: Based on the vehicle position information, the vehicle speed information, and the first position information, an acceleration function in the second IMU information is obtained, wherein the acceleration function represents a relationship between the acceleration of the control device in the vehicle coordinate system and the vehicle acceleration; Based on the vehicle rotation information and the first rotation information, an angular velocity function in the second IMU information is obtained, and the angular velocity function represents the relationship between the angular velocity of the control device in the vehicle coordinate system and the angular velocity of the vehicle.

3. The method according to claim 2, characterized in that The obtaining of the acceleration function in the second IMU information based on the vehicle position information, the vehicle speed information and the first position information includes: Based on the vehicle position information and the first position information, a position function is obtained, wherein the position function represents a relationship between the position information of the control device in a vehicle coordinate system and the vehicle position information and the first position information; The acceleration function is obtained based on the position function and the vehicle speed information.

4. The method according to claim 1, characterized in that: Before obtaining the position information of the control device relative to the vehicle based on the second IMU information, the method further includes: Based on a plurality of images captured by a multi-eye camera in the control device, a photometric error function is obtained, wherein the photometric error function represents the photometric error between a plurality of feature point pairs in the plurality of images, and two feature points in each of the feature point pairs come from different images; The obtaining, based on the second IMU information, the position information of the control device relative to the vehicle, comprises: Based on the second IMU information, the photometric error function and the extended Kalman filter algorithm, the position information of the control device relative to the vehicle is obtained.

5. The method according to claim 4, characterized in that The method of obtaining a photometric error function based on a plurality of images collected by a multi-eye camera in the control device comprises: Based on the multiple images and the direct method, a feature point pair position relationship function is obtained, wherein the feature point pair position relationship function can characterize the position relationship between two feature points in each feature point pair; Based on the feature point pair position relationship function and the photometric calibration algorithm, the photometric error function is obtained.

6. The method according to claim 4, characterized in that The obtaining, based on the second IMU information, the photometric error function and the extended Kalman filter algorithm, the position and posture information of the control device relative to the vehicle includes: Based on the second IMU information, the state prediction equation and the covariance matrix prediction equation, the predicted state information and the predicted covariance matrix are obtained; Based on the predicted state information, the predicted covariance matrix and the photometric error function, the position information of the control device relative to the vehicle is obtained.

7. The method according to any one of claims 1 to 6, characterized in that: The vehicle motion information is synchronized with the collection time of the first IMU information.

8. A posture estimation device, characterized in that: The device comprises: An IMU information acquisition unit, configured to acquire vehicle motion information and first IMU information of a control device; and convert the first IMU information into second IMU information in a vehicle coordinate system based on the vehicle motion information; A posture information acquisition unit is used to obtain the posture information of the control device relative to the vehicle based on the second IMU information.

9. A control device, characterized in that: comprising one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, wherein when the program code is run, the method according to any one of claims 1 to 7 is executed.

Images