Multi-sensor based coordinate system relative matrix determination method, apparatus and device

Abstract

This application relates to a method, apparatus, and device for determining the relative matrix of a coordinate system based on multiple sensors. The method includes: acquiring Global Navigation Satellite System (GNSS) data, visual sensor data, and Inertial Measurement Unit (IMU) data at multiple consecutive time points; aligning the IMU data with the visual sensor data based on a first relative matrix between the visual sensor and the IMU to obtain aligned data; determining the relative velocity, absolute velocity, and absolute position of the GNSS between the GNSS and the visual sensor at each time point based on the GNSS data, aligned data, GNSS time, and satellite time; and determining a second relative matrix between the GNSS and the visual sensor based on the relative velocity, absolute velocity, absolute position, and the first relative matrix. This method can improve the accuracy and reliability of multi-sensor alignment.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method, apparatus and device for determining the relative matrix of a coordinate system based on multiple sensors. Background Technology

[0002] With the development of autonomous driving technology, localization technology for autonomous driving systems has emerged. However, localization algorithms are essentially a state estimation problem, and relying solely on a single sensor cannot guarantee the stability and accuracy of that estimation. Currently, autonomous driving system localization often employs the fusion of multiple sensors (such as visual sensors, IMUs (Inertial Measurement Units), and GNSS (Global Navigation Satellite System)) to achieve this.

[0003] However, visual sensors and IMUs must run only when the system starts up. Existing sensor fusion methods simply align the coordinate systems of multiple sensors at system startup, which fails to effectively guarantee the accuracy and reliability of the positioning results. Therefore, how to align the coordinate systems of multiple sensors in a short time and improve the accuracy and reliability of multi-sensor alignment is an urgent problem to be solved. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, and device for determining the relative matrix of a coordinate system based on multiple sensors, which can improve the accuracy and reliability of multi-sensor alignment, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a method for determining the relative matrix of a coordinate system based on multiple sensors.

[0006] The method includes:

[0007] Acquire GNSS data, visual sensor data, and inertial measurement unit (IMU) data at multiple consecutive time points;

[0008] Based on the first relative matrix between the visual sensor and the IMU, the IMU data and the visual sensor data are aligned to obtain aligned data;

[0009] Based on the GNSS data, alignment data, GPS time, and satellite time corresponding to each time point, determine the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the absolute position of GNSS at each time point;

[0010] Based on the relative velocity, absolute velocity, and absolute position at each moment, and the first relative matrix, the second relative matrix between the GNSS and the visual sensor is determined.

[0011] In one embodiment, based on the GNSS data, alignment data, GPS time, and satellite time corresponding to each time moment, the relative velocity between the GNSS and the visual sensor, the absolute velocity of the GNSS, and the absolute position of the GNSS at each time moment are determined, including:

[0012] Based on the GNSS data and alignment data corresponding to each time point, determine the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the GNSS time at each time point;

[0013] Based on the Global Positioning System time, satellite time, and GNSS time, determine the clock difference information for each moment;

[0014] Based on the clock difference information at each time point, the absolute position of the GNSS at each time point is determined using a pseudorange single-point positioning algorithm.

[0015] In one embodiment, the second relative matrix between the GNSS and the visual sensor includes a relative attitude matrix and a relative position matrix; determining the second relative matrix between the GNSS and the visual sensor based on the relative velocity, absolute velocity, and absolute position at each time point, and the first relative matrix, includes:

[0016] Based on the relative velocity, absolute velocity, and first relative matrix at each time point, determine the clock difference change at each time point, as well as the relative attitude matrix between the GNSS and the visual sensor;

[0017] Update the clock difference information for each time point based on the change in clock difference at each time point and the time difference between adjacent time points;

[0018] Based on the absolute position at each time point and the updated clock difference information at each time point, the relative position matrix between the GNSS and the visual sensor is determined.

[0019] In one embodiment, the clock difference change at each moment, and the relative attitude matrix between the GNSS and the visual sensor are determined based on the relative velocity, absolute velocity, and a first relative matrix at each moment, including:

[0020] Based on the relative velocity, absolute velocity, first relative matrix, and first constant at each moment, a first residual function corresponding to each moment is constructed using the Gauss-Newton nonlinear optimization algorithm. The first residual function includes the clock error change to be determined and the relative attitude matrix between GNSS and the visual sensor. The first constant includes: positioning value, speed of light, Doppler carrier wavelength, Doppler frequency shift, and pseudorange single-point compensation amount.

[0021] Solve for the first residual function at each time step to obtain the clock error change at each time step, as well as the relative attitude matrix between the GNSS and the visual sensor.

[0022] In one embodiment, the first residual function corresponding to each time step is solved to obtain the clock error change at each time step, and the relative attitude matrix between the GNSS and the visual sensor, including:

[0023] For each time step, adjust the values ​​of clock error change and relative attitude matrix in the residual function to obtain the clock error change and relative attitude matrix corresponding to the function value of the residual function at the preset value, and use them as the clock error change and relative attitude matrix corresponding to that time step.

[0024] Based on the relative attitude matrix at each moment, the final relative attitude matrix between the GNSS and the visual sensor is determined.

[0025] In one embodiment, the relative position matrix between the GNSS and the visual sensor is determined based on the absolute position at each time point and the updated clock difference information at each time point, including:

[0026] Based on the absolute position at each time point, the updated clock difference information at each time point, and the second constant, a second residual function corresponding to each time point is constructed based on the Gauss-Newton nonlinear optimization algorithm. The second residual function contains the relative position matrix between the GNSS and the visual sensor to be determined. The second constant includes: the speed of light, pseudorange, preset identity matrix, and delay information.

[0027] Solve for the second residual function at each time point to obtain the relative position matrix between the GNSS and the visual sensor.

[0028] Secondly, this application also provides a coordinate system relative matrix determination device based on multiple sensors. The device includes:

[0029] The acquisition module is used to acquire GNSS data, visual sensor data, and IMU data from multiple consecutive time points.

[0030] The alignment module is used to align IMU data with visual sensor data based on the first relative matrix between the visual sensor and the IMU to obtain aligned data.

[0031] The first determining module is used to determine the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the absolute position of GNSS at each time point based on the GNSS data, alignment data, GPS time, and satellite time corresponding to each time point.

[0032] The second determining module is used to determine the second relative matrix between the GNSS and the visual sensor based on the relative velocity, absolute velocity, and absolute position at each time point, as well as the first relative matrix.

[0033] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0034] Acquire GNSS data, visual sensor data, and inertial measurement unit (IMU) data at multiple consecutive time points;

[0035] Based on the first relative matrix between the visual sensor and the IMU, the IMU data and the visual sensor data are aligned to obtain aligned data;

[0036] Based on the GNSS data, alignment data, GPS time, and satellite time corresponding to each time point, determine the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the absolute position of GNSS at each time point;

[0037] Based on the relative velocity, absolute velocity, and absolute position at each moment, and the first relative matrix, the second relative matrix between the GNSS and the visual sensor is determined.

[0038] Fourthly, this application also provides a computer-readable storage medium. This computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0039] Acquire GNSS data, visual sensor data, and inertial measurement unit (IMU) data at multiple consecutive time points;

[0040] Based on the first relative matrix between the visual sensor and the IMU, the IMU data and the visual sensor data are aligned to obtain aligned data;

[0041] Based on the GNSS data, alignment data, GPS time, and satellite time corresponding to each time point, determine the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the absolute position of GNSS at each time point;

[0042] Based on the relative velocity, absolute velocity, and absolute position at each moment, and the first relative matrix, the second relative matrix between the GNSS and the visual sensor is determined.

[0043] Fifthly, this application also provides a computer program product. This computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0044] Acquire GNSS data, visual sensor data, and inertial measurement unit (IMU) data at multiple consecutive time points;

[0045] Based on the first relative matrix between the visual sensor and the IMU, the IMU data and the visual sensor data are aligned to obtain aligned data;

[0046] Based on the GNSS data, alignment data, GPS time, and satellite time corresponding to each time point, determine the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the absolute position of GNSS at each time point;

[0047] Based on the relative velocity, absolute velocity, and absolute position at each moment, and the first relative matrix, the second relative matrix between the GNSS and the visual sensor is determined.

[0048] The aforementioned method, apparatus, and device for determining the coordinate system relative matrix based on multiple sensors acquire GNSS data, visual sensor data, and IMU data at multiple consecutive time points. Based on the first relative matrix between the visual sensor and the IMU, alignment data can be accurately obtained. Processing the alignment data, GNSS data, GNSS time, and satellite time at multiple consecutive time points allows for more precise determination of the relative velocity, absolute velocity, and absolute position of the GNSS at each time point. Based on the relative velocity, absolute velocity, and absolute position at each time point, as well as the first relative matrix, the second relative matrix between the GNSS and the visual sensor can be accurately and reliably determined, thereby completing the fusion of GNSS, visual sensor, and IMU. This achieves the goal of improving the accuracy and reliability of multi-sensor alignment. Attached Figure Description

[0049] Figure 1 This embodiment provides an application environment diagram for a method for determining the relative matrix of a coordinate system based on multiple sensors.

[0050] Figure 2 This is a flowchart illustrating the first method for determining the relative matrix of a coordinate system based on multiple sensors provided in this embodiment.

[0051] Figure 3 This embodiment provides a flowchart for determining the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the absolute position of GNSS at various times.

[0052] Figure 4 This embodiment provides a flowchart illustrating the process of determining a second relative matrix between GNSS and a visual sensor.

[0053] Figure 5 This is a flowchart illustrating the second method for determining the relative matrix of a coordinate system based on multiple sensors provided in this embodiment;

[0054] Figure 6 This is a structural block diagram of the first multi-sensor-based coordinate system relative matrix determination device provided in this embodiment;

[0055] Figure 7 This is a structural block diagram of the second type of coordinate system relative matrix determination device based on multiple sensors provided in this embodiment;

[0056] Figure 8 This is a structural block diagram of the third type of coordinate system relative matrix determination device based on multiple sensors provided in this embodiment;

[0057] Figure 9 This is an internal structural diagram of a computer device provided in this embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] The coordinate system relative matrix determination method based on multiple sensors provided in this application can be applied to, for example... Figure 1 The application environment shown. It can be applied to, for example... Figure 1 In the application environment shown, in one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows. Figure 1 As shown. This computer device includes a processor, memory, and network interface connected via a system bus.

[0060] The computer device's processor provides computational and control capabilities. Its memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The computer device's database stores relevant data for determining the relative matrix of a coordinate system based on multiple sensors. The computer device's network interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a method for determining the relative matrix of a coordinate system based on multiple sensors.

[0061] Before introducing this embodiment, it should be noted that although GNSS can obtain the receiver's absolute velocity and absolute position information at the current moment, the GNSS used in mass-produced autonomous vehicles for autonomous driving projects is generally not very accurate in order to reduce production costs. This makes it difficult to meet the requirements for absolute positioning accuracy; that is, the absolute velocity and absolute position information of the receiver obtained from parsing raw GNSS data is not precise. Parsing visual sensor data and IMU data can obtain more accurate relative position and relative velocity information of the receiver, but it cannot obtain the receiver's absolute position and absolute velocity information. Therefore, aligning the coordinate systems of GNSS, visual sensors, and IMU under the same coordinate system can obtain more accurate absolute position and absolute velocity information of the receiver.

[0062] In one embodiment, such as Figure 2 As shown, a method for determining the relative matrix of a coordinate system based on multiple sensors is provided. This method can be applied to scenarios involving coordinate system alignment of three sensors in a vehicle: a vision sensor, an IMU, and a GNSS sensor. This method can be applied to… Figure 1 Taking a computer as an example, the explanation includes the following steps:

[0063] S201 acquires GNSS data, visual sensor data, and inertial measurement unit (IMU) data at multiple consecutive time points.

[0064] Among these, GNSS data can be absolute velocity and absolute position information received by a GNSS receiver at the current moment. Visual sensor data can be image data received by visual sensors, such as environmental images of the vehicle's surroundings captured by in-vehicle cameras and dashcams. IMU data can be information such as the vehicle's three-axis attitude angles, angular rates, and accelerations measured by the IMU.

[0065] Understandably, because IMUs update much more frequently than GNSS and visual sensors, a dataset acquired within a single sampling period (including GNSS data, visual sensor data, and IMU data) will contain more IMU data than GNSS and visual sensor data. For example, a dataset might contain one GNSS data point, one visual sensor data point, and multiple IMU data points.

[0066] In this embodiment, GNSS data, visual sensor data, and IMU data are acquired at multiple consecutive times (e.g., 10 times) using GNSS, visual sensor data, and IMU data, respectively, and the GNSS data, visual sensor data, and IMU data at the same time are used as a set of sample data.

[0067] S202, based on the first relative matrix between the visual sensor and the IMU, aligns the IMU data with the visual sensor data to obtain aligned data.

[0068] The first relative matrix can be an extrinsic parameter matrix between the visual sensor and the IMU, i.e., a transformation matrix between the coordinate systems of the visual sensor and the IMU. Optionally, the first relative matrix may include a relative attitude matrix and a relative position matrix between the visual sensor and the IMU. Data in the visual sensor coordinate system (i.e., visual sensor data) and data in the IMU coordinate system (i.e., IMU data) can be aligned based on this first relative matrix. The alignment data can be the data information for aligning the visual sensor data and IMU data to the visual sensor coordinate system.

[0069] It should be noted that the alignment data in this embodiment can be the average of the data information of the visual sensor data and IMU data calibrated in the aligned coordinate system, or it can be the data of the visual sensor data calibrated in the aligned coordinate system.

[0070] Optionally, the first relative matrix can be pre-calculated using an offline calibration algorithm, or it can be determined by a pre-trained first relative matrix determination model; there is no limitation on this. For example, a calibration board can be used to calibrate the visual sensor coordinate system and the IMU coordinate system on the board, thereby determining the relative position between the two coordinate systems. Then, based on the relative position between the two coordinate systems and the offline calibration algorithm, the first relative matrix between the visual sensor coordinate system and the IMU coordinate system can be calculated.

[0071] In this embodiment, based on the first relative matrix between the visual sensor coordinate system and the IMU coordinate system obtained by calculation, the visual sensor coordinate system and the IMU coordinate system can be aligned to obtain an aligned coordinate system. The visual sensor data and IMU data are then calibrated in the aligned coordinate system to obtain aligned data.

[0072] S203. Based on the GNSS data, alignment data, GPS time, and satellite time corresponding to each time point, determine the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the absolute position of GNSS at each time point.

[0073] Among them, Global Positioning System time can be the time when navigation and positioning satellites transmit GNSS data, and satellite time can be the most accurate astronomical time.

[0074] Optionally, in this embodiment, the alignment data at each time point can be parsed to obtain the relative velocity between the GNSS and the visual sensor at each time point. The absolute velocity of the GNSS can be obtained through GNSS data parsing; by parsing GNSS data at multiple time points, the absolute velocity of the GNSS at each time point can be obtained. Correspondingly, the absolute position of the GNSS can also be obtained through GNSS data parsing; by parsing GNSS data at multiple time points, the absolute position of the GNSS at each time point can be obtained.

[0075] It should be noted that the absolute position of GNSS obtained from GNSS data parsing is not accurate. In order to make the absolute position of GNSS more accurate, this embodiment can input GNSS data, alignment data, GPS time and satellite time into a pre-trained absolute position determination model. The absolute position determination model processes the GNSS data, alignment data, GPS time and satellite time to obtain the absolute position of GNSS.

[0076] In this embodiment, based on the GNSS data, alignment data, GPS time, and satellite time corresponding to each time moment, and using a pre-trained absolute position determination model, the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the absolute position of GNSS at each time moment can be determined.

[0077] S204. Based on the relative velocity, absolute velocity, and absolute position at each moment, and the first relative matrix, determine the second relative matrix between the GNSS and the visual sensor.

[0078] In this embodiment, based on the relative velocity, absolute velocity, and absolute position of GNSS at each time point and the first relative matrix between the visual sensor and the IMU, the second relative matrix between GNSS and the visual sensor can be obtained by using a pre-set formula for solving the second relative matrix. Alternatively, the second relative matrix between GNSS and the visual sensor can be determined based on a pre-trained model for determining the second relative matrix, thereby completing the fusion of GNSS, visual sensor, and IMU.

[0079] The aforementioned method for determining the coordinate system relative matrix based on multiple sensors acquires GNSS data, visual sensor data, and IMU data at multiple consecutive time points. Based on the first relative matrix between the visual sensor and the IMU, alignment data can be accurately obtained. Processing the alignment data, GNSS data, GNSS time, and satellite time at multiple consecutive time points allows for more precise determination of the relative velocity, absolute velocity, and absolute position of the GNSS at each time point. Based on the relative velocity, absolute velocity, and absolute position at each time point, as well as the first relative matrix, the second relative matrix between the GNSS and the visual sensor can be accurately and reliably determined, thereby completing the fusion of GNSS, visual sensor, and IMU. This achieves the goal of improving the accuracy and reliability of multi-sensor alignment.

[0080] Furthermore, to ensure more accurate absolute position of the GNSS, and to guarantee the subsequent determination of the second relative matrix between the GNSS and the visual sensor based on the absolute position, relative velocity, and first relative matrix of the GNSS at each time point, in one embodiment, such as... Figure 3 As shown, based on the GNSS data, alignment data, GPS time, and satellite time corresponding to each moment, the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the absolute position of GNSS at each moment are determined, including:

[0081] S301, based on the GNSS data and alignment data corresponding to each time moment, determines the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the GNSS time at each time moment.

[0082] GNSS time can be obtained from parsing GNSS data and is used to represent the time when GNSS receives GNSS data. It's understandable that since GNSS continuously acquires GNSS data at multiple times, the GNSS time will correspondingly be the GNSS time at those multiple times.

[0083] Optionally, the relative velocity between GNSS and the visual sensor at each time point can be determined by parsing and processing the obtained alignment data to obtain the relative velocity between GNSS and the visual sensor, or by inputting the alignment data into a pre-set relative velocity determination model, which will automatically output the relative velocity between GNSS and the visual sensor.

[0084] Optionally, the absolute velocity of GNSS at each time point can be determined by parsing and processing the GNSS data obtained at each time point, based on the GNSS data and alignment data corresponding to each time point. Alternatively, the GNSS data can be input into a pre-set absolute velocity determination model, and the model can automatically output the absolute velocity of GNSS.

[0085] In this embodiment, after parsing and processing the GNSS data and alignment data corresponding to each time moment, the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the GNSS time at each time moment can be obtained.

[0086] S302 determines the clock difference information for each moment based on the Global Positioning System time, satellite time, and GNSS time.

[0087] Here, clock difference can refer to the time between the clock time indicating the accurate world at the same instant and the astronomical clock time. It should be noted that the clock difference in this embodiment includes two clock differences, namely the clock difference between GNSS time and Global Positioning System time, and the clock difference between Global Positioning System time and satellite time.

[0088] In this embodiment, based on the Global Positioning System (GPS) time, satellite time, and GNSS time, two clock differences can be determined for each moment: the clock difference between GNSS time and GPS time, and the clock difference between GPS time and satellite time. The clock difference between GNSS time and GPS time can be the time difference between the GNSS time when GNSS data is received and the time when GPS data is transmitted. The clock difference between GPS time and satellite time can be the time difference between GPS time and satellite time.

[0089] S303 determines the absolute position of the GNSS at each time point based on the clock difference information at each time point and the pseudorange single-point positioning algorithm.

[0090] Optionally, in this embodiment, the absolute position of the GNSS at each time point can be determined by calling a pseudorange point positioning algorithm based on the clock difference between the GNSS time and the Global Positioning System time at each time point.

[0091] Optionally, this embodiment can also calculate the absolute position of GNSS at each time point based on the clock difference between GNSS time and Global Positioning System time and the speed of light, according to a preset calculation rule.

[0092] In this embodiment, the absolute position of the GNSS at each time can be determined based on the clock difference between the GNSS time and the Global Positioning System time.

[0093] In the above embodiments, based on the GNSS data and alignment data corresponding to each time moment, the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the GNSS time at each time moment are determined. Then, based on the GPS time, satellite time, and GNSS time, the clock difference information at each time moment is determined. Finally, based on the clock difference information at each time moment, the absolute position of GNSS at each time moment is determined using a pseudorange point positioning algorithm. Determining the absolute position of GNSS at each time moment based on the clock difference information and a pseudorange point positioning algorithm, rather than directly parsing the GNSS data, allows for a more accurate determination of the GNSS absolute position, providing a guarantee for the subsequent determination of the second relative matrix between GNSS and the visual sensor.

[0094] Furthermore, the second relative matrix between GNSS and the vision sensor includes a relative attitude matrix and a relative position matrix. To make the determined second relative matrix between GNSS and the vision sensor more accurate and reliable, the relative attitude matrix between GNSS and the vision sensor can be determined first, then the relative position matrix between GNSS and the vision sensor can be determined, and finally the second relative matrix can be determined. In one embodiment, such as... Figure 4 As shown, based on the relative velocity, absolute velocity, and absolute position at each moment, and the first relative matrix, the second relative matrix between the GNSS and the visual sensor is determined, including:

[0095] S401, based on the relative velocity, absolute velocity and first relative matrix at each time moment, determine the clock difference change at each time moment, as well as the relative attitude matrix between the GNSS and the visual sensor.

[0096] The variation in time difference can be the relationship between the clock difference between GNSS time and GPS time at each moment and the clock difference between GNSS time and GPS time at adjacent moments, as well as the relationship between the clock difference between GPS time and satellite time at each moment and the clock difference between GPS time and satellite time at adjacent moments. The relative attitude matrix can be the rotational transformation relationship between the GNSS coordinate system and the coordinate system aligned with the visual sensor and IMU. In other words, the relative attitude matrix can be used to rotate the GNSS coordinate system and the coordinate system aligned with the visual sensor and IMU to the same direction.

[0097] Optionally, the change in clock bias can be determined based on the clock bias between each moment and adjacent moments, as well as the time difference between adjacent moments. It should be noted that in this embodiment, the corresponding clock bias change information needs to be determined for each moment in which GNSS data, visual sensor data, and IMU data are acquired.

[0098] In this embodiment, determining the clock error change and the relative attitude matrix between the GNSS and the visual sensor at each time step, based on the relative velocity, absolute velocity, and first relative matrix at each time step, can include: constructing a first residual function corresponding to each time step based on a Gauss-Newton nonlinear optimization algorithm, using the relative velocity, absolute velocity, first relative matrix, and a first constant at each time step. Solving for the first residual function at each time step yields the clock error change and the relative attitude matrix between the GNSS and the visual sensor at each time step.

[0099] The first residual function includes the clock bias variation to be determined and the relative attitude matrix between the GNSS and the visual sensor; the first constant includes: positioning value, speed of light, Doppler carrier wavelength, Doppler frequency shift, and pseudorange single-point compensation. For example, the first residual function can be represented by the following formula (1):

[0100]

[0101] In the formula, res represents the residual. This represents the first relative matrix between the vision sensor and the IMU. The attitude variables consist of yaw, roll, and pitch. It should be noted that in this embodiment, the roll and pitch angles can be assumed to remain constant, and only the effect of the yaw angle is considered. w v represents the relative velocity at each moment. s e represents the absolute velocity at each moment. s This represents the location value, which is a constant, and c is the speed of light. f represents the change in clock difference between GNSS time and Global Positioning System time at various times, where λ represents the Doppler carrier wavelength, which is a constant. d Indicates Doppler frequency shift, n represents the change in clock difference between Global Positioning System time and satellite time. d The amount of compensation for a single pseudo-distance point is a constant.

[0102] In this embodiment, based on the relative velocity, absolute velocity, first relative matrix, and first constant at each time point, and using the constructed first residual function, the clock error change at each time point and the relative attitude matrix between the GNSS and the visual sensor at each time point can be solved, providing a basis for determining the relative attitude matrix between the GNSS and the visual sensor.

[0103] Optionally, in this embodiment, for the residual function corresponding to each moment, the values ​​of the clock error change and the relative attitude matrix in the residual function can be adjusted to obtain the clock error change and the relative attitude matrix corresponding to the function value of the residual function when the function value is a preset value, which are used as the clock error change and the relative attitude matrix corresponding to that moment; based on the relative attitude matrix corresponding to each moment, the final relative attitude matrix between GNSS and the visual sensor is determined.

[0104] Optionally, the measured v at each time point w and v s The first relative matrix between the visual sensor and the IMU And by substituting each constant into the above formula (1), and adjusting... and This process brings the residual res close to a preset value (e.g., close to 0), completing the optimization and obtaining the result at this point. and As the relative attitude matrix and clock error change at that moment, that is, through the first residual function, the corresponding values ​​at each moment can be obtained. and Then, for each time point... and The process is performed to obtain the clock difference changes at each time point. and and the relative attitude matrix between GNSS and visual sensors Optionally, for each time point... and The processing can be performed according to a pre-set processing strategy to obtain the clock difference change at each time point. and and the relative attitude matrix between GNSS and visual sensors

[0105] In this embodiment, for the first residual function corresponding to each time moment, the values ​​of the clock error change and the relative attitude matrix in the residual function are adjusted to obtain the clock error change and the relative attitude matrix corresponding to the function value of the residual function at a preset value, making the process of obtaining the relative attitude matrix between GNSS and visual sensor at each time moment more convenient; and based on the relative attitude matrix between GNSS and visual sensor at each time moment, the relative attitude matrix between GNSS and visual sensor is determined, making the relative attitude matrix between GNSS and visual sensor more accurate and reliable.

[0106] S402, update the clock difference information for each time point based on the change in clock difference at each time point and the time difference between adjacent time points.

[0107] Optionally, updating the clock difference information at each time point can be done by updating it based on a pre-set clock difference update model, using the change in clock difference at each time point and the time difference between adjacent time points, to obtain the updated clock difference information at each time point. Alternatively, it can be done by updating the clock difference at each time point based on the relationship between the change in clock difference, the time difference between adjacent time points, and the clock difference itself. For example, the clock difference at the second time point can be the change in clock difference at the first time point multiplied by the time difference between the first and second time points.

[0108] In this embodiment, based on the change in clock difference at each time point and the time difference between adjacent times points, and based on the relationship between the change in clock difference, the time difference between adjacent times points and the clock difference, the clock difference information at each time point can be updated to obtain the updated clock difference information at each time point.

[0109] S403 determines the relative position matrix between the GNSS and the visual sensor based on the clock difference information updated after the absolute position at each time.

[0110] The relative position matrix can be the translation transformation relationship between the GNSS coordinate system and the coordinate system aligned with the vision sensor and IMU. In other words, the GNSS coordinate system and the coordinate system aligned with the vision sensor and IMU can be translated to the same position according to the relative attitude matrix.

[0111] In this embodiment, based on the absolute position at each time step, the updated clock difference information at each time step, and the second constant, a second residual function corresponding to each time step is constructed using a Gauss-Newton nonlinear optimization algorithm. Solving for the second residual function at each time step yields the relative position matrix between the GNSS and the visual sensor.

[0112] The second residual function includes the relative position matrix between the GNSS and the visual sensor to be determined; the second constant includes: the speed of light, pseudorange, a preset identity matrix, and delay information; wherein the delay information includes delays caused by the troposphere, delays caused by the ionosphere, and delays caused by other factors. For example, the second residual function can be represented by the following formula (2):

[0113]

[0114] In the formula, res represents the residual, P e This indicates the absolute position of the GNSS in the GNSS coordinate system, which can be inferred from the position of the GNSS in the coordinate system after aligning the line-of-sight sensor coordinate system and the IMU coordinate system, i.e.: in, This represents the first relative matrix between the vision sensor and the IMU. This represents the attitude variables optimized by the first residual function. It can be calculated from the positions of the GNSS receiver to the visual sensor and the IMU; it is a constant. This represents the relative position matrix between the GNSS and the visual sensor. [0] k [1] k and [2] k Let c represent the preset identity matrix, which consists of preset k-order 0, k-order 1, and k-order 2 matrices, respectively. Let c be the speed of light, and δ be the value of the matrix. t ρ represents the clock difference between GNSS time and Global Positioning System time at each moment, ρ represents the delay caused by other factors, and δ represents the time difference. t,s The clock difference between GPS time and satellite time is represented by I, which represents the delay caused by the atmospheric ionosphere, T, which represents the delay caused by the atmospheric troposphere, and ζ, which represents the pseudorange.

[0115] According to formula (2) above, the relative position matrix between GNSS and the visual sensor at each time step can be determined by using the same method as determining the relative attitude matrix between GNSS and the visual sensor at each time step. The specific method will not be elaborated here. Alternatively, the residual res can be set to a preset value and solved directly. As the relative position matrix between GNSS and visual sensor at each time point, the relative position matrix between GNSS and visual sensor at each time point is then processed according to a pre-set processing strategy to obtain the relative position matrix between GNSS and visual sensor.

[0116] In this embodiment, by adjusting the value of the relative attitude matrix, the relative position matrix corresponding to the function value of the second residual function at a preset value is obtained, making the process of obtaining the relative position matrix between GNSS and the visual sensor at each time point more convenient. Furthermore, the clock error used in the second residual function is an updated clock error, which makes the solution of the second residual function more accurate. Based on the relative position matrix between GNSS and the visual sensor at each time point, the relative position matrix between GNSS and the visual sensor is determined, making the relative position matrix between GNSS and the visual sensor more accurate and reliable.

[0117] In the above embodiments, firstly, based on the relative velocity, absolute velocity, and first relative matrix at each moment, the clock difference change and the relative attitude matrix between GNSS and the visual sensor are determined. The clock difference is then updated using the determined clock difference change and the time difference between adjacent moments to obtain a more accurate clock difference at each moment. Based on the updated clock difference and absolute position at each moment, the relative position matrix between GNSS and the visual sensor is determined. Finally, a second relative matrix between GNSS and the visual sensor is determined based on the relative position matrix and the relative attitude matrix, making the second relative matrix between GNSS and the visual sensor more accurate and reliable.

[0118] To facilitate understanding by those skilled in the art, the above-described method for determining the relative matrix of a coordinate system based on multiple sensors will be described in detail, such as... Figure 5 As shown, the method may include:

[0119] S501 acquires GNSS data, visual sensor data, and IMU data from multiple consecutive time points.

[0120] S502 aligns IMU data with visual sensor data based on the first relative matrix between the visual sensor and the IMU to obtain aligned data.

[0121] S503 determines the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the GNSS time at each time point based on the GNSS data and alignment data corresponding to each time point.

[0122] S504 determines the clock difference information for each moment based on the Global Positioning System time, satellite time, and GNSS time.

[0123] S505 determines the absolute position of the GNSS at each time point based on the clock difference information and the pseudorange single-point positioning algorithm.

[0124] S506. Based on the relative velocity, absolute velocity, first relative matrix, and first constant at each time step, construct the first residual function corresponding to each time step using the Gauss-Newton nonlinear optimization algorithm.

[0125] The first residual function includes the clock error change to be determined and the relative attitude matrix between the GNSS and the visual sensor; the first constant includes: positioning value, speed of light, Doppler carrier wavelength, Doppler frequency shift and pseudorange single-point compensation.

[0126] S507, for the residual function corresponding to each time moment, adjust the values ​​of the clock error change and the relative attitude matrix in the residual function, and obtain the clock error change and the relative attitude matrix corresponding to the function value of the residual function when the function value is a preset value, and use them as the clock error change and the relative attitude matrix corresponding to that time moment.

[0127] S508 determines the final relative attitude matrix between the GNSS and the visual sensor based on the relative attitude matrix corresponding to each moment.

[0128] S509, update the clock difference information for each time point based on the change in clock difference at each time point and the time difference between adjacent time points.

[0129] S510: Based on the absolute position at each time step, the updated clock difference information at each time step, and the second constant, construct the second residual function corresponding to each time step using the Gauss-Newton nonlinear optimization algorithm.

[0130] The second residual function contains the relative position matrix between the GNSS and the vision sensor to be determined; the second constant includes: the speed of light, pseudorange, preset identity matrix and delay information.

[0131] S511, solve for the second residual function at each time point to obtain the relative position matrix between the GNSS and the visual sensor.

[0132] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0133] Based on the same inventive concept, this application also provides a multi-sensor-based coordinate system relative matrix determination device for implementing the above-described multi-sensor-based coordinate system relative matrix determination method. The solution provided by this device is similar to the implementation described in the above-described method. Therefore, the specific limitations of one or more embodiments of the multi-sensor-based coordinate system relative matrix determination device provided below can be found in the limitations of the multi-sensor-based coordinate system relative matrix determination method described above, and will not be repeated here.

[0134] In one embodiment, such as Figure 6 As shown, a coordinate system relative matrix determination device 1 based on multiple sensors is provided, comprising: an acquisition module 10, an alignment module 11, a first determination module 12, and a second determination module 13, wherein:

[0135] The acquisition module 10 is used to acquire GNSS data, visual sensor data and IMU data at multiple consecutive time points.

[0136] Alignment module 11 is used to align IMU data and visual sensor data based on a first relative matrix between the visual sensor and the IMU to obtain aligned data.

[0137] The first determining module 12 is used to determine the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the absolute position of GNSS at each time point based on the GNSS data, alignment data, global positioning system time, and satellite time corresponding to each time point.

[0138] The second determining module 13 is used to determine the second relative matrix between the GNSS and the visual sensor based on the relative velocity, absolute velocity, and absolute position at each time point, as well as the first relative matrix.

[0139] In one embodiment, such as Figure 7 As shown, the first determining module 12 includes a first determining unit 120, a second determining unit 121, and a third determining unit 122. Wherein:

[0140] The first determining unit 120 is used to determine the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the GNSS time at each time point based on the GNSS data and alignment data corresponding to each time point.

[0141] The second determining unit 121 is used to determine the clock difference information at each time based on the global positioning system time, satellite time, and GNSS time.

[0142] The third determining unit 122 is used to determine the absolute position of the GNSS at each time based on the clock difference information at each time and the pseudorange single-point positioning algorithm.

[0143] In one embodiment, the second relative matrix between the GNSS and the visual sensor includes a relative attitude matrix and a relative position matrix. For example... Figure 8 As shown, the second determining module 13 includes a fourth determining unit 130, an updating unit 131, and a fifth determining unit 132. Wherein:

[0144] The fourth determining unit 130 is used to determine the clock difference change at each time moment and the relative attitude matrix between the GNSS and the visual sensor based on the relative velocity, absolute velocity and the first relative matrix at each time moment.

[0145] The update unit 131 is used to update the clock difference information at each time point based on the change in clock difference at each time point and the time difference between adjacent time points.

[0146] The fifth determining unit 132 is used to determine the relative position matrix between the GNSS and the visual sensor based on the absolute position at each time and the updated clock difference information at each time.

[0147] In one embodiment, the fourth determining unit 130 may include a constructing subunit and a solving subunit.

[0148] in:

[0149] Construct sub-units to build the first residual function corresponding to each time step based on the relative velocity, absolute velocity, first relative matrix, and first constant at each time step using the Gauss-Newton nonlinear optimization algorithm.

[0150] The first residual function includes the clock error change to be determined and the relative attitude matrix between the GNSS and the visual sensor; the first constant includes: positioning value, speed of light, Doppler carrier wavelength, Doppler frequency shift and pseudorange single-point compensation.

[0151] The solution sub-element is used to solve the first residual function corresponding to each time step, obtain the clock error change at each time step, and the relative attitude matrix between GNSS and visual sensor.

[0152] In one embodiment, the solving subunit is specifically used to adjust the values ​​of the clock error change and the relative attitude matrix in the residual function corresponding to each time moment, and obtain the clock error change and the relative attitude matrix corresponding to the function value of the residual function when the function value is a preset value, as the clock error change and the relative attitude matrix corresponding to that time moment; and determine the final relative attitude matrix between GNSS and visual sensor based on the relative attitude matrix corresponding to each time moment.

[0153] In one embodiment, the fifth determining unit 132 is specifically used to construct a second residual function corresponding to each time moment based on the absolute position at each time moment, the updated clock difference information at each time moment, and a second constant, using a Gauss-Newton nonlinear optimization algorithm; wherein, the second residual function contains the relative position matrix between the GNSS and the visual sensor to be determined; the second constant includes: the speed of light, pseudorange, a preset identity matrix, and delay information; and solves the second residual function corresponding to each time moment to obtain the relative position matrix between the GNSS and the visual sensor.

[0154] Each module in the aforementioned multi-sensor-based coordinate system relative matrix determination device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0155] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for determining the relative matrix of a coordinate system based on multiple sensors. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0156] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0157] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0158] Acquire GNSS data, visual sensor data, and IMU data from multiple consecutive time points;

[0159] Based on the first relative matrix between the visual sensor and the IMU, the IMU data and the visual sensor data are aligned to obtain aligned data;

[0160] Based on the GNSS data, alignment data, GPS time, and satellite time corresponding to each time point, determine the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the absolute position of GNSS at each time point;

[0161] Based on the relative velocity, absolute velocity, and absolute position at each moment, and the first relative matrix, the second relative matrix between the GNSS and the visual sensor is determined.

[0162] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0163] Acquire GNSS data, visual sensor data, and IMU data from multiple consecutive time points;

[0164] Based on the first relative matrix between the visual sensor and the IMU, the IMU data and the visual sensor data are aligned to obtain aligned data;

[0165] Based on the GNSS data, alignment data, GPS time, and satellite time corresponding to each time point, determine the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the absolute position of GNSS at each time point;

[0166] Based on the relative velocity, absolute velocity, and absolute position at each moment, and the first relative matrix, the second relative matrix between the GNSS and the visual sensor is determined.

[0167] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0168] Acquire GNSS data, visual sensor data, and IMU data from multiple consecutive time points;

[0169] Based on the first relative matrix between the visual sensor and the IMU, the IMU data and the visual sensor data are aligned to obtain aligned data;

[0170] Based on the GNSS data, alignment data, GPS time, and satellite time corresponding to each time point, determine the relative velocity between GNSS and the visual sensor, the absolute velocity of GNSS, and the absolute position of GNSS at each time point;

[0171] Based on the relative velocity, absolute velocity, and absolute position at each moment, and the first relative matrix, the second relative matrix between the GNSS and the visual sensor is determined.

[0172] It should be noted that the information (including but not limited to GNSS information, visual sensor information, and IMU information) and data (including but not limited to alignment data, GNSS data, and visual data) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0173] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0174] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0175] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A multi-sensor based method for determining a relative matrix of coordinate systems, characterized by, The method includes: Acquire GNSS data, visual sensor data, and inertial measurement unit (IMU) data at multiple consecutive time points; Based on the first relative matrix between the visual sensor and the IMU, the IMU data and the visual sensor data are aligned to obtain aligned data; Based on the GNSS data, alignment data, GPS time, and satellite time corresponding to each time point, determine the relative velocity between the GNSS and the visual sensor, the absolute velocity of the GNSS, and the absolute position of the GNSS at each time point; Based on the relative velocity, the absolute velocity, and the first relative matrix at each moment, determine the clock difference change at each moment, as well as the relative attitude matrix between the GNSS and the visual sensor; Update the clock difference information for each time point based on the change in clock difference at each time point and the time difference between adjacent time points; Based on the absolute position at each time point and the updated clock difference information at each time point, the relative position matrix between the GNSS and the visual sensor is determined.

2. The method of claim 1, wherein, The step of determining the relative velocity between the GNSS and the visual sensor, the absolute velocity of the GNSS, and the absolute position of the GNSS at each time point based on the GNSS data, the alignment data, the Global Positioning System time, and the satellite time, includes: Based on the GNSS data and alignment data corresponding to each time moment, determine the relative velocity between the GNSS and the visual sensor, the absolute velocity of the GNSS, and the GNSS time at each time moment; Based on the Global Positioning System time, satellite time, and the GNSS time, determine the clock difference information for each moment; Based on the clock difference information at each time point, the absolute position of the GNSS at each time point is determined using a pseudorange single-point positioning algorithm.

3. The method according to claim 1, characterized in that, The step of determining the clock difference change at each moment, and the relative attitude matrix between the GNSS and the visual sensor, based on the relative velocity, the absolute velocity, and the first relative matrix at each moment, includes: Based on the relative velocity, absolute velocity, first relative matrix, and first constant at each moment, a first residual function corresponding to each moment is constructed using the Gauss-Newton nonlinear optimization algorithm. The first residual function includes the clock error change to be determined and the relative attitude matrix between the GNSS and the visual sensor. The first constant includes: positioning value, speed of light, Doppler carrier wavelength, Doppler frequency shift, and pseudorange single-point compensation. Solve for the first residual function at each time step to obtain the clock error change at each time step, as well as the relative attitude matrix between the GNSS and the visual sensor.

4. The method according to claim 3, characterized in that, The process of solving the first residual function corresponding to each time step to obtain the clock error change at each time step, and the relative attitude matrix between the GNSS and the visual sensor, includes: For each time step, the values ​​of clock error change and relative attitude matrix in the residual function are adjusted to obtain the clock error change and relative attitude matrix corresponding to the function value of the residual function when the function value is a preset value, which are used as the clock error change and relative attitude matrix corresponding to that time step. Based on the relative attitude matrix corresponding to each moment, the final relative attitude matrix between the GNSS and the visual sensor is determined.

5. The method according to claim 1, characterized in that, The step of determining the relative position matrix between the GNSS and the visual sensor based on the absolute position at each time moment and the updated clock difference information at each time moment includes: Based on the absolute position at each time point, the updated clock difference information at each time point, and the second constant, a second residual function corresponding to each time point is constructed based on the Gauss-Newton nonlinear optimization algorithm; wherein, the second residual function contains the relative position matrix between the GNSS and the visual sensor to be determined; the second constant includes: speed of light, pseudorange, preset identity matrix, and delay information; Solve for the second residual function at each time point to obtain the relative position matrix between the GNSS and the visual sensor.

6. The method according to claim 2, characterized in that, The determination of clock difference information for each moment based on Global Positioning System time, satellite time, and GNSS time includes: The time difference between the GNSS time when the GNSS receives the GNSS data and the time when the Global Positioning System (GPS) transmits the GNSS data is taken as the clock difference between the GNSS time and the GPS time; and, The time difference between the Global Positioning System (GPS) time and the satellite time is used as the clock difference between GPS time and satellite time.

7. A coordinate system relative matrix determination device based on multiple sensors, characterized in that, The device includes: The acquisition module is used to acquire GNSS data, visual sensor data, and IMU data from multiple consecutive time points. An alignment module is used to align the IMU data with the visual sensor data based on a first relative matrix between the visual sensor and the IMU to obtain aligned data. The first determining module is used to determine the relative velocity between the GNSS and the visual sensor, the absolute velocity of the GNSS, and the absolute position of the GNSS at each time point based on the GNSS data, the alignment data, the Global Positioning System time, and the satellite time corresponding to each time point. The second determining module is used to determine the clock difference change at each time moment and the relative attitude matrix between the GNSS and the visual sensor based on the relative velocity, the absolute velocity, and the first relative matrix at each time moment; update the clock difference information at each time moment based on the clock difference change at each time moment and the time difference between adjacent time moments; and determine the relative position matrix between the GNSS and the visual sensor based on the absolute position at each time moment and the updated clock difference information at each time moment.

8. The apparatus according to claim 7, characterized in that, The first determining module includes: The first determining unit is configured to determine the relative velocity between the GNSS and the visual sensor, the absolute velocity of the GNSS, and the GNSS time at each time point based on the GNSS data and the alignment data corresponding to each time point. The second determining unit is used to determine the clock difference information at each moment based on the global positioning system time, satellite time, and the GNSS time. The third determining unit is used to determine the absolute position of the GNSS at each time based on the clock difference information at each time and the pseudorange single-point positioning algorithm.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images