Inertial navigation device and system based on multi-modal data analysis

Abstract

The invention discloses an inertial navigation device and system based on multi-modal data analysis, and relates to the field of navigation, and the inertial navigation device comprises an IMU measurement unit, an RTC, a PMIC module, a security module, a processor module, a storage module, an interface module, an NPU, a visual sensor and a terminal. According to the invention, a visual sensor and an NPU are added in a traditional inertial navigation system, and a corresponding algorithm is matched, so that the inertial navigation information is corrected by combining real-time visual information with an AI model before the inertial navigation information is output in order to solve the problem that inertial navigation errors are accumulated along with time in the inertial navigation process; and the information output accuracy of the inertial navigation system is improved.

Description

Technical Field

[0001] The present invention relates to the field of navigation, and in particular to an inertial navigation device and system based on multimodal data analysis. Background Art

[0002] As an important pillar in the field of modern navigation, inertial navigation has a long history and a solid technical foundation. Its core advantage lies in its high degree of autonomy, which enables it to break away from its dependence on external information sources and operate steadily in extremely complex environments such as the absence of satellite signals or severe electromagnetic interference, providing solid support for the precise navigation of various unmanned equipment and transportation vehicles, while also relying on its good concealment.

[0003] Today's inertial navigation systems have taken the following main measures: first, regular calibration strategy, second, the use of combined navigation solutions, and third, optimization from within the system.

[0004] Today's inertial navigation systems fall into the category of post-error correction or pre-optimization of sensor performance. In the actual operation of the inertial navigation system, there is a relative lack of real-time and dynamic correction methods for errors, making it difficult to make timely and accurate corrections to errors during the navigation process. Summary of the invention

[0005] Based on this, the purpose of the present invention is to provide an inertial navigation device and system based on multimodal data analysis to solve the technical problem.

[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an inertial navigation device based on multimodal data analysis, comprising an IMU measurement unit, an RTC, a PMIC module, a security module, a processor module, a storage module, an interface module, an NPU, a visual sensor, and a terminal, wherein the IMU measurement unit is electrically connected to the PMIC module, the IMU measurement unit is signal-connected to the processor module, the RTC is electrically connected to the PMIC module, the RTC is signal-connected to the processor module, the PMIC module is electrically connected to the security module, the PMIC module is electrically connected to the processor module, and the PMI The C module is electrically connected to the storage module, the PMIC module is electrically connected to the interface module, the PMIC module is electrically connected to the NPU, the PMIC module is electrically connected to the visual sensor, the security module is signal-connected to the processor module, the security module is signal-connected to the interface module, the processor module is signal-connected to the storage module, the processor module is signal-connected to the NPU, the processor module is signal-connected to the visual sensor, the storage module is signal-connected to the visual sensor, the interface module is signal-connected to the terminal, and the NPU is signal-connected to the visual sensor.

[0007] The present invention is further configured such that the processor module is responsible for data processing of the IMU measurement unit, parameter correction model calculation and data processing during the inertial navigation process and overall control of the entire system.

[0008] IMU measurement unit: measures the acceleration and angular velocity of the carrier in real time, and uses corresponding algorithms to calculate the carrier's posture, velocity and position information.

[0009] Visual sensor: Provides external scenes within a fixed range around the carrier and provides real-time visual information for the AI ​​reasoning model.

[0010] NPU: Accelerates a large number of matrix operations in the image AI model, efficiently processes image data, and cooperates with the AI ​​model to infer position, posture, and speed information.

[0011] Security module: Security isolation in the execution software prevents external intrusion into the inertial navigation system through the interface module software, and uses data isolation to protect data information security.

[0012] RTC: Provides real-time clock for the entire inertial navigation system.

[0013] Storage module: stores acquired relevant data, execution programs, algorithm programs, etc.

[0014] PMIC module: provides power to different components of the entire inertial navigation system.

[0015] Interface module: provides various common external interfaces to facilitate the access of various mobile terminal devices.

[0016] The present invention is further configured such that the system process is specifically as follows: S1: the inertial navigation system is started, the processor module starts to load the driver program, configures predetermined parameters, and configures corresponding navigation modes: two-dimensional plane navigation mode and three-dimensional space navigation mode. Subsequently, the accelerometer and gyroscope in the IMU measurement unit and the visual sensor perform self-inspection, and at the same time, the accelerometer and gyroscope perform initial alignment, and the visual sensor performs automatic visual correction.

[0017] S2: The accelerometer, gyroscope and visual sensor in the IMU measurement unit start working synchronously, and start measuring the acceleration and angular velocity of the carrier and the environmental image of the preset range around the carrier respectively.

[0018] S3: According to the measured angular velocity, acceleration, and the preset carrier mode, the corresponding algorithm is configured, and the corresponding attitude angles are calculated through the gyroscope measurement data, including: the carrier's pitch angle, roll angle and heading angle. It is often necessary to convert the coordinates to obtain the updated carrier's attitude information in the reference coordinate system, and then use the acceleration information measured by the accelerometer and the updated attitude information to calculate the carrier's position information and speed information in the navigation coordinate system, and obtain the carrier's rough inertial navigation information. The updated position, attitude and speed information of the carrier, after obtaining the carrier's posture information, perform corresponding algorithm reasoning through the visual sensor, construct a parameter correction model, and obtain the final carrier's inertial navigation information.

[0019] S4: After a minimum carrier inertial navigation information update, the error compensation model of the IMU measurement unit and the vision-based parameter correction model are multimodally fused, and the corresponding AI model is used for accelerated reasoning to compensate and correct errors of the accelerometer and gyroscope in the IMU measurement unit. The S, S, and S processes are continued until the terminal turns off the inertial navigation system.

[0020] The present invention is further configured such that in step S3, a visual sensor and an NPU module that cooperates with AI model reasoning are added to the inertial navigation system to construct an inertial navigation parameter correction model, which can compensate and correct the carrier inertial navigation information in real time during the inertial navigation process, thereby improving the accuracy of the inertial navigation information.

[0021] The present invention is further configured such that in step S4, multimodal fusion is performed through the error compensation model of the traditional IMU measurement unit and the vision-based parameter correction model, and the corresponding AI model is used for accelerated reasoning to achieve fast sensor parameter correction, thereby reducing the cumulative error caused by the inertial navigation time, thereby further improving the accuracy of inertial navigation and the effective duration of inertial navigation.

[0022] In summary, the present invention mainly has the following beneficial effects:

[0023] The present invention adds visual sensors and NPU to the traditional inertial navigation system, cooperates with corresponding algorithms, and realizes that in the process of inertial navigation, in order to solve the problem that the inertial navigation error accumulates over time, the inertial navigation information is corrected by using real-time visual information in combination with an AI model before the inertial navigation information is output, thereby improving the accuracy of the information output by the inertial navigation system. At the same time, after a minimum inertial navigation data output is completed, the relevant parameters of the IMU measurement unit are timely calibrated, thereby reducing the error accumulated by the IMU measurement unit with the inertial navigation, further improving the accuracy of the inertial navigation information and extending the effective time of the inertial navigation. Compared with traditional solutions, the present invention is more accurate, intelligent and has a longer effective time, so that the application accuracy of the inertial navigation is higher, the application time is longer, and the application scenarios are more. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a system framework diagram of the present invention;

[0025] Figure 2 It is an operation flow chart of the present invention.

[0026] In the figure: 1. IMU measurement unit; 2. RTC; 3. PMIC module; 4. Security module; 5. Processor module; 6. Storage module; 7. Interface module; 8. NPU; 9. Visual sensor; 10. Terminal. DETAILED DESCRIPTION

[0027] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0028] The following describes an embodiment of the present invention based on its overall structure.

[0029] An inertial navigation device and system based on multimodal data analysis, such as Figure 1As shown, it includes an IMU measurement unit 1, an RTC2, a PMIC module 3, a security module 4, a processor module 5, a storage module 6, an interface module 7, an NPU8, a visual sensor 9, and a terminal 10. The IMU measurement unit 1 is electrically connected to the PMIC module 3, the IMU measurement unit 1 is signal-connected to the processor module 5, the RTC2 is electrically connected to the PMIC module 3, the RTC2 is signal-connected to the processor module 5, the PMIC module 3 is electrically connected to the security module 4, the PMIC module 3 is electrically connected to the processor module 5, and the PMIC module 3 is signal-connected to the storage module 6 is electrically connected, the PMIC module 3 is electrically connected to the interface module 7, the PMIC module 3 is electrically connected to the NPU 8, the PMIC module 3 is electrically connected to the visual sensor 9, the security module 4 is signal-connected to the processor module 5, the security module 4 is signal-connected to the interface module 7, the processor module 5 is signal-connected to the storage module 6, the processor module 5 is signal-connected to the NPU 8, the processor module 5 is signal-connected to the visual sensor 9, the storage module 6 is signal-connected to the visual sensor 9, the interface module 7 is signal-connected to the terminal 10, and the NPU 8 is signal-connected to the visual sensor 9;

[0030] The IMU measurement unit 1, the RTC 2, the PMIC module 3, the processor module 5, the storage module 6, the NPU 8, and the visual sensor 9 can realize inertial navigation based on multimodal data analysis.

[0031] It should be pointed out that the terminal operator turns on the inertial navigation function through the terminal 10. This instruction will be transmitted to the security module 4 through the interface module 9 for corresponding identification and security verification after certain security processing. After the identification and verification are completed, the processor module 5 will issue instructions, and the entire system will enter the pre-startup. The IMU measurement unit 1 and the visual sensor 9 will start preloading and perform corresponding calibrations. The driver in the storage module 6 will be preloaded, and the corresponding model will be loaded. The processor module 5 starts to deploy the AI ​​reasoning algorithm to the NPU 8. When the pre-startup work is completed, the inertial navigation system will perform a self-test. After the self-test is completed, the IMU measurement unit will collect data and enter the inertial navigation process.

[0032] When the inertial navigation system enters the inertial navigation process, the gyroscope and accelerometer in the IMU measurement unit 1 collect angular velocity and acceleration respectively, and the collected data will be stored in the memory module 6. The processor module 5 will calculate the pitch angle, roll angle and heading angle parameters of the current carrier through the angular velocity measured by the gyroscope in conjunction with the corresponding algorithm, and then through the coordinate system conversion matrix and related calculations, the updated attitude information of the carrier can be obtained. Subsequently, the acceleration measured by the accelerometer in the IMU measurement unit 1 needs to be converted from the carrier coordinate system to the navigation coordinate system, and then the acceleration in the navigation coordinate system and the updated carrier attitude information are used for vector calculation to obtain the position information and speed information of the carrier in the navigation coordinate system. At this point, the rough inertial navigation information of the carrier will be obtained, and the real-time image information collected by the visual sensor will be combined with the corresponding AI model, and then the NPU8 will be used to accelerate the inference of the AI ​​model, and the correction data will be output by the AI ​​model. The correction data is used to correct the model of the rough inertial navigation information of the previous carrier, and finally more accurate inertial navigation information will be obtained).

[0033] Embodiment 2

[0034] Processor module: responsible for the data processing of the IMU measurement unit, parameter correction model calculation, data processing and overall control of the entire system during the inertial navigation process.

[0035] IMU measurement unit 1: measures the acceleration and angular velocity of the carrier in real time, and calculates the information of the carrier's posture, velocity and position with the corresponding algorithm.

[0036] Visual sensor 9: Provides external scenes within a fixed range around the carrier and provides real-time visual information for the AI ​​reasoning model. Visual sensor 9 is usually fixed on the carrier or has a fixed coordinate system correspondence. First, the image coordinate system and the carrier coordinate system are bound to obtain the conversion matrix between the two. Then, the real-time image is combined with the corresponding dynamic pose estimation algorithm, and then NPU8 is used for accelerated deduction to update the carrier's pose information in real time. Then, this pose information is used to adjust and correct the previously obtained rough inertial navigation information of the carrier, and finally output a more accurate inertial navigation information.

[0037] NPU8: Accelerates a large number of matrix operations in image AI models, efficiently processes image data, and cooperates with AI models to infer position, posture, and speed information.

[0038] Security module 4: Security isolation on the execution software prevents external intrusion into the inertial navigation system through the interface module software, and uses data isolation to protect data information security.

[0039] RTC2: Provides real-time clock for the entire inertial navigation system.

[0040] Storage module 6: stores acquired relevant data, execution programs, algorithm programs, etc.

[0041] PMIC module 3: Provides power to the different components of the entire inertial navigation system.

[0042] Interface module 7: provides various common external interfaces to facilitate access to various mobile terminal devices. It can be understood that the IMU measurement unit 1, RTC 2, PMIC module 3, processor module 5, storage module 6, NPU 8, and visual sensor 9 can realize inertial navigation based on multimodal data analysis.

[0043] Embodiment 3

[0044] The specific system process is as follows: S1: The inertial navigation system is started, the processor module starts to load the driver, configure the preset parameters, and configure the corresponding navigation mode two-dimensional plane navigation mode and three-dimensional space navigation mode. Then the accelerometer and gyroscope in the IMU measurement unit and the visual sensor perform self-inspection. At the same time, the accelerometer and gyroscope are initially aligned, and the visual sensor performs automatic visual correction.

[0045] S2: The accelerometer, gyroscope and visual sensor in the IMU measurement unit start working synchronously, and start measuring the acceleration and angular velocity of the carrier and the environmental image of the preset range around the carrier respectively.

[0046] S3: According to the measured angular velocity, acceleration, and the preset carrier mode, the corresponding algorithm is configured, and the corresponding attitude angles are calculated through the gyroscope measurement data, including: the carrier's pitch angle, roll angle and heading angle. It is often necessary to convert the coordinates to obtain the updated carrier's attitude information in the reference coordinate system, and then use the acceleration information measured by the accelerometer and the updated attitude information to calculate the carrier's position information and speed information in the navigation coordinate system, and obtain the carrier's rough inertial navigation information. The updated position, attitude and speed information of the carrier, after obtaining the carrier's posture information, perform corresponding algorithm reasoning through the visual sensor, construct a parameter correction model, and obtain the final carrier's inertial navigation information.

[0047] S4: After the minimum carrier inertial navigation information is updated, the error compensation model of the IMU measurement unit and the vision-based parameter correction model are multimodally fused, and the corresponding AI model is used for accelerated reasoning to compensate and correct the errors of the accelerometer and gyroscope in the IMU measurement unit. The S2, S3 and S4 processes are then continued until the terminal shuts down the inertial navigation system.

[0048] It should be noted that in step S3, a visual sensor and an NPU module that cooperates with AI model reasoning are added to the inertial navigation system to construct an inertial navigation parameter correction model, which can compensate and correct the carrier inertial navigation information in real time during the inertial navigation process, thereby improving the accuracy of the inertial navigation information; in step S4, multimodal fusion is performed through the error compensation model of the traditional IMU measurement unit and the vision-based parameter correction model, and the corresponding AI model is used for accelerated reasoning to achieve fast sensor parameter correction, thereby reducing the cumulative error caused by the inertial navigation time, so as to further improve the accuracy of inertial navigation and the effective duration of inertial navigation. It can be understood that: when the inertial navigation system measures angular velocity and acceleration through the gyroscope and accelerometer in the IMU measurement unit 1, the processor module 5 calculates the preliminary carrier update information according to the corresponding algorithm, and then uses the visual sensor 9 to collect the carrier environment data in real time, and then cooperates with the corresponding reasoning algorithm and cooperates with NPU8 to accelerate the deduction, and then the correction data deduced is used to correct the preliminary carrier inertial navigation information, and finally obtain more accurate carrier inertial navigation information. At the same time, after the single minimum carrier update information is output, the errors of the gyroscope and accelerometer in the IMU measurement unit will be compensated according to the deduced correction data. Compared with previous inertial navigation systems, this method is more accurate, intelligent and has a longer effective time, which makes the application accuracy of inertial navigation higher, the application time longer, and the application scenarios more.

[0049] Although an embodiment of the present invention has been shown and described, this specific embodiment is merely an explanation of the present invention and is not a limitation of the invention. The specific features, structures, materials or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. After reading this specification, those skilled in the art may make modifications, substitutions and variations to the embodiments without creative contributions as needed without departing from the principles and purpose of the present invention. However, as long as they are within the scope of the claims of the present invention, they are protected by patent law.

Claims

1. An inertial navigation device based on multimodal data analysis, comprising an IMU measurement unit (1), an RTC (2), a PMIC module (3), a security module (4), a processor module (5), a storage module (6), an interface module (7), an NPU (8), a visual sensor (9), and a terminal (10), characterized in that: The IMU measurement unit (1) is electrically connected to the PMIC module (3), the IMU measurement unit (1) is signal-connected to the processor module (5), the RTC (2) is electrically connected to the PMIC module (3), the RTC (2) is signal-connected to the processor module (5), the PMIC module (3) is electrically connected to the security module (4), the PMIC module (3) is electrically connected to the processor module (5), the PMIC module (3) is electrically connected to the storage module (6), the PMIC module (3) is electrically connected to the interface module (7), and the PMIC module (3) is electrically connected to the N The PU (8) is electrically connected, the PMIC module (3) is electrically connected to the visual sensor (9), the security module 4 is signal-connected to the processor module 5, the security module 4 is signal-connected to the interface module (7), the processor module (5) is signal-connected to the storage module (6), the processor module (5) is signal-connected to the NPU (8), the processor module (5) is signal-connected to the visual sensor (9), the storage module (6) is signal-connected to the visual sensor (9), the interface module (7) is signal-connected to the terminal (10), and the NPU (8) is signal-connected to the visual sensor (9).

2. The inertial navigation device based on multimodal data analysis according to claim 1, characterized in that: The processor module is responsible for data processing of the IMU measurement unit, parameter correction model calculation and data processing during the inertial navigation process, and overall control of the entire system. IMU measurement unit (1): measures the acceleration and angular velocity of the carrier in real time, and uses corresponding algorithms to calculate the carrier's posture, velocity and position information. Visual sensor (9): Provides external scenes within a fixed range around the carrier and provides real-time visual information for the AI ​​reasoning model. NPU(8): Accelerates the large number of matrix operations in the image AI model, efficiently processes image data, and cooperates with the AI ​​model to infer position, posture, and speed information. Security module (4): security isolation in the execution software to prevent external intrusion into the inertial navigation system through the interface module software, and to protect data information security by means of data isolation. RTC(2): Provides real-time clock for the entire inertial navigation system. Storage module (6): stores the acquired relevant data, execution programs, algorithm programs, etc. PMIC module (3): provides power to the different components of the entire inertial navigation system. Interface module (7): provides various common external interfaces to facilitate access to various mobile terminal devices.

3. An inertial navigation system based on multimodal data analysis, applied to an inertial navigation device based on multimodal data analysis as claimed in any one of claims 1 to 3, characterized in that: The system process is specifically as follows: S1: the inertial navigation system is started, the processor module starts to load the driver, configures the preset parameters, and configures the corresponding navigation mode (two-dimensional plane navigation mode, three-dimensional space navigation mode). Then the accelerometer and gyroscope in the IMU measurement unit and the visual sensor perform self-inspection. At the same time, the accelerometer and gyroscope are initially aligned, and the visual sensor performs automatic visual correction. S2: The accelerometer, gyroscope and visual sensor in the IMU measurement unit start working synchronously, and start measuring the acceleration and angular velocity of the carrier and the environmental image of the preset range around the carrier respectively. S3: According to the measured angular velocity, acceleration, and the preset carrier mode, the corresponding algorithm is configured to calculate the corresponding attitude angle (including: the carrier's pitch angle, roll angle and heading angle) through the gyroscope measurement data. It is often necessary to convert the coordinates to update the carrier's attitude information in the reference coordinate system, and then use the acceleration information measured by the accelerometer and the updated attitude information to calculate the carrier's position information and speed information in the navigation coordinate system to obtain the carrier's rough inertial navigation information (the carrier's updated position, attitude and speed information). After obtaining the carrier's posture information, the corresponding algorithm reasoning is performed through the visual sensor to construct a parameter correction model to obtain the final carrier's inertial navigation information. S4: After the minimum carrier inertial navigation information is updated, the error compensation model of the IMU measurement unit and the vision-based parameter correction model are multimodally fused, and the corresponding AI model is used for accelerated reasoning to compensate and correct the errors of the accelerometer and gyroscope in the IMU measurement unit. The S2, S3 and S4 processes are then continued until the terminal shuts down the inertial navigation system.

4. The inertial navigation system based on multimodal data analysis according to claim 3, characterized in that: In step S3, a visual sensor and an NPU module that cooperates with AI model reasoning are added to the inertial navigation system to construct an inertial navigation parameter correction model, which can compensate and correct the carrier inertial navigation information in real time during the inertial navigation process, thereby improving the accuracy of the inertial navigation information.

5. The inertial navigation system based on multimodal data analysis according to claim 3, characterized in that: In step S4, multimodal fusion is performed through the error compensation model of the traditional IMU measurement unit and the vision-based parameter correction model, and the corresponding AI model is used for accelerated reasoning to achieve fast sensor parameter correction, thereby reducing the cumulative error caused by inertial navigation time, thereby further improving the accuracy of inertial navigation and the effective duration of inertial navigation.

Images