Unmanned ship integrated navigation system based on multi-source data fusion in GNSS denial environment and operation method

Abstract

The invention provides an unmanned ship integrated navigation system based on multi-source data fusion in a GNSS denial environment and an operation method, high-precision positioning and self-adaptive navigation of an unmanned ship in a complex environment are realized by fusing data of inertia, vision and sonar sensors, and the unmanned ship integrated navigation system comprises a multi-source sensor module and a central control unit, the system is configured to execute a multi-source data fusion algorithm based on extended Kalman filtering so as to realize state prediction and observation updating and further perform accurate positioning and navigation on the unmanned surface vehicle in a GNSS denial environment, and the system and the method are specially used for meeting autonomous navigation requirements in the GNSS denial or signal limited environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned boat navigation, and specifically relates to an unmanned boat combined navigation system and an operation method based on multi-source data fusion in a GNSS denied environment, which is used for unmanned boat navigation in an environment where GNSS signals are limited or denied. Background Art

[0002] In unmanned boat navigation, the Global Navigation Satellite System (GNSS) is widely used for positioning. However, in complex environments such as deep oceans, canyons, and extreme weather, GNSS signals are easily limited or interrupted, resulting in positioning failure. Therefore, in order to achieve autonomous navigation, existing technologies gradually adopt a combination of multiple sensors to achieve unmanned boat positioning in a GNSS-free environment through data fusion.

[0003] Commonly used sensor combinations include inertial sensors, visual sensors, and sonar sensors: Inertial sensors can provide continuous position information for a short period of time, but over time, cumulative drift will occur, affecting accuracy.

[0004] Vision sensors capture images for positioning and generally work well in well-lit conditions, but perform poorly in low-light or underwater environments.

[0005] Sonar sensors use sound waves to measure distance and are suitable for low-visibility environments, such as underwater or in fog, but their accuracy may be affected in open or complex terrain.

[0006] In order to fuse the data from these different sensors, algorithms such as the Extended Kalman Filter (EKF) are used to estimate the state of the unmanned boat. Although this fusion method improves the accuracy, it still has the following shortcomings: 1. Weights cannot be adjusted dynamically: Existing systems usually use fixed sensor weights, which cannot be adaptively adjusted according to environmental changes, resulting in insufficient data utilization in some cases and reducing the adaptability of the system.

[0007] 2. Inertial drift is difficult to compensate: Inertial sensors will accumulate drift errors during long-term use. Existing technologies lack effective compensation methods, resulting in a gradual decrease in positioning accuracy.

[0008] 3. Lack of adaptive optimization: Existing systems cannot use historical data to dynamically adjust parameters, which limits performance in complex environments.

[0009] Therefore, the current multi-source data fusion system still has much room for improvement in adaptability, stability and accuracy. Summary of the invention

[0010] The present invention proposes an unmanned boat integrated navigation system and operation method based on multi-source data fusion in a GNSS-denied environment, which is specifically aimed at the autonomous navigation needs in GNSS-denied or signal-limited environments. By integrating the data of inertial, visual, and sonar sensors, the present invention realizes high-precision positioning and adaptive navigation of unmanned boats in complex environments.

[0011] The technical solution of the present invention is as follows: An unmanned boat integrated navigation system based on multi-source data fusion in a GNSS denial environment, comprising: A multi-source sensor module is used to collect motion data, environmental image data and underwater distance data of the unmanned boat, and the multi-source sensor module includes: Inertial sensors, used to collect acceleration and angular velocity data; Visual sensor, used to collect environmental image data to obtain relative displacement information; Sonar sensors, used to collect underwater distance data to detect underwater terrain or obstacles; It also includes: The central control unit is configured to execute a multi-source data fusion algorithm based on an extended Kalman filter to achieve state prediction and observation update, thereby accurately positioning and navigating the unmanned vehicle in a GNSS-denied environment, including the following steps: State prediction: predict the state of the unmanned boat through the motion data of the inertial sensor and the system dynamic model; Observation update: Based on the observation data from the visual sensor and sonar sensor, the Kalman gain is used to dynamically adjust the weight of the observation data, and the observation value is integrated with the predicted state to update the state estimate; Dynamic weight adjustment: During the Kalman filtering process, the weight of each observation data is adjusted according to the signal-to-noise ratio or measurement accuracy of the multi-source sensors to achieve adaptive data fusion for environmental changes.

[0012] Optimally, based on the motion data of the inertial sensor, the state transfer matrix and the control input matrix are used to predict the current position and attitude of the unmanned boat, and the state prediction formula is set as: ; in For the predicted state, is the state transfer matrix, is the control input matrix, is the control input; Optimally, based on the observations of the visual sensor and sonar sensor, the weight of the observation data is dynamically adjusted through the Kalman gain, and the observations are fused with the predicted state to update the state estimate. The observation update formula is set as: ; in, is the Kalman gain, which is dynamically adjusted according to the confidence of each sensor; is the current observation value, which is a fusion of visual and sonar data; is the observation matrix.

[0013] Optimally, dynamic weight adjustment dynamically adjusts the observation value according to the signal-to-noise ratio of each sensor The weight of fused current observations is as follows: = ; in , and Dynamic weights for inertial, visual, and sonar sensors; , , are the Kalman gains of inertial, visual and sonar sensors respectively.

[0014] Optimally, it also includes drift compensation and error correction, using the observation data of vision and sonar sensors to compensate for the inertial navigation drift. The correction formula is: ); in, is the corrected position information; is the position information output by the inertial sensor module; Observation location provided for vision or sonar sensor modules; It is a dynamic adjustment coefficient.

[0015] Optimally, the correction factor for drift compensation Dynamic adjustments are made based on the confidence of visual or sonar observation data to ensure the long-term stability of inertial navigation.

[0016] Optimally, it also includes a feedback loop, through which the fused position information is fed back in real time, the actual position is compared with the preset track to adjust the fusion parameters, and the navigation data of the unmanned boat is output.

[0017] Optimally, the visual sensor of the multi-source sensor module is configured to extract feature points in the environment image through the ORB algorithm, and remove abnormal matching data in combination with the RANSAC algorithm. The ORB algorithm and the RANSAC algorithm for removing abnormal matching data are existing mature technologies and will not be described in detail in this application.

[0018] An operation method of the unmanned boat integrated navigation system based on multi-source data fusion in the above-mentioned GNSS denial environment comprises the following steps: Step 1: System initialization, initial alignment of the inertial sensor, and calibration of the multi-source sensor module to obtain the initial state; Step 2: Data collection and preprocessing: collect data from inertial, visual and sonar sensors, and remove abnormal observations through denoising and feature point matching; Step 3: Multi-source data fusion and state prediction. The state of the unmanned boat is predicted and updated through the extended Kalman filter. The observation data of multi-source sensors and the system dynamic model are used to complete the state prediction and observation update through the extended Kalman filter, including: Status prediction: ; in For the predicted state, is the state transfer matrix, is the control input matrix, is the control input; Observation Update: ; in, is the Kalman gain, which is dynamically adjusted according to the confidence of each sensor; is the current observation value, which is a fusion of visual and sonar data; is the observation matrix; Step 4: Adaptive weight adjustment, dynamically adjust the observation value according to the signal-to-noise ratio of each sensor The weight of fused current observations is as follows: = ; in , and Dynamic weights for inertial, vision, and sonar sensors.

[0019] , , are the Kalman gains of inertial, visual and sonar sensors respectively; Step 5: Drift compensation and error correction. Use the observation data of the visual and sonar sensors to compensate for the inertial navigation drift. The correction formula is: ); in, is the corrected position information; is the position information output by the inertial sensor module; Observation location provided for vision or sonar sensor modules; is the dynamic adjustment coefficient; Step 6: Navigation output and feedback: The fused position information is fed back in real time through the feedback loop, the actual position is compared with the preset track to adjust the fusion parameters, and the navigation data of the unmanned boat is output; The present invention may also include a learning module to adaptively adjust sensor weights based on historical observation data. The learning module analyzes the quality of observation data of the system in different environments to automatically optimize the weight distribution of each sensor in a specific scene, so that the system can maintain high-precision navigation performance in a constantly changing environment.

[0020] The technical effects of the invention are as follows: 1. Construction of multi-source sensor module This system integrates inertial, visual and sonar sensors to collect the motion data of the unmanned boat, environmental images and underwater ranging information respectively. The combination of various sensors enables the system to obtain diverse data sources under different environmental conditions to ensure the reliability of positioning. Inertial sensors provide continuous position information in a short time, visual sensors provide relative displacement of image feature points when the lighting is good, and sonar sensors provide underwater or obstacle distance information in low visibility environments.

[0021] 2. Multi-source data fusion algorithm based on extended Kalman filter The system uses the Extended Kalman Filter (EKF) algorithm to fuse observation data from multiple source sensors to achieve high-precision state estimation of the unmanned boat. The EKF algorithm includes two core steps: state prediction and observation update: During the state prediction process, the system uses data from inertial sensors and the dynamic model of the unmanned boat to make a preliminary estimate of the current position and attitude.

[0022] During the observation update process, the system integrates the observation values ​​of vision and sonar sensors, and through dynamically adjusted Kalman gain, makes the state estimation gradually approach the actual position of the unmanned boat.

[0023] 3. Dynamic weight adjustment mechanism The present invention introduces a dynamic weight adjustment mechanism to calculate the weight of each sensor in real time according to its signal-to-noise ratio. For the fusion of observation data, the system dynamically adjusts the weights of inertial, visual, and sonar sensors according to environmental changes to improve the accuracy and adaptability of data fusion. In a well-lit environment, the weight of visual sensor data increases; underwater or in complex terrain, the system prioritizes sonar data.

[0024] 4. Drift compensation and error correction During long-term operation, the system compensates for the drift error of the inertial sensor through the observation data of the visual or sonar sensor. Using the observation update step in the extended Kalman filter, the system can dynamically adjust the coefficient of drift correction to ensure the long-term accuracy and stability of inertial navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a step diagram of the operating method of the present invention. DETAILED DESCRIPTION

[0026] Embodiment 1: This embodiment relates to an unmanned boat integrated navigation system based on multi-source data fusion in a GNSS denied environment, which is used in an environment where GNSS is denied or the signal is limited. The system structure is as follows: Figure 1 As shown, including: A multi-source sensor module is used to collect motion data, environmental image data and underwater distance data of the unmanned boat, and the multi-source sensor module includes: Inertial sensors, used to collect acceleration and angular velocity data; Visual sensor, used to collect environmental image data to obtain relative displacement information; Sonar sensors, used to collect underwater distance data to detect underwater terrain or obstacles; It also includes: The central control unit is configured to execute a multi-source data fusion algorithm based on an extended Kalman filter to achieve state prediction and observation update, thereby accurately positioning and navigating the unmanned vehicle in a GNSS-denied environment, including the following steps: State prediction: predict the state of the unmanned boat through the motion data of the inertial sensor and the system dynamic model; Observation update: Based on the observation data from the visual sensor and sonar sensor, the Kalman gain is used to dynamically adjust the weight of the observation data, and the observation value is integrated with the predicted state to update the state estimate; Dynamic weight adjustment: During the Kalman filtering process, the weight of each observation data is adjusted according to the signal-to-noise ratio or measurement accuracy of the multi-source sensors to achieve adaptive data fusion for environmental changes.

[0027] Embodiment 2: The difference between this embodiment and embodiment 1 is that this embodiment predicts the current position and attitude of the unmanned boat based on the motion data of the inertial sensor using the state transfer matrix and the control input matrix, and the state prediction formula is set as: ; in For the predicted state, is the state transfer matrix, is the control input matrix, For control input.

[0028] Furthermore, based on the observations of the visual sensor and sonar sensor, the weight of the observation data is dynamically adjusted through the Kalman gain, and the observations are fused with the predicted state to update the state estimate. The observation update formula is set as: ; in, is the Kalman gain, which is dynamically adjusted according to the confidence of each sensor; is the current observation value, which is a fusion of visual and sonar data; is the observation matrix.

[0029] Furthermore, dynamic weight adjustment dynamically adjusts the observation value according to the signal-to-noise ratio of each sensor The weight of fused current observations is as follows: = ; in , and Dynamic weights for inertial, visual, and sonar sensors; , , are the Kalman gains of inertial, visual and sonar sensors respectively.

[0030] Optimally, it also includes drift compensation and error correction, using the observation data of vision and sonar sensors to compensate for the inertial navigation drift. The correction formula is: ); in, is the corrected position information; is the position information output by the inertial sensor module; Observation location provided for vision or sonar sensor modules; It is a dynamic adjustment coefficient.

[0031] Furthermore, the correction factor of drift compensation is Dynamic adjustments are made based on the confidence of visual or sonar observation data to ensure the long-term stability of inertial navigation.

[0032] Furthermore, it also includes a feedback loop, through which the fused position information is fed back in real time, the actual position is compared with the preset track to adjust the fusion parameters, and the navigation data of the unmanned boat is output.

[0033] Embodiment 3: An operation method of the unmanned boat integrated navigation system based on multi-source data fusion in the above-mentioned GNSS denial environment comprises the following steps: Step 1: System initialization, initial alignment of the inertial sensor, and calibration of the multi-source sensor module to obtain the initial state; Step 2: Data collection and preprocessing: collect data from inertial, visual and sonar sensors, and remove abnormal observations through denoising and feature point matching; Step 3: Multi-source data fusion and state prediction. The state of the unmanned boat is predicted and updated through the extended Kalman filter. The observation data of multi-source sensors and the system dynamic model are used to complete the state prediction and observation update through the extended Kalman filter, including: Status prediction: ; in For the predicted state, is the state transfer matrix, is the control input matrix, is the control input; Observation Update: ; in, is the Kalman gain, which is dynamically adjusted according to the confidence of each sensor; is the current observation value, which is a fusion of visual and sonar data; is the observation matrix; Step 4: Adaptive weight adjustment, dynamically adjust the observation value according to the signal-to-noise ratio of each sensor The weight of fused current observations is as follows: = ; in , and Dynamic weights for inertial, vision, and sonar sensors.

[0034] , , Kalman gains for inertial, visual, and sonar sensors respectively Step 5: Drift compensation and error correction. Use the observation data of the visual and sonar sensors to compensate for the inertial navigation drift. The correction formula is: ); in, is the corrected position information; is the position information output by the inertial sensor module; Observation location provided for vision or sonar sensor modules; It is a dynamic adjustment coefficient.

[0035] Step 6: Navigation output and feedback: Provide real-time feedback on the fused position information through the feedback loop, compare the actual position with the preset track to adjust the fusion parameters, and output the navigation data of the unmanned boat.

Claims

1. An unmanned boat integrated navigation system based on multi-source data fusion in a GNSS denial environment, comprising: A multi-source sensor module is used to collect motion data, environmental image data and underwater distance data of the unmanned boat, and the multi-source sensor module includes: Inertial sensors, used to collect acceleration and angular velocity data; Visual sensor, used to collect environmental image data to obtain relative displacement information; Sonar sensors, used to collect underwater distance data to detect underwater terrain or obstacles; It also includes: The central control unit is configured to execute a multi-source data fusion algorithm based on an extended Kalman filter to achieve state prediction and observation update, thereby accurately positioning and navigating the unmanned vehicle in a GNSS-denied environment, including the following steps: State prediction: predict the state of the unmanned boat through the motion data of the inertial sensor and the system dynamic model; Observation update: Based on the observation data from the visual sensor and sonar sensor, the Kalman gain is used to dynamically adjust the weight of the observation data, and the observation value is integrated with the predicted state to update the state estimate; Dynamic weight adjustment: During the Kalman filtering process, the weight of each observation data is adjusted according to the signal-to-noise ratio or measurement accuracy of the multi-source sensors to achieve adaptive data fusion for environmental changes.

2. The unmanned boat integrated navigation system based on multi-source data fusion in a GNSS-denied environment according to claim 1 is characterized in that: Based on the motion data of the inertial sensor, the state transfer matrix and the control input matrix are used to predict the current position and attitude of the unmanned boat. The state prediction formula is set as: ； in For the predicted state, is the state transfer matrix, is the control input matrix, For control input.

3. The unmanned boat integrated navigation system based on multi-source data fusion in a GNSS-denied environment according to claim 1, characterized in that: Based on the observations of the visual sensor and sonar sensor, the weight of the observation data is dynamically adjusted through the Kalman gain, and the observations are fused with the predicted state to update the state estimate. The observation update formula is set as: ； in, is the Kalman gain, which is dynamically adjusted according to the confidence of each sensor; is the current observation value, which is a fusion of visual and sonar data; is the observation matrix.

4. The unmanned boat integrated navigation system based on multi-source data fusion in a GNSS-denied environment according to claim 1, characterized in that: Dynamic weight adjustment dynamically adjusts the observation value according to the signal-to-noise ratio of each sensor The weight of fused current observations is as follows: = ； in , and Dynamic weights for inertial, visual, and sonar sensors; , , are the Kalman gains of inertial, visual and sonar sensors respectively.

5. The unmanned boat integrated navigation system based on multi-source data fusion in a GNSS-denied environment according to claim 1, characterized in that: It also includes drift compensation and error correction, using the observation data of vision and sonar sensors to compensate for inertial navigation drift. The correction formula is: ) in, is the corrected position information; is the position information output by the inertial sensor module; Observation location provided for vision or sonar sensor modules; It is a dynamic adjustment coefficient.

6. The unmanned boat integrated navigation system based on multi-source data fusion in a GNSS-denied environment according to claim 5, characterized in that: The correction factor of drift compensation is Dynamic adjustments are made based on the confidence of visual or sonar observation data to ensure the long-term stability of inertial navigation.

7. The unmanned boat integrated navigation system based on multi-source data fusion in a GNSS-denied environment according to claim 1, characterized in that: It also includes a feedback loop, which provides real-time feedback on the fused position information, compares the actual position with the preset track, adjusts the fusion parameters, and outputs the navigation data of the unmanned boat.

8. The unmanned boat integrated navigation system based on multi-source data fusion in a GNSS-denied environment according to claim 1, characterized in that: The visual sensor of the multi-source sensor module is configured to extract feature points in the environment image through the ORB algorithm and remove abnormal matching data in combination with the RANSAC algorithm.

9. An operating method of an unmanned boat integrated navigation system based on multi-source data fusion in a GNSS denied environment as claimed in any one of claims 1 to 8, characterized in that: It includes the following steps: Step 1: System initialization, initial alignment of the inertial sensor, and calibration of the multi-source sensor module to obtain the initial state; Step 2: Data collection and preprocessing: collect data from inertial, visual and sonar sensors, and remove abnormal observations through denoising and feature point matching; Step 3: Multi-source data fusion and state prediction. The state of the unmanned boat is predicted and updated through the extended Kalman filter. The observation data of multi-source sensors and the system dynamic model are used to complete the state prediction and observation update through the extended Kalman filter, including: Status prediction: ； in For the predicted state, is the state transfer matrix, is the control input matrix, is the control input; Observation Update: ； in, is the Kalman gain, which is dynamically adjusted according to the confidence of each sensor; is the current observation value, which is a fusion of visual and sonar data; is the observation matrix; Step 4: Adaptive weight adjustment, dynamically adjust the observation value according to the signal-to-noise ratio of each sensor The weight of fused current observations is as follows: = ； in , and Dynamic weights for inertial, visual, and sonar sensors; , , Kalman gains for inertial, visual, and sonar sensors respectively Step 5: Drift compensation and error correction. Use the observation data of the visual and sonar sensors to compensate for the inertial navigation drift. The correction formula is: ； in, is the corrected position information; is the position information output by the inertial sensor module; Observation location provided for vision or sonar sensor modules; is the dynamic adjustment coefficient; Step 6: Navigation output and feedback: Provide real-time feedback on the fused position information through the feedback loop, compare the actual position with the preset track to adjust the fusion parameters, and output the navigation data of the unmanned boat.

Images