An inertial vision-based integrated navigation method and device based on multi-domain transformation

Abstract

This application provides an inertial vision integrated navigation method and apparatus based on multi-domain transformation. The method includes: S1, acquiring database information of the inertial vision integrated navigation system; S2, initializing the system error state in the navigation domain, and updating the system state in the navigation domain according to the time update principle of the system in the navigation domain; S3, converting the navigation domain state and uncertainty matrix into the runway domain state and uncertainty matrix; S4, if the camera captures a visual image at the current moment, obtaining visual measurements in the measurement domain from the visual image at the current moment through an intelligent detection algorithm; updating the system state and its uncertainty matrix in the runway domain according to the visual measurements; if the camera does not capture a visual image at the current moment, returning to step S2; S5, converting the runway domain state and its uncertainty matrix into the navigation domain state and its uncertainty matrix, thereby realizing inertial vision integrated navigation.

Description

Technical Field

[0001] This invention relates to the field of aviation technology, and in particular to an inertial vision integrated navigation method and apparatus based on multi-domain transformation. Background Technology

[0002] During the landing of large aircraft, pilots require navigation systems to provide information such as the aircraft's current position and attitude to ensure a safe landing on the target runway. This navigation technology incorporates various devices and techniques. In modern aviation, commonly used navigation devices include the Global Positioning System (GPS), Inertial Navigation System (INS), Microwave Landing System (MLS), Instrument Landing System (ILS), and GBAS. Currently, the mainstream landing navigation system is ILS, which is the most widely used precision approach and landing guidance system, relying on radio signals to guide the aircraft's approach.

[0003] ILS systems are divided into three categories, with only the most expensive Category III systems capable of achieving the ability to land without a pilot. However, existing ILS equipment suffers from dependence on ground equipment and is prohibitively expensive.

[0004] For airports lacking high-precision ground navigation facilities, establishing a navigation system that is reasonably priced and accurate enough for autonomous landing is an urgent need worldwide. Summary of the Invention

[0005] This invention provides an inertial vision-based integrated navigation method and apparatus based on multi-domain transformation, which can solve the problems of existing navigation equipment relying on ground equipment and being expensive.

[0006] The technical solution of this invention:

[0007] Firstly, this application provides an inertial vision integrated navigation method based on multi-domain transformation, the method comprising:

[0008] S1. Obtain the database information of the inertial vision integrated navigation system;

[0009] S2, Initialize the system error state in the navigation domain, and update the system state in the navigation domain according to the time update principle of the system in the navigation domain;

[0010] S3 transforms the navigation domain state and uncertainty matrix into the runway domain state and uncertainty matrix;

[0011] S4, if the camera captures a visual image at the current moment, the visual measurement in the measurement domain is obtained from the visual image at the current moment through an intelligent detection algorithm; based on the visual measurement, the runway domain state is updated. and its uncertainty matrix If the camera does not capture a visual image at the current moment, return to step S2.

[0012] S5, runway domain status and its uncertainty matrix Transform into navigation domain state and its uncertainty matrix This enables inertial-visual integrated navigation;

[0013] Specifically, S2 includes:

[0014] S21. Determine the navigation domain of the inertial vision integrated navigation system based on the WGS 84 coordinate system, local navigation system, machine system, and camera system.

[0015] S22. Determine the navigation domain state. :

[0016]

[0017] In the above formula, This represents the position error projection in the navigation system. This represents the velocity error in the local navigation system. This represents the projection of the misalignment angle error in the navigation system.

[0018] S23. Initialize navigation domain state ;

[0019] S24. Based on the time update principle of the system under the navigation domain, the navigation domain state at time k is... Update the time.

[0020] Specifically, S3 includes:

[0021] S31. Determine the runway domain based on the target runway coordinate system;

[0022] S32. Determine the runway domain status based on user concerns and visual measurement characteristics. :

[0023]

[0024] In the above formula, This represents the position error projection in the runway system. This represents the projection of the misalignment angle error in the runway frame.

[0025] S33, System position error in the navigation domain Converted to system position error in the runway domain ;

[0026]

[0027] in, Let be the rotation matrix from the navigation frame to the runway frame. This represents the projected position error under the runway system. This represents the position error projection in the navigation system;

[0028] S34. Project the misalignment angle error under the navigation system. This is converted into the projection of the misalignment angle error in the runway frame. ;

[0029] S35. The error uncertainty matrix in the navigation domain Converted into the error uncertainty matrix in the runway domain ;

[0030] Specifically, S34 includes:

[0031] Using formula Projecting the misalignment angle error under the navigation system This is converted into the projection of the misalignment angle error in the runway frame. ;

[0032] in, Projection of misalignment angle error in navigation system The corresponding rotation matrix, Projection of misalignment angle error in the runway frame The corresponding rotation matrix, The rotation matrix representing the navigation frame to the runway frame;

[0033] Specifically, S35 includes:

[0034] S351, Solve eigenvalues and eigenvectors ;

[0035] S352. Use formula: , Solve eigenvalues and eigenvectors ;

[0036] S353. Use formula Solve the error uncertainty matrix in the runway domain. .

[0037] Specifically, S4 includes:

[0038] S41 determines the measurement domain based on sensor data from the inertial vision integrated navigation system;

[0039] S42. Obtain the full position and velocity information of the aircraft from the external inertial navigation high-frequency system, whereby the inertial navigation system calculates the position as follows: ;

[0040] S43. Acquire image information and corresponding timestamps from the vision sensor at a lower frequency, and use an intelligent detection algorithm to obtain the visual measurement results. ;in, The pixel slope representing the two feature lines. The pixel coordinates representing a feature point;

[0041] S44. Determine the visual measurement results respectively. Predicted value , , , ;

[0042] S45. Based on the predicted value , , , and visual measurement results ,use Determine the total measurement of the filter. ;

[0043] S46. Based on the database information of the inertial vision integrated navigation system and the full attitude and velocity information of the aircraft under the navigation system, solve for the filter observation matrix H;

[0044] S47 performs a status update and obtains the updated runway domain status. and its runway domain uncertainty matrix .

[0045] Specifically, S46 includes:

[0046] S461, Based on the rotation matrix from the navigation frame to the camera frame. Determine parameters

[0047] ;

[0048] ;

[0049] S462, according to Using the formula Solve for parameters , ;

[0050] S463, According to parameters , Using formula

[0051] Determine parameters ;

[0052] S464. Based on the database information of the inertial vision integrated navigation system, the aircraft's full pose and velocity information under the navigation system, and the actual runway domain position of the observed feature points. Determine the camera system position of the feature points to be observed. :

[0053]

[0054] in, Let be the rotation matrix from the runway system to the camera system;

[0055] S465. Calculate the position based on the inertial navigation system. Rotation matrix from navigation system to camera system ,state With camera and airplane boom Using the formula

[0056] ,

[0057] Determine the runway coordinates of the current camera position. ;

[0058] S466, According to parameters runway width Determine parameters ;

[0059]

[0060] ;

[0061] ;

[0062] ;

[0063] S467. Determine the parameters based on the camera's intrinsic parameter K. , ;

[0064] S468, According to parameters , , The current camera position in the runway coordinate system.

[0065] ,parameter Determine parameters , , , ;

[0066] ;

[0067] ;

[0068] ;

[0069] ;

[0070] S469, According to parameters , , , , , , Camera system location of the feature points to be observed The actual runway domain location of the feature point to be observed Rotation matrix from runway system to camera system Solve for matrix H;

[0071] ;

[0072] .

[0073] Specifically, S44 includes:

[0074] The prediction of point features can be expressed as:

[0075]

[0076] The prediction of line features can be expressed as:

[0077]

[0078] In the above formula, Here are the coordinates of the feature point in the camera frame. Here are the coordinates of the feature point in the runway system. This represents the rotation and translation relationship of projecting point coordinates from the runway frame to the camera frame, where K is an intrinsic parameter. For normalization parameters, The Plück matrix represents a three-dimensional straight line. The Plück matrix represents a two-dimensional straight line.

[0079] Secondly, this application provides an inertial vision integrated navigation device based on multi-domain transformation, the device being used to implement the above-mentioned inertial vision integrated navigation method based on multi-domain transformation.

[0080] The beneficial effects of this invention are as follows: This invention addresses the issue that traditional runway autonomous landing navigation (ILS) technology relies on ground equipment. It provides an inertial vision-based integrated navigation method and device based on multi-domain conversion. It utilizes visually acquired feature information and inertial navigation information to achieve navigation performance under high-precision requirements such as large aircraft runway landing. It is a navigation means that complements ILS / GNSS and can ensure high-precision and reliable navigation and positioning capabilities even when relying solely on airborne equipment. Attached Figure Description

[0081] Figure 1 This is a data flow diagram for traditional inertial and visual integrated navigation methods.

[0082] Figure 2 This is a data flow diagram for an inertial vision-based integrated navigation method based on multi-domain transformation.

[0083] Figure 3 This is a schematic diagram of the pixel system.

[0084] Figure 4 This is a schematic diagram of the runway system;

[0085] Figure 5 A schematic diagram of the WGS84 series;

[0086] Figure 6 Schematic diagram of the machine system;

[0087] Figure 7 This is a schematic diagram of the overall filtering algorithm. Detailed Implementation

[0088] Visual navigation technology, as an emerging navigation technology, combined with inertial navigation technology, can achieve high-precision autonomous navigation capabilities using information such as images and internal states. Thanks to the development of artificial intelligence visual perception technology, in situations where satellite signals are limited or completely denied, and ground equipment cannot provide information, inertial-visual integrated navigation technology can rely on ground features and existing databases to provide navigation information such as the current position of the vehicle.

[0089] To address the above challenges, this invention proposes an inertial vision integrated navigation method based on multi-domain transformation. This system aims to solve the problems of low accuracy, susceptibility to electromagnetic interference, and high cost of traditional runway autonomous landing navigation (ILS) technology when lacking ground-aided equipment, and ensures high-precision and reliable navigation and positioning capabilities even in complex environments.

[0090] In this scenario, the traditional navigation filter maintains the system's navigation state in the navigation domain to correct inertial navigation information; only the navigation domain data is continuous. Runway domain information is only available when measurements are taken. See details... Figure 1Generally, users are more concerned with the aircraft's relative position to the runway, and information in the navigation domain is less intuitive than information in the runway domain. The algorithm proposed in this paper maintains the state in both the navigation and runway domains, with continuous data in both coordinate systems. Compared to traditional methods, it has significant advantages: it meets user concerns, conforms to visual observation characteristics, provides the possibility for subsequent integrity calculations, and can correct inertial navigation information.

[0091] Referring to Figure 2, the technical solution is described in detail. After determining the database, the required point and line features are detected in the image data using artificial intelligence algorithms. At this point, the point and line features are represented using pixel coordinates in the measurement domain. A compact combination method is used to fuse inertial navigation and visual information to obtain runway domain navigation data. Then, the navigation domain navigation information is obtained by utilizing the inter-domain transformation relationship. When there is no observation, the navigation domain data is only updated over time before being transformed to the runway domain.

[0092] Example 1

[0093] This application provides an inertial vision integrated navigation method based on multi-domain transformation, comprising the following steps:

[0094] S1. Obtain database information from the inertial vision integrated navigation system, including various parameters of the inertial navigation system, various parameters of the camera, calibration parameters of the camera and the aircraft, calibration parameters of the inertial navigation system and the aircraft, latitude and longitude of the target runway entrance, runway orientation, runway length and width, flight trajectory, and other information.

[0095] Specifically, S1 includes the following steps:

[0096] Inertial navigation parameter acquisition. Based on the selected inertial navigation signal, refer to the instruction manual to obtain the zero-bias, random walk, and zero-position first-order Markov parameters of the inertial gyroscope, as well as the zero-bias, random walk, and zero-position first-order Markov parameters of the inertial accelerator. Also determine the inertial navigation update frequency. .

[0097] Camera parameter acquisition. This includes camera intrinsic parameters K, camera distortion parameters C, and camera sampling frequency. .

[0098] Determine the calibration parameters for the camera and the body, including the translation vector from the camera system to the origin of the body system. Rotation matrix of camera system to machine system Determine the calibration parameters for the camera and inertial navigation system, including the translation vector from the camera frame to the origin of the inertial frame. Rotation matrix from camera frame to inertial frame .

[0099] Determine the target runway information, including the latitude, longitude, and altitude of the runway entrance. , , runway orientation angle runway length and width A landing trajectory is determined, which ultimately lands 400m from the runway threshold. Runway coordinates are then determined to define the characteristics of the observation point. .

[0100] S2, initialize the system error state in the navigation domain, and update the system state in the navigation domain according to the time update principle of the system in the navigation domain.

[0101] Specifically, S2 includes:

[0102] S21. Determine the navigation domain of the inertial vision integrated navigation system based on the WGS 84 coordinate system, local navigation system, machine system, and camera system.

[0103] It should be noted that the navigation domain is related to the WGS-84 coordinate system, the local navigation system, the aircraft system, and the camera system. WGS-84 is currently the main coordinate system used by satellite navigation systems, and will not be elaborated on here; see Figure 5 for details. The local navigation system is set as the n-system, with its three axes coinciding with the northeast-central coordinate system, and its origin coinciding with the runway system. The aircraft system is set as the b-system, defined as the lower right of the aircraft, with its origin at the aircraft's center of mass; see Figure 6 for details. The camera system is set as the c-system, defined as the lower right of the aircraft, with its origin at the camera's optical center.

[0104] S22. Determine the navigation domain state. :

[0105]

[0106] In the above formula, This represents the position error projection in the navigation system. This represents the velocity error in the local navigation system. This represents the projection of the misalignment angle error in the navigation system.

[0107] S23. Initialize navigation domain state .

[0108] Based on the inertial navigation parameters in the database, initialize a reasonable navigation domain state.

[0109] S24. Based on the time update principle of the system under the navigation domain, the navigation domain state at time k is... Update the time.

[0110] In this invention, time updates always occur in the navigation domain system state, and measurement updates always occur in the runway domain state. Time updates reference the update matrix of a high-precision inertial navigation system. This will not be elaborated upon here. Based on this, the time update equation for the navigation domain state is obtained:

[0111]

[0112] In the above formula, represent The predicted value of the navigation domain state. It is white noise.

[0113] S3 converts the navigation domain state and uncertainty matrix into the runway domain state and uncertainty matrix.

[0114] Specifically, S3 includes:

[0115] S31. Determine the runway domain based on the target runway coordinate system.

[0116] The runway domain is based on the target runway coordinate system. The characteristic results obtained from measurements are transformed into the runway domain to determine the position of the target vehicle relative to the runway system. Generally, the runway is considered a horizontal planar rectangle. When acquiring prior data for the runway, it is necessary to obtain the latitude, longitude, and altitude coordinates of the runway threshold (or other deterministic location), the runway orientation, the runway length and width, and the runway horizontal inclination angle, etc., to determine the position and angle of the runway system in the geocentric coordinate system. The runway system is denoted as the r-system, defined as follows: the center of the runway threshold in the current flight direction is the origin of the runway system; the z-direction is perpendicular to the runway plane and points towards the Earth's center; the x-direction is along the runway and points towards the runway tail; and the y-direction is perpendicular to the runway direction, following the right-hand rule. See details... Figure 4 .

[0117] S32. Determine the runway domain status based on user concerns and visual measurement characteristics. :

[0118]

[0119] In the above formula, This represents the positional error of the system projection under the runway frame. This represents the angular error of the system projection under the runway system.

[0120] S33, System position error in the navigation domain Converted to system position error in the runway domain ;

[0121]

[0122] in, Let be the rotation matrix from the navigation frame to the runway frame. This represents the projected position error under the runway system. This represents the position error projection in the navigation system;

[0123] S34. Project the misalignment angle error under the navigation system. This is converted into the projection of the misalignment angle error in the runway frame. ;

[0124] Specifically, S34 includes:

[0125] Using formula Projecting the misalignment angle error under the navigation system This is converted into the projection of the misalignment angle error in the runway frame. ;

[0126] in, Projection of misalignment angle error in navigation system The corresponding rotation matrix, Projection of misalignment angle error in the runway frame The corresponding rotation matrix, The rotation matrix representing the navigation frame to the runway frame;

[0127] It needs to be explained that the rotation vector The corresponding rotation matrix can be represented as This is because the rotation vector can be viewed as a Lie algebra, and the exponential mapping of the antisymmetric matrix corresponding to the Lie algebra is the rotation matrix.

[0128] S35. The error uncertainty matrix in the navigation domain Converted into the error uncertainty matrix in the runway domain .

[0129] Specifically, S35 includes:

[0130] S351, Solve eigenvalues and eigenvectors ;

[0131] S352. Use formula: , Solve eigenvalues and eigenvectors ;

[0132] S353. Use formula Solve the error uncertainty matrix in the runway domain. .

[0133] S4, if the camera captures a visual image at the current moment, the visual measurement in the measurement domain is obtained from the visual image at the current moment through an intelligent detection algorithm; based on the visual measurement, the system state in the runway domain is updated. and its uncertainty matrix If the camera does not capture a visual image at the current moment, return to step S2.

[0134] Intelligent detection algorithms include methods such as neural networks.

[0135] Specifically, S4 includes:

[0136] S41 determines the measurement domain based on sensor data from the inertial vision integrated navigation system.

[0137] Specifically, the measurement domain refers to the location of the sensor's measurement results, involving the acquisition and processing of sensor data. This includes devices such as visual sensors, radio altimeters, GNSS, and inertial navigation systems that capture environmental information. The measurement results from radio altimeters, GNSS, and inertial navigation systems provide information such as position and velocity, while visual sensors provide pixel-level feature information; the types of information differ significantly. Since the processing of measurement results from radio altimeters, GNSS, and inertial navigation systems is relatively mature, this discussion primarily focuses on the measurement domain of visual sensors. The camera's measurement results are dimensionless pixel-level features, based on a pixel coordinate system. This pixel coordinate system is defined as the s-coordinate system, with the origin at the top left corner of the image. The positive x-axis runs horizontally to the right, and the positive y-axis runs vertically downwards. Since the image is a 2D plane, it lacks a z-axis, only having x and y axes. During imaging, the camera reduces the 3D plane to a 2D plane, losing depth information along the camera's optical axis. See details... Figure 3 .

[0138] S42. Obtain the full position and velocity information of the aircraft from the external inertial navigation high-frequency system. The inertial navigation system calculates the position as follows: .

[0139] S43. Acquire image information and corresponding timestamps from the vision sensor at a lower frequency, and use an intelligent detection algorithm to obtain the visual measurement results. .

[0140] in, The pixel slope representing the two feature lines. The pixel coordinates represent a feature point.

[0141] S44. Determine the visual measurement results respectively. Predicted value , , , .

[0142] The prediction of point features can be expressed as:

[0143]

[0144] The prediction of line features can be expressed as:

[0145]

[0146] In the above formula, Here are the coordinates of the feature point in the camera frame. Here are the coordinates of the feature point in the runway system. This represents the rotation and translation relationship of projecting point coordinates from the runway frame to the camera frame, where K is an intrinsic parameter. For normalization parameters, The Plück matrix represents a three-dimensional straight line. The Plück matrix represents a two-dimensional straight line.

[0147] S45. Based on the predicted value , , , and visual measurement results ,use Determine the total measurement of the filter. .

[0148] S46. Based on the database information of the inertial vision integrated navigation system and the full attitude and velocity information of the aircraft under the navigation system, solve for the filter observation matrix H.

[0149] Specifically, S46 includes:

[0150] S461, Based on the rotation matrix from the navigation frame to the camera frame. Determine parameters

[0151] ;

[0152] ;

[0153] S462, according to Using the formula Solve for parameters , ;

[0154] It should be noted that, for A normalized direction vector with a magnitude of 1. for The length of the module.

[0155] S463, According to parameters , Using formula

[0156] Determine parameters ;

[0157] S464. Based on the database information of the inertial vision integrated navigation system, the aircraft's full pose and velocity information under the navigation system, and the actual runway domain position of the observed feature points. Determine the camera system position of the feature points to be observed. :

[0158]

[0159] in, Let be the rotation matrix from the runway system to the camera system;

[0160] S465. Calculate the position based on the inertial navigation system. Rotation matrix from navigation system to camera system ,state With camera and airplane boom Using the formula

[0161] ,

[0162] Determine the runway coordinates of the current camera position. ;

[0163] S466, According to parameters runway width Determine parameters ;

[0164]

[0165] ;

[0166] ;

[0167] ;

[0168] S467. Determine the parameters based on the camera's intrinsic parameter K. , ;

[0169] S468, According to parameters , , The current camera position in the runway coordinate system.

[0170] ,parameter Determine parameters , , , ;

[0171] ;

[0172] ;

[0173] ;

[0174] ;

[0175] S469, According to parameters , , , , , , Camera system location of the feature points to be observed The actual runway domain location of the feature point to be observed Rotation matrix from runway system to camera system Solve for matrix H;

[0176] ;

[0177] .

[0178] S47 performs a status update and obtains the updated runway domain status. and its runway domain uncertainty matrix .

[0179]

[0180]

[0181]

[0182] In the above formula, This is the visual observation error, given by the detection algorithm. This is the covariance matrix before the runway domain state update. To measure the updated runway domain status, This is the uncertainty matrix of the runway domain state before the measurement update. This is the uncertainty matrix of the updated runway domain state.

[0183] S5, runway domain status and its uncertainty matrix Transform into navigation domain state and its uncertainty matrix This enables inertial-visual integrated navigation.

[0184] The overall filtering algorithm flow is shown below, connecting all the above results. Figure 7In the diagram, red lines represent branches that are running continuously at any given time, while green lines represent data branches that only run when observations are made. At any given time, time updates occur in the navigation domain state and are projected onto the runway domain state using the transformation relationship described in S3. When observations are made, measurement updates occur in the runway domain state, and the updated state and uncertainty matrix are projected onto the navigation domain state using the inter-domain transformation relationship described in S5. This process repeats continuously.

[0185] Example 2

[0186] This application provides an inertial vision integrated navigation method based on multi-domain transformation, comprising the following steps:

[0187] S1. Obtain database information from the inertial vision integrated navigation system, including various parameters of the inertial navigation system, various parameters of the camera, calibration parameters of the camera and the aircraft, calibration parameters of the inertial navigation system and the aircraft, latitude and longitude of the target runway entrance, runway orientation, runway length and width, flight trajectory, and other information.

[0188] Specifically, S1 includes the following steps:

[0189] Inertial navigation parameter acquisition. Based on the selected inertial navigation signal, refer to the instruction manual to obtain the zero-bias, random walk, and zero-position first-order Markov parameters of the inertial gyroscope, as well as the zero-bias, random walk, and zero-position first-order Markov parameters of the inertial accelerator. Also determine the inertial navigation update frequency. .

[0190] Camera parameter acquisition. This includes camera intrinsic parameters K, camera distortion parameters C, and camera sampling frequency. .

[0191] Determine the calibration parameters for the camera and the body, including the translation vector from the camera system to the origin of the body system. Rotation matrix of camera system to machine system Determine the calibration parameters for the camera and inertial navigation system, including the translation vector from the camera frame to the origin of the inertial frame. Rotation matrix from camera frame to inertial frame .

[0192] Determine the target runway information, including the latitude, longitude, and altitude of the runway entrance. , , runway orientation angle runway length and width A landing trajectory is determined, which ultimately lands 400m from the runway threshold. Runway coordinates are then determined to define the characteristics of the observation point. .

[0193] S2, initialize the system error state in the navigation domain, and update the system state in the navigation domain according to the time update principle of the system in the navigation domain.

[0194] Specifically, S2 includes:

[0195] S21. Determine the navigation domain of the inertial vision integrated navigation system based on the WGS 84 coordinate system, local navigation system, machine system, and camera system.

[0196] S22. Determine the navigation domain state. :

[0197]

[0198] In the above formula, This represents the position error in the local navigation system. This represents the velocity error in the local navigation system. This represents the misalignment angle error in the local navigation system.

[0199] From the definition of the misalignment angle, we know that . This represents the zero-bias error of the gyroscope in the b-series. This represents the zero bias error of the accelerometer in the b-series.

[0200] S23. Initialize navigation domain state. Confirm.

[0201]

[0202] Based on the inertial navigation parameters in the database, initialize a reasonable navigation domain state.

[0203] S24. Based on the time update principle of the system under the navigation domain, the navigation domain state at time k is... Update the time.

[0204] In this invention, time updates always occur in the navigation domain system state, and measurement updates always occur in the runway domain state. Time updates reference the update matrix of a high-precision inertial navigation system. This will not be elaborated upon here. Based on this, the time update equation for the navigation domain state is obtained:

[0205]

[0206] In the above formula, represent The predicted value of the navigation domain state. It is white noise.

[0207] S3 converts the navigation domain state and uncertainty matrix into the runway domain state and uncertainty matrix.

[0208] Specifically, S3 includes:

[0209] S31. Determine the runway domain based on the target runway coordinate system.

[0210] The runway domain is based on the target runway coordinate system. The characteristic results obtained from measurements are transformed into the runway domain to determine the position of the target vehicle relative to the runway system. Generally, the runway is considered a horizontal planar rectangle. When acquiring prior data for the runway, it is necessary to obtain the latitude, longitude, and altitude coordinates of the runway threshold (or other deterministic location), the runway orientation, the runway length and width, and the runway horizontal inclination angle, etc., to determine the position and angle of the runway system in the geocentric coordinate system. The runway system is denoted as the r-system, defined as follows: the center of the runway threshold in the current flight direction is the origin of the runway system; the z-direction is perpendicular to the runway plane and points towards the Earth's center; the x-direction is along the runway and points towards the runway tail; and the y-direction is perpendicular to the runway direction, following the right-hand rule. See details... Figure 4 .

[0211] S32. Determine the runway domain status based on user concerns and visual measurement characteristics. :

[0212]

[0213] In the above formula, This represents the positional error of the system projection under the runway frame. This represents the angular error of the system projection under the runway system.

[0214] S33, System position error in the navigation domain Converted to system position error in the runway domain Specifically, when the rotation matrix from the navigation system to the runway system... hour:

[0215]

[0216] in, This represents the projected position error of the system in the r-frame. This represents the position error projection of the system in the n-system.

[0217] S34. System angle error in the navigation domain Converted to system angle error in the runway domain .

[0218] Specifically, S34 includes:

[0219] Using formula The system angle error in the navigation domain Converted to system angle error in the runway domain Specifically, when the rotation matrix from the navigation system to the runway system... hour, .

[0220] S35. The error uncertainty matrix in the navigation domain Converted into the error uncertainty matrix in the runway domain Specifically, when the rotation matrix from the navigation system to the runway system...

[0221] hour, .

[0222] S4, if the camera captures a visual image at the current moment, the visual measurement in the measurement domain is obtained from the visual image at the current moment through an intelligent detection algorithm; based on the visual measurement, the system state in the runway domain is updated. and its uncertainty matrix If the camera does not capture a visual image at the current moment, return to step S2.

[0223] Intelligent detection algorithms include methods such as neural networks.

[0224] Specifically, S4 includes:

[0225] S41 determines the measurement domain based on sensor data from the inertial vision integrated navigation system.

[0226] Specifically, the measurement domain refers to the location of the sensor's measurement results, involving the acquisition and processing of sensor data. This includes devices such as visual sensors, radio altimeters, GNSS, and inertial navigation systems that capture environmental information. The measurement results from radio altimeters, GNSS, and inertial navigation systems provide information such as position and velocity, while visual sensors provide pixel-level feature information; the types of information differ significantly. Since the processing of measurement results from radio altimeters, GNSS, and inertial navigation systems is relatively mature, this discussion primarily focuses on the measurement domain of visual sensors. The camera's measurement results are dimensionless pixel-level features, based on a pixel coordinate system. This pixel coordinate system is defined as the s-coordinate system, with the origin at the top left corner of the image. The positive x-axis runs horizontally to the right, and the positive y-axis runs vertically downwards. Since the image is a 2D plane, it lacks a z-axis, only having x and y axes. During imaging, the camera reduces the 3D plane to a 2D plane, losing depth information along the camera's optical axis. See details... Figure 3 .

[0227] S42. Obtain the full position and velocity information of the aircraft from the external inertial navigation high-frequency system. The inertial navigation system calculates the position as follows: .

[0228] S43. Acquire image information and corresponding timestamps from the vision sensor at a lower frequency, and use an intelligent detection algorithm to obtain the visual measurement results. .

[0229] in, The pixel slope representing the two feature lines. The pixel coordinates represent a feature point.

[0230] S44. Determine the visual measurement results respectively. Predicted value , , , .

[0231] The prediction of point features can be expressed as:

[0232]

[0233] The prediction of line features can be expressed as:

[0234]

[0235] In the above formula, Here are the coordinates of the feature point in the camera frame. Here are the coordinates of the feature point in the runway system. This represents the rotation and translation relationship of projecting point coordinates from the runway frame to the camera frame, where K is an intrinsic parameter. For normalization parameters, The Plück matrix represents a three-dimensional straight line. The Plück matrix represents a two-dimensional straight line.

[0236] S45. Based on the predicted value , , , and visual measurement results ,use

[0237] Determine the total measurement of the filter. .

[0238] S46. Based on the database information of the inertial vision integrated navigation system and the full attitude and velocity information of the aircraft under the navigation system, solve for the filter observation matrix H.

[0239] Specifically, S46 includes:

[0240] S461. Based on the rotation matrix from the n-system to the c-system. Determine parameters .

[0241] ,

[0242] S462, according to Using the formula Solve for parameters , .

[0243] It should be noted that, for A normalized direction vector with a magnitude of 1. for The length of the module.

[0244] S463, According to parameters , Using formula

[0245] Determine parameters .

[0246] S464. Calculate the position based on the inertial navigation system. The navigation system provides the aircraft's full attitude and velocity information, as well as the actual runway domain location of the observed feature points. Determine the camera system position of the feature points to be observed. Specifically, when the rotation matrix from the runway system to the camera system...

[0247] hour,

[0248]

[0249]

[0250] S465. Calculate the position based on the inertial navigation system. Rotation matrix from navigation system to camera system ,state With camera and airplane boom Using the formula

[0251] Determine the runway coordinates of the current camera position. Specifically, when the rotation matrix from the navigation system to the runway system...

[0252] , hour,

[0253]

[0254] S466, According to parameters runway width Determine parameters .

[0255]

[0256]

[0257]

[0258]

[0259] S467. Determine the parameters based on the camera's intrinsic parameter K. , .

[0260] S468, According to parameters , , The current camera position in the runway coordinate system. ,parameter Determine parameters , , , .

[0261]

[0262]

[0263]

[0264]

[0265] S469, According to parameters , , , , , , Camera system location of the feature points to be observed The actual runway domain location of the feature point to be observed Rotation matrix from runway system to camera system Solve for matrix H.

[0266]

[0267]

[0268] S48 performs a status update and obtains the updated runway domain status. and its runway domain uncertainty matrix .

[0269]

[0270]

[0271]

[0272] In the above formula, This is the visual observation error, given by the detection algorithm. This is the covariance matrix before the runway domain state update. To measure the updated runway domain status, This is the uncertainty matrix of the runway domain state before the measurement update. This is the uncertainty matrix of the updated runway domain state.

[0273] S5, runway domain status and its uncertainty matrix Transform into navigation domain state and its uncertainty matrix .

[0274] Connect all the above results, Figure 7 In the diagram, red lines represent branches that operate continuously at any given time, while green lines represent data branches that only operate when observations are received. At any given time, time updates occur in the navigation domain state and are projected onto the runway domain state using the transformation relationship described in S3. When observations are received, measurement updates occur in the runway domain state, and the updated state and uncertainty matrix are projected onto the navigation domain state using the inter-domain transformation relationship described in S5. This process is repeated continuously.

[0275] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An inertial vision integrated navigation method based on multi-domain transformation, characterized in that, The methods include: S1. Obtain the database information of the inertial vision integrated navigation system; S2, Initialize the system error state in the navigation domain, and update the system state in the navigation domain according to the time update principle of the system in the navigation domain; S3 transforms the navigation domain state and uncertainty matrix into the runway domain state and uncertainty matrix; S4, if the camera captures a visual image at the current moment, the visual measurement in the measurement domain is obtained from the visual image at the current moment through an intelligent detection algorithm; based on the visual measurement, the runway domain state is updated. and its uncertainty matrix If the camera does not capture a visual image at the current moment, return to step S2. S5, runway domain status and its uncertainty matrix Transform into navigation domain state and its uncertainty matrix This enables inertial-visual integrated navigation; S3 includes: S31. Determine the runway domain based on the target runway coordinate system; S32. Determine the runway domain status based on user concerns and visual measurement characteristics. : In the above formula, This represents the position error projection in the runway system. This represents the projection of the misalignment angle error in the runway frame. S33, System position error in the navigation domain Converted to system position error in the runway domain ; in, Let be the rotation matrix from the navigation frame to the runway frame. This represents the projected position error under the runway system. This represents the position error projection in the navigation system; S34. Project the misalignment angle error under the navigation system. This is converted into the projection of the misalignment angle error in the runway frame. ; S35. The error uncertainty matrix in the navigation domain Converted into the error uncertainty matrix in the runway domain ; S34 includes: Using formula Projecting the misalignment angle error under the navigation system This is converted into the projection of the misalignment angle error in the runway frame. ; in, Projection of misalignment angle error in navigation system The corresponding rotation matrix, Projection of misalignment angle error in the runway frame The corresponding rotation matrix, The rotation matrix representing the navigation frame to the runway frame; S35 includes: S351, Solve eigenvalues and eigenvectors ; S352. Use formula: , Solve eigenvalues and eigenvectors ; S353. Use formula Solve the error uncertainty matrix in the runway domain. .

2. The inertial vision integrated navigation method according to claim 1, characterized in that, S2 include: S21. Determine the navigation domain of the inertial vision integrated navigation system based on the WGS 84 coordinate system, local navigation system, machine system, and camera system. S22. Determine the navigation domain state. : In the above formula, This represents the position error projection in the navigation system. This represents the velocity error in the local navigation system. This represents the projection of the misalignment angle error in the navigation system. S23. Initialize navigation domain state ; S24. Based on the time update principle of the system under the navigation domain, the navigation domain state at time k is... Update the time.

3. The inertial vision integrated navigation method according to claim 1, characterized in that, S4 include: S41 determines the measurement domain based on sensor data from the inertial vision integrated navigation system; S42. Obtain the full position and velocity information of the aircraft from the external inertial navigation high-frequency system, whereby the inertial navigation system calculates the position as follows: ; S43. Acquire image information and corresponding timestamps from the vision sensor at a lower frequency, and use an intelligent detection algorithm to obtain the visual measurement results. ;in, The pixel slope representing the two feature lines. The pixel coordinates representing a feature point; S44. Determine the visual measurement results respectively. Predicted value , , , ; S45. Based on the predicted value , , , and visual measurement results ,use Determine the total measurement of the filter. ; S46. Based on the database information of the inertial vision integrated navigation system and the full attitude and velocity information of the aircraft under the navigation system, solve for the filter observation matrix H; S47 performs a status update and obtains the updated runway domain status. and its runway domain uncertainty matrix .

4. The inertial vision integrated navigation method according to claim 3, characterized in that, S46 includes: S461, Based on the rotation matrix from the navigation frame to the camera frame. Determine parameters ； ； S462, according to Using the formula Solve for parameters , ; S463, According to parameters , Using formula Determine parameters ; S464. Based on the database information of the inertial vision integrated navigation system, the aircraft's full pose and velocity information under the navigation system, and the actual runway domain position of the observed feature points. Determine the camera system position of the feature points to be observed. : in, Let be the rotation matrix from the runway system to the camera system; S465. Calculate the position based on the inertial navigation system. Rotation matrix from navigation system to camera system ,state With camera and airplane boom Using the formula ， Determine the runway coordinates of the current camera position. ; S466, According to parameters runway width Determine parameters ; ； ； ； S467. Determine the parameters based on the camera's intrinsic parameter K. , ; S468, According to parameters , , The current camera position in the runway coordinate system. ,parameter Determine parameters , , , ; ； ； ； ； S469, According to parameters , , , , , , Camera system location of the feature points to be observed The actual runway domain location of the feature point to be observed Rotation matrix from runway system to camera system Solve for matrix H; ； 。 5. The inertial vision integrated navigation method according to claim 3, characterized in that, S44 includes: The prediction of point features can be expressed as: The prediction of line features can be expressed as: In the above formula, Here are the coordinates of the feature point in the camera frame. Here are the coordinates of the feature point in the runway system. This represents the rotation and translation relationship of projecting point coordinates from the runway frame to the camera frame, where K is an intrinsic parameter. For normalization parameters, The Plück matrix represents a three-dimensional straight line. The Plück matrix represents a two-dimensional straight line.

6. An inertial vision-based integrated navigation device based on multi-domain transformation, characterized in that, The device is used to implement the inertial vision integrated navigation method based on multi-domain transformation as described in claim 1.

Images