Low-light visual-inertial positioning system

Abstract

This invention discloses a low-light visual-inertial positioning system, including a power supply, a visual-inertial sensor, and a system motherboard. The visual-inertial sensor includes a camera and an inertial measurement unit (IMU). The system motherboard includes an image enhancement and denoising module, a pre-integration module, and a visual-inertial odometry module. One end of the image enhancement and denoising module is connected to the camera, and the other end is connected to the visual-inertial odometry module. One end of the pre-integration module is connected to the IMU, and the other end is connected to the visual-inertial odometry module. The low-light visual-inertial positioning system provided by this invention combines image enhancement technology with a camera response model to preprocess low-light images, improving image brightness and contrast. Simultaneously, it employs optical flow to track features and constructs a tightly coupled visual-inertial optimization model for pose estimation, achieving more stable and accurate low-light scene positioning.

Description

Technical Field

[0001] This invention relates to the field of navigation and positioning technology, and in particular to a low-light visual inertial positioning system. Background Technology

[0002] Since traditional GPS (Global Positioning System)-based positioning methods fail to function properly in denied environments, autonomous navigation and positioning technology has become a research hotspot in recent years. Visual-inertial positioning (VIS) is a key technology for autonomous navigation, offering advantages such as low sensor cost and small size. Furthermore, the characteristics of visual sensors and inertial measurement units (IMUs) are complementary. When the visual sensor moves rapidly or lacks texture and fails, IMU data can provide accurate positioning for the system within a short time. Conversely, the visual sensor maintains higher accuracy during low-speed system movement, effectively reducing the long-term cumulative error of the IMU.

[0003] Existing visual-inertial localization methods are mainly divided into two schemes: loosely coupled and tightly coupled. In the loosely coupled scheme, pure visual pose estimation is performed based on visual sensor information, while the inertial sensor performs pose estimation through integration. The two pose estimation results are then fused using methods such as extended Kalman filtering to obtain the final pose estimate. In the tightly coupled scheme, visual reprojection error and inertial measurement residuals are used as constraints to construct an objective optimization function that minimizes the overall error, thereby obtaining the optimal pose estimate.

[0004] Visual-inertial positioning systems (VIS systems) have high requirements for input image quality and poor robustness to changes in lighting. In low-light scenarios, due to the lack of clear image texture and the difficulty in feature point extraction, existing technologies are prone to problems such as accumulated errors, pose drift, and even tracking failures. Many low-light scenarios exist in practical applications of navigation systems, such as all-weather navigation, underground robots, and nighttime individual soldier navigation. Existing VIS methods struggle to achieve accurate positioning in low-light conditions. Summary of the Invention

[0005] To address the limitations and defects of existing technologies, this invention provides a low-light visual-inertial positioning system, comprising a power supply, a visual-inertial sensor, and a system motherboard. The visual-inertial sensor includes a camera and an inertial measurement unit. The system motherboard includes an image enhancement and denoising module, a pre-integration module, and a visual-inertial odometry module. The power supply provides 12 volts to the camera, 5 volts to the inertial measurement unit, and 12 volts to the system motherboard.

[0006] One end of the image enhancement and denoising module is connected to the camera, and the other end of the image enhancement and denoising module is connected to the visual inertial odometry module. One end of the pre-integration module is connected to the inertial measurement unit, and the other end of the pre-integration module is connected to the visual inertial odometry module.

[0007] The camera and the inertial measurement unit are used to acquire visual and inertial information, and the system motherboard is used to process the visual and inertial information to obtain pose information;

[0008] The pre-integration module is used to estimate the pose and velocity changes between adjacent time points;

[0009] The image enhancement and denoising module is used to determine the camera response model based on the histogram of images with different exposures, determine the model parameters through curve fitting, obtain the illumination map and exposure rate matrix of the low-light image using nonlinear optimization, preprocess the low-light image according to the camera response model, brighten the low-light image as a whole while increasing the contrast, and denoise the image at the same time.

[0010] The visual-inertial odometry module is used to perform feature tracking based on optical flow to obtain feature matching point pairs, perform joint calibration and adjustment of the visual and inertial information to form a visual and inertial tightly coupled optimization model, and correct the keyframe pose according to the sliding window optimization method to control the dimension of optimization variables and achieve real-time pose updates.

[0011] Optionally, the image enhancement and denoising module is further configured to form an image enhancement process model based on the camera response model, the expression of which is as follows:

[0012] P′=g(P,K) (1)

[0013] Where P is the input low-light image, P′ is the enhanced output image, g represents the camera response model, and K represents the exposure rate matrix, which includes the expected exposure rate of each pixel;

[0014] The image enhancement and denoising module is also used to observe the histograms of images with different exposures of the same scene, first linearly enlarge the pixels of the low-light image, and then perform gamma correction to obtain the enhanced image. The expression of the camera response model is determined as follows:

[0015]

[0016] Where p and k are any pixel value and the exposure rate corresponding to that pixel value, respectively, β represents the linear magnification factor, γ represents the gamma correction factor, c represents the irradiance corresponding to that pixel, and a and b are the parameters of the camera response model, with a set to -0.3293 and b set to 1.1258.

[0017] Optionally, the image enhancement and denoising module is further used to solve for the illumination map T of the low-light image, using the difference and smoothness between the illumination value to be optimized and the initial illumination estimate as constraints to form an objective optimization function, the expression of which is as follows:

[0018]

[0019] Where x represents any pixel in the image, I(x) represents the initial illumination estimate of pixel x, T(x) is the illumination value to be optimized, α is the coefficient balancing the two residual terms, α is set to 1, h and v represent the horizontal and vertical directions respectively, and W d (x) represents the weight of pixel x. Let represent the first derivatives of the lighting value to be optimized and the initial lighting estimate in the d direction, respectively, and ε represent an infinitesimal quantity;

[0020] W d The expression for (x) is as follows:

[0021]

[0022] Where |*| represents the absolute value operator, and ω(x) is a local window centered at pixel x. This represents the first derivative of the initial illumination value of pixel n in the d direction;

[0023] The image enhancement and denoising module is also used to solve the target optimization function to obtain the optimal solution of the illumination map T, and to obtain the exposure rate matrix K by taking the reciprocal of all elements in the illumination map T.

[0024] Optionally, the image enhancement and denoising module is further configured to denoise the image using a Gaussian filter. The Gaussian filter takes the weighted average of the pixels within the convolution kernel as the gray value of the center point as the output. The expression for the Gaussian filter is as follows:

[0025]

[0026] Where (x,y) represents the coordinates of any pixel within the convolution kernel, G(x,y) represents the weight of that pixel, σ represents the standard deviation, σ is set to 1.5, the convolution kernel size is set to 3×3, and e is the natural constant.

[0027] Optionally, the visual inertial odometry module is further configured to obtain optical flow constraint equations based on the LK optical flow algorithm, the expression of which is as follows:

[0028] I x u+I y v+I t =0 (8)

[0029] in, Let x, y, and t represent the partial derivatives of the gray value with respect to x, y, and t, respectively. (x, y) are the coordinates of any pixel in the image. The gray value of the pixel at time t is I(x, y, t). u represents the moving speed of the pixel in the horizontal axis direction, and v represents the moving speed of the pixel in the vertical axis direction.

[0030] The visual inertial odometry module is also used to obtain the neighborhood Ω containing the pixel. The neighborhood Ω is set to include n pixels, each pixel having the same velocity vector (u,v), resulting in the following overdetermined equation set:

[0031]

[0032] The visual inertial odometry module is also used to find the optimal solution (u) of the overdetermined system of equations using the least squares method. * ,v * This allows us to obtain the position of the pixel after time Δt, thus enabling feature tracking of low-light images.

[0033] Optionally, the visual inertial odometry module is further configured to use a Gaussian pyramid combined with the LK optical flow algorithm to perform optical flow calculation on the nth layer, obtain the tracking result of the nth layer, use the tracking result of the nth layer as the initial value of the optical flow of the (n-1)th layer, obtain the tracking result of the (n-1)th layer, use the tracking result of the (n-1)th layer as the initial value of the next layer, until the optical flow result of the original image pair is obtained, wherein the original image is at the 0th layer.

[0034] Optionally, the visual inertial odometry module is also used to solve the rotation and translation between two frames using an epipolar geometry model, obtain the three-dimensional coordinates of the observed feature points in the two frames using triangulation, and obtain the pose and three-dimensional feature points of other frames within the sliding window using the PnP algorithm based on the three-dimensional feature points and the two-dimensional feature points observed in other frames.

[0035] The visual inertial odometry module is also used to fuse visual and inertial measurement unit information, using the rotation information obtained by pre-integration of the inertial measurement unit as the estimated value and the rotation information obtained by the visual initialization process as the observed value. Under the premise of ignoring measurement noise, the error between the estimated value and the observed value is only related to the gyroscope bias.

[0036] The visual inertial odometry module is also used to form a minimum error model based on the difference between the rotation information of all frames in the sliding window and the rotation information of the inertial measurement unit, and to obtain the gyroscope bias using the least squares method based on the minimum error model.

[0037] The visual inertial odometry module is also used to calibrate the velocity, gravitational acceleration, and scale of the low-light visual inertial positioning system using translational and velocity information from the visual and inertial measurement units as constraints.

[0038] Optionally, the visual inertial odometry module is further configured to form a visual inertial tightly coupled optimization model based on the sliding window, and optimize the pose, velocity, inertial measurement unit offset, extrinsic parameters from the inertial measurement unit to the camera, and inverse depth of the three-dimensional feature points for each frame within the sliding window based on the visual inertial tightly coupled optimization model.

[0039] The visual inertial odometry module is also used to obtain inertial constraints based on the error between the measured value and the state estimate of the inertial measurement unit within the sliding window, to obtain visual constraints based on the reprojection error of the feature points within the sliding window, and to obtain prior constraints based on the old frame that has been removed from the sliding window.

[0040] The visual inertial odometry module is also used to optimize the objective function based on visual error, inertial measurement unit error, and prior error as constraints, and obtain the optimal pose estimate when the overall error is minimized.

[0041] The present invention has the following beneficial effects:

[0042] This invention provides a low-light visual-inertial positioning system, including a power supply, a visual-inertial sensor, and a system motherboard. The visual-inertial sensor includes a camera and an inertial measurement unit (IMU). The system motherboard includes an image enhancement and denoising module, a pre-integration module, and a visual-inertial odometry module. One end of the image enhancement and denoising module is connected to the camera, and the other end is connected to the visual-inertial odometry module. One end of the pre-integration module is connected to the IMU, and the other end is connected to the visual-inertial odometry module. The low-light visual-inertial positioning system provided by this invention combines image enhancement technology with a camera response model to preprocess low-light images, improving image brightness and contrast. Simultaneously, it employs optical flow to track features and constructs a tightly coupled visual-inertial optimization model for pose estimation, achieving more stable and accurate low-light scene positioning. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the low-light visual inertial positioning system provided in Embodiment 1 of the present invention.

[0044] Figure 2 This is a flowchart illustrating the operation of a low-light visual inertial positioning system provided in Embodiment 1 of the present invention.

[0045] Figure 3This is a schematic diagram of optical flow tracing provided in Embodiment 1 of the present invention.

[0046] Figure 4a This is a schematic diagram of the Gaussian pyramid, provided as the first image in Embodiment 1 of the present invention.

[0047] Figure 4b This is a schematic diagram of the Gaussian pyramid, provided as a second image in Embodiment 1 of the present invention.

[0048] Figure 5 This is a schematic diagram of the calibration process provided in Embodiment 1 of the present invention.

[0049] Figure 6 This is a schematic diagram of the sliding window optimization provided in Embodiment 1 of the present invention.

[0050] Figure 7 This is a schematic diagram of visual-inertial tight coupling optimization provided in Embodiment 1 of the present invention. Detailed Implementation

[0051] To enable those skilled in the art to better understand the technical solution of the present invention, the low-light visual inertial positioning system provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0052] Example 1

[0053] Figure 1 This is a schematic diagram of the low-light visual-inertial positioning system provided in Embodiment 1 of the present invention. Figure 1 As shown, the hardware consists of a visual-inertial sensor and a system motherboard. The camera and system motherboard are powered by 12V, while the IMU is powered by 5V. The system motherboard comprises an image enhancement and denoising module, a pre-integration module, and a visual-inertial odometry module. The visual and inertial information acquired by the camera and IMU is processed by the system motherboard to obtain pose information. The pre-integration module estimates the pose and velocity changes between adjacent time points.

[0054] Figure 2 This is a flowchart illustrating the operation of a low-light visual-inertial positioning system provided in Embodiment 1 of the present invention. Figure 2 As shown, the low-light image acquired by the visual sensor first enters the image enhancement module for image preprocessing, which brightens the overall image and increases contrast while denoising the image to facilitate subsequent feature tracking. However, the quality of the enhanced low-light image is still difficult to reach the level of a normal image, making feature tracking methods based on feature point matching difficult to implement. In this embodiment, optical flow is used for feature tracking to obtain matching point pairs. Next, visual and inertial information are jointly calibrated, and the backend uses a sliding window optimization method to correct the keyframe pose, thereby controlling the optimization variable dimension and achieving real-time pose updates.

[0055] This embodiment uses a camera response model combined with Gaussian filtering for denoising as an image enhancement module. First, the camera response model is determined based on the histograms of images with different exposures, and the model parameters are determined through curve fitting. Second, nonlinear optimization is used to obtain the illumination map and exposure rate matrix of the low-light image. Finally, the low-light image is preprocessed and denoised based on the camera response model.

[0056] Since low-light images are not globally underexposed, some areas are normal while others are underexposed. To avoid overexposure in some areas, a uniform global exposure rate cannot be used to expose all pixels. Based on the camera response model, the image enhancement process is modeled as follows:

[0057] P′=g(P,K) (1)

[0058] Where P and P′ are the input low-light image and the enhanced output image, respectively; g represents the camera response model; and K represents the exposure rate matrix, which contains the expected exposure rate of each pixel, with bright areas corresponding to smaller exposure rates and dark areas corresponding to larger exposure rates.

[0059] By observing the histograms of images with different exposures of the same scene, linearly enlarging the pixels of the low-light image before gamma correction yields a well-exposed image, i.e., the enhanced image. The camera response model is determined as follows:

[0060]

[0061] Where p and k are a certain pixel value and the exposure rate corresponding to that pixel value, respectively; β represents the linear magnification factor, γ represents the gamma correction factor; e represents the irradiance corresponding to that pixel; a and b are the parameters of the camera response model, which can be obtained through curve fitting, with a set to -0.3293 and b set to 1.1258.

[0062] To avoid localized overexposure in the image, brighter areas should be assigned lower exposure rates. Therefore, the exposure rate map is inversely proportional to the illumination map T. First, the illumination map T of the low-light image is solved using the Local Interpretable Model-agnostic Explanations (LIME) algorithm. The difference between the illumination estimate to be optimized and the initial estimate, as well as the smoothness, are used as constraints to establish the objective optimization function as follows:

[0063]

[0064] Where x represents a pixel in the image; I(x) represents the initial illumination estimate of pixel x; T(x) is the illumination value to be optimized; α is the coefficient balancing the two residual terms, and α is set to 1; h and v represent the horizontal and vertical directions, respectively; W d(x) represents the weight of pixel x; Let represent the first derivatives of the illumination value to be optimized and the initial illumination value in the d direction, respectively; ε represents an infinitesimal. Optimize the objective function to obtain the optimal solution for the illumination map T. Taking the reciprocal of all elements in T yields the exposure rate matrix K. W d The expression for (x) is as follows:

[0065]

[0066] Where |*| represents the absolute value operator, and ω(x) is a local window centered at pixel x. It represents the first derivative of the initial illumination value of n pixels in the d direction.

[0067] Low-light images, after being enhanced by algorithms, generate a large amount of Gaussian noise. The Gaussian noise in the image exhibits abrupt changes in grayscale values ​​with surrounding pixels, which can easily be mistaken for corner points and hinder subsequent feature tracking. To reduce the impact of image noise on feature tracking and improve the system's localization accuracy, this embodiment incorporates a Gaussian filter into the aforementioned algorithm to denoise the image.

[0068] The Gaussian filter takes the weighted average of the pixels within the convolution kernel as the gray value of the center point as the output, thereby reducing noise and smoothing the image. The equation of the Gaussian filter is as follows:

[0069]

[0070] Where (x,y) represents the coordinates of a pixel within the convolution kernel, G(x,y) represents the weight of that pixel, σ represents the standard deviation, and e is the natural constant. In this embodiment, the convolution kernel size is set to 3×3, and σ is set to 1.5.

[0071] In this embodiment, optical flow is used for feature tracking to obtain feature matching point pairs. Then, visual and inertial information are jointly calibrated to construct a tightly coupled optimization model of vision and inertial. A sliding window optimization method is used to correct the keyframe pose in order to control the dimension of optimization variables and achieve real-time pose updates.

[0072] Because image features and textures are not obvious in low-light scenes, feature point extraction and descriptor matching are difficult, making feature point methods prone to failure. This embodiment uses optical flow for feature tracking, which reduces the time consumption of descriptor matching while ensuring accuracy.

[0073] Figure 3 This is a schematic diagram of optical flow tracing provided in Embodiment 1 of the present invention. Figure 3 As shown, visual information in low-light scenes satisfies the assumptions of constant brightness, temporal continuity, and spatial consistency. This paper uses the LK (Lucas-Kanade) optical flow algorithm for solving this problem. The optical flow tracing process is as follows: Figure 3 As shown, suppose the pixel coordinates of a point in the image are (x, y), and the gray value of this point at time t is I(x, y, t). Due to camera movement, after time Δt, the point moves to a new position (x+Δx, y+Δy), and its gray value becomes I(x+Δx, y+Δy, t+Δt). According to the assumption of constant brightness, the gray value of this point remains unchanged, that is:

[0074] I(x+Δx,y+Δy,t+Δt)=I(x,y,t) (5)

[0075] Performing a Taylor expansion on the left-hand side of the above equation yields the following expression:

[0076]

[0077] β is a second-order infinitesimal term, which can be ignored. Simplifying formula (6), we obtain the following expression:

[0078]

[0079] Dividing both sides of equation (7) by dt, we obtain the optical flow constraint equation as follows:

[0080] I x u+I y v+I t =0 (8)

[0081] in, represents the partial derivatives of the grayscale value with respect to x, y, and t, respectively. u and v represent the movement speed of the pixel along the horizontal and vertical coordinate axes, respectively.

[0082] In this embodiment, a neighborhood Ω containing the pixel is taken. Assume that there are n pixels in the neighborhood Ω, and each pixel has the same velocity vector (u,v). The following overdetermined equations are obtained:

[0083]

[0084] Figure 4a This is a schematic diagram of the Gaussian pyramid provided in Embodiment 1 of the present invention. Figure 4b This is a schematic diagram of a Gaussian pyramid provided in Embodiment 1 of the present invention. Figures 4a-4b As shown, this embodiment uses the least squares method to find the optimal solution (u*, v*) of the equation system, thereby obtaining the position of the pixel after time Δt, and realizing feature tracking of low-light images. To ensure that the requirements are met when the camera is moving rapidly, a Gaussian pyramid combined with LK optical flow is used to improve accuracy.

[0085] This embodiment utilizes a Gaussian pyramid to implement a multi-layer optical flow method, avoiding the result from getting trapped in local minima due to excessive image changes during rapid camera movement. When calculating the optical flow between the first and second images, the original image is used as layer 0. The first layer image is obtained by scaling the layer 0 image to a certain ratio, the second layer image is obtained by scaling the first layer image to a certain ratio, and so on until the top layer (layer n). First, optical flow calculation is performed on layer n, and the tracking result is used as the initial value for the optical flow of layer n-1. The tracking result of layer n-1 is then optimized and used as the initial value for the next layer, until the optical flow result of the original image pair is obtained.

[0086] Figure 5 This is a schematic diagram of the calibration process provided in Embodiment 1 of the present invention. Figure 5 As shown, to ensure the positioning accuracy of the system, visual and IMU information will be combined to calibrate the gyroscope bias, velocity (v), gravitational acceleration (g), and scale (s).

[0087] This embodiment first uses an epipolar geometry model to solve for the rotation and translation between two frames, and uses triangulation to obtain the three-dimensional coordinates of the observed feature points in the two frames. Then, based on these three-dimensional feature points and the two-dimensional feature points observed in other frames, PnP (Perspective-n-Point) is used to solve for the pose (translation, rotation) and three-dimensional feature points of other frames in the sliding window.

[0088] This embodiment fuses visual and IMU information, using the rotation information obtained from IMU pre-integration as the estimated value and the rotation information obtained from the aforementioned visual initialization process as the observed value. Measurement noise is ignored, and the error between the two is only related to the gyroscope bias. A minimum error model is constructed by subtracting the rotation information of all frames within the sliding window from the IMU rotation information. The gyroscope bias is then obtained using the least squares method. Similarly, the translation and velocity information from both vision and IMU are used as constraints to calibrate the system's velocity, gravitational acceleration, and scale.

[0089] Figure 6 This is a schematic diagram illustrating the optimized sliding window provided in Embodiment 1 of the present invention. Figure 6 As shown, to reduce estimation errors and further improve the system's positioning accuracy, this embodiment employs a sliding window-based optimization method to correct the pose. The number of image frames within the sliding window remains constant; whenever a new frame is added to the window, an older frame is removed, thereby controlling the computational scale and enabling real-time updates to the pose estimation.

[0090] This embodiment constructs a visual-inertial tightly coupled optimization model based on a sliding window. The optimization variables include the pose, velocity, IMU bias, extrinsic parameters from the IMU to the camera, and inverse depth of the 3D feature points for each frame within the sliding window. Visual error, IMU error, and prior error are used as constraints to optimize the objective function, minimizing the overall error and obtaining the optimal pose estimation.

[0091] Figure 7 This is a schematic diagram of visual-inertial tight coupling optimization provided in Embodiment 1 of the present invention. Figure 7 As shown, the inertial constraint is the error between the IMU measurement and the state estimate within the sliding window; the visual constraint is the reprojection error of the feature points within the sliding window; and the prior constraint comes from the old frame that has been removed from the sliding window. After being removed from the sliding window, the pose of the old frame no longer participates in the optimization, but the constraint relationship between the old frame and other frames within the sliding window is added to the optimization function as prior information. This controls the dimensionality of the optimization variables without reducing the constraint information, which is beneficial to improving the solution accuracy.

[0092] This embodiment provides a low-light visual-inertial positioning system, including a power supply, a visual-inertial sensor, and a system motherboard. The visual-inertial sensor includes a camera and an inertial measurement unit (IMU). The system motherboard includes an image enhancement and denoising module, a pre-integration module, and a visual-inertial odometry module. One end of the image enhancement and denoising module is connected to the camera, and the other end is connected to the visual-inertial odometry module. One end of the pre-integration module is connected to the IMU, and the other end is connected to the visual-inertial odometry module. The low-light visual-inertial positioning system provided in this embodiment combines image enhancement technology with a camera response model to preprocess low-light images, improving image brightness and contrast. Simultaneously, it employs optical flow to track features and constructs a tightly coupled visual-inertial optimization model for pose estimation, achieving more stable and accurate low-light scene positioning.

[0093] It is understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A low-light visual-inertial positioning system, characterized in that, The system includes a power supply, a visual inertial sensor, and a system motherboard. The visual inertial sensor includes a camera and an inertial measurement unit. The system motherboard includes an image enhancement and denoising module, a pre-integration module, and a visual inertial odometry module. The power supply provides 12 volts to the camera, 5 volts to the inertial measurement unit, and 12 volts to the system motherboard. One end of the image enhancement and denoising module is connected to the camera, and the other end of the image enhancement and denoising module is connected to the visual inertial odometry module. One end of the pre-integration module is connected to the inertial measurement unit, and the other end of the pre-integration module is connected to the visual inertial odometry module. The camera and the inertial measurement unit are used to acquire visual and inertial information, and the system motherboard is used to process the visual and inertial information to obtain pose information; The pre-integration module is used to estimate the pose and velocity changes between adjacent time points; The image enhancement and denoising module is used to determine the camera response model based on the histogram of images with different exposures, determine the model parameters through curve fitting, obtain the illumination map and exposure rate matrix of the low-light image using nonlinear optimization, preprocess the low-light image according to the camera response model, brighten the low-light image as a whole while increasing the contrast, and denoise the image at the same time. The visual-inertial odometry module is used to perform feature tracking based on optical flow to obtain feature matching point pairs, perform joint calibration and adjustment of the visual and inertial information to form a visual and inertial tightly coupled optimization model, and correct the keyframe pose according to the sliding window optimization method to control the dimension of optimization variables and realize real-time pose updates. The image enhancement and denoising module is also used to solve the illumination map of the low-light image. The difference and smoothness between the lighting value to be optimized and the initial lighting estimate are used as constraints to form an objective optimization function, the expression of which is as follows: （3） in, Represents any pixel in the image. express Initial illumination estimate of the pixel, The lighting value to be optimized, To balance the coefficients of the two residual terms, Set to 1, , These represent the horizontal and vertical directions, respectively. express Pixel weights , The light value to be optimized and the initial light estimate respectively represent the light values ​​in the range of the light source. The first derivative of the direction, Indicates an infinitesimal quantity; The expression is as follows: ， in, This represents the absolute value operator. Therefore A local window centered on a pixel. express Initial illumination value of pixel The first derivative in the direction; The image enhancement and denoising module is further configured to solve for the illumination map based on the objective optimization function. The optimal solution for the lighting map The exposure rate matrix is ​​obtained by taking the reciprocal of all elements. K .

2. The low-light visual-inertial positioning system according to claim 1, characterized in that, The image enhancement and denoising module is further configured to form an image enhancement process model based on the camera response model, the expression of which is as follows: （1） in, To input a low-light image, To enhance the output image, Represents the camera response model. This represents the exposure rate matrix, which includes the expected exposure rate for each pixel; The image enhancement and denoising module is also used to observe the histograms of images with different exposures of the same scene, first linearly enlarge the pixels of the low-light image, and then perform gamma correction to obtain the enhanced image. The expression of the camera response model is determined as follows: （2） in, , These represent any pixel value and its corresponding exposure rate. Indicates the linear amplification factor. Indicates the gamma correction factor. This indicates the irradiance corresponding to that pixel. , The parameters of the camera response model are... Set to -0.3293, Set to 1.1258.

3. The low-light visual-inertial positioning system according to claim 2, characterized in that, The image enhancement and denoising module is further used to denoise the image using a Gaussian filter. The Gaussian filter takes the weighted average of the pixels within the convolution kernel as the gray value of the center point as the output. The expression of the Gaussian filter is as follows: （4） in, This represents the coordinates of any pixel within the convolution kernel. This indicates the weight of that pixel. Indicates standard deviation, Set to 1.5, kernel size set to , It is a natural constant.

4. The low-light visual-inertial positioning system according to claim 3, characterized in that, The visual inertial odometry module is also used to obtain optical flow constraint equations based on the LK optical flow algorithm. The expression of the optical flow constraint equations is as follows: （8） in, , , Representing grayscale value pairs , , The partial derivatives, Let be the coordinates of any pixel in the image, where the pixel is located. The grayscale value at time 1 , This indicates the speed at which the pixel moves along the horizontal axis. This indicates the speed at which the pixel moves along the vertical axis. The visual inertial odometry module is also used to acquire the neighborhood containing the pixel. Set the neighborhood include There are 10 pixels, each with the same velocity vector. The following overdetermined system of equations is obtained: （9） The visual inertial odometry module is also used to find the optimal solution to the overdetermined system of equations using the least squares method. , obtain the elapsed time The position of the next pixel enables feature tracking in low-light images.

5. The low-light visual-inertial positioning system according to claim 4, characterized in that, The visual inertial odometry module is also used to calculate the optical flow of the nth layer using a Gaussian pyramid combined with the LK optical flow algorithm to obtain the tracking result of the nth layer. The tracking result of the nth layer is used as the initial value of the optical flow of the (n-1)th layer to obtain the tracking result of the (n-1)th layer. The tracking result of the (n-1)th layer is used as the initial value of the next layer until the optical flow result of the original image pair is obtained. The original image is at the 0th layer.

6. The low-light visual-inertial positioning system according to claim 5, characterized in that, The visual inertial odometry module is also used to solve the rotation and translation between two frames using an epipolar geometry model, to obtain the three-dimensional coordinates of the observed feature points in the two frames using triangulation, and to obtain the pose and three-dimensional feature points of other frames within the sliding window using the PnP algorithm based on the three-dimensional feature points and the two-dimensional feature points observed in other frames. The visual inertial odometry module is also used to fuse visual and inertial measurement unit information, using the rotation information obtained by pre-integration of the inertial measurement unit as the estimated value and the rotation information obtained by the visual initialization process as the observed value. Under the premise of ignoring measurement noise, the error between the estimated value and the observed value is only related to the gyroscope bias. The visual inertial odometry module is also used to form a minimum error model based on the difference between the rotation information of all frames in the sliding window and the rotation information of the inertial measurement unit, and to obtain the gyroscope bias using the least squares method based on the minimum error model. The visual inertial odometry module is also used to calibrate the velocity, gravitational acceleration, and scale of the low-light visual inertial positioning system using translational and velocity information from the visual and inertial measurement units as constraints.

7. The low-light visual-inertial positioning system according to claim 6, characterized in that, The visual inertial odometry module is also used to form a visual inertial tightly coupled optimization model based on the sliding window, and to optimize the pose, velocity, inertial measurement unit offset, extrinsic parameters from the inertial measurement unit to the camera, and inverse depth of the three-dimensional feature points for each frame within the sliding window based on the visual inertial tightly coupled optimization model. The visual inertial odometry module is also used to obtain inertial constraints based on the error between the measured value and the state estimate of the inertial measurement unit within the sliding window, to obtain visual constraints based on the reprojection error of the feature points within the sliding window, and to obtain prior constraints based on the old frame that has been removed from the sliding window. The visual inertial odometry module is also used to optimize the objective function based on the visual error, inertial measurement unit error, and prior error as constraints, and obtain the optimal pose estimate when the overall error is minimized.

Images