Inertial vision positioning method and system for mobile robot in weak texture dark light environment

Abstract

The invention discloses an inertial vision positioning method and system for a mobile robot in a weak texture dark light environment. The method comprises the following steps: weak light image enhancement; extracting feature information for matching and performing IMU pre-integration to obtain a pose relation of adjacent key frames; visual initialization and visual inertia joint initialization are carried out, and a point-line feature and inertia error model is established; and constructing a system residual model based on a sliding window method to carry out local optimization, and outputting position and attitude information. According to the method, the problem that the feature discrimination and uniqueness of the mobile robot in the weak light environment are remarkably reduced is effectively solved, and the positioning precision and robustness in the weak texture and dark light environment are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of navigation technology, and in particular to an inertial vision positioning method and system for a mobile robot in a weak-texture dark-light environment. Background Art

[0002] With the rapid advancement of technology, mobile robots have been widely used in various fields and show great potential for future development. The key to achieving autonomous navigation for mobile robots lies in being able to achieve positioning without relying on Global Navigation Satellite System (GNSS) signals. However, some mobile robots often operate in environments with weak textures and low light levels, which limits the performance of visual odometry in these scenarios. For example, when warehouse robots operate in large logistics warehouses, shelf surfaces are often single-colored or have regularly arranged textures, and the floors are typically smooth industrial floors, lacking sufficient visual features for positioning. In underground parking lots, insufficient lighting and uniform wall materials make it difficult for cameras to capture clear feature points. In wilderness or desert environments, the monotonous terrain and sparse textures also lead to insufficient features. These harsh imaging conditions in weak texture and low light levels result in poor image quality from the camera output, resulting in a small number of feature points of a single type, and a sharp decline in visual odometry positioning accuracy. Furthermore, visual odometry is highly sensitive to image blur caused by rapid camera motion and suffers from poor short-term stability. For example, heading angle divergence can occur when turning quickly around corners. Summary of the Invention

[0003] Purpose of the invention: In response to the deficiencies in the current technical field, the present invention proposes an inertial vision method and system for mobile robots in a weak-texture dark-light environment, aiming to achieve high-precision visual navigation and positioning of mobile robots under dark-light conditions using the assistance of low-cost inertial navigation.

[0004] Technical solution: In order to achieve the above invention objectives, the present invention adopts the following technical solution:

[0005] A method for inertial vision positioning of a mobile robot in a weak-texture dark light environment includes the following steps:

[0006] Step S1, acquiring an image frame and determining the illumination value of the image frame. If the illumination value of the image frame is lower than a specified threshold, the image is enhanced and the process proceeds to step S2; otherwise, the process proceeds directly to step S2;

[0007] Step S2: extracting point features using the ORB algorithm based on the image frame, extracting line features using the improved EDLines algorithm, matching point and line features using the Hamming distance based on the reference image, and obtaining an initial pose estimate using the PnP method if the match is successful. The improved EDLines algorithm filters anchor points based on gradient directional filtering, performs line segment fitting on the filtered anchor points using an iterative weighted method, merges similar line segments, and dynamically adjusts the pixel chain threshold of the EDLines algorithm based on the tracking of the point features; pre-integrates the inertial measurement unit (IMU) measurement data to obtain the pose relationship between adjacent key frames; and performs visual initialization and visual-inertial joint initialization.

[0008] In step S3, the visual-inertial information is fused in a tightly coupled manner based on a sliding window, a system residual model is constructed for local optimization, and position and attitude information is output.

[0009] An inertial vision positioning system for a mobile robot in a weak-texture dark-light environment, comprising:

[0010] The image preprocessing module is used to obtain image frames and judge the illumination value of the image frames. If the illumination value of the image frame is lower than the specified threshold, the image is enhanced and then enters the front-end processing module; otherwise, it directly enters the front-end processing module;

[0011] A front-end processing module is configured to extract point features based on image frames using the ORB algorithm, extract line features using the improved EDLines algorithm, match point and line features using the Hamming distance based on a reference image, and obtain an initial pose estimate using the PnP method if a match is successful. The improved EDLines algorithm filters anchor points based on gradient directional filtering, performs line segment fitting on the filtered anchor points using an iterative weighted method, merges similar line segments, and dynamically adjusts the pixel chain threshold of the EDLines algorithm based on the tracking of point features. The module also pre-integrates IMU measurement data to obtain the pose relationship between adjacent key frames, and performs visual initialization and visual-inertial joint initialization.

[0012] The back-end optimization module is used to fuse visual and inertial information in a tightly coupled sliding window manner, build a system residual model for local optimization, and output position and attitude information.

[0013] The present invention also provides a computer device comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the inertial vision positioning method for a mobile robot in a weak-texture dark light environment as described above are implemented.

[0014] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the inertial vision positioning method for a mobile robot in a weak-texture dark light environment as described above are implemented.

[0015] The beneficial effects of the present invention are as follows: visual positioning and navigation technology is widely used in the field of robotics, but weak texture and low light environments will lead to insufficient information collected by visual sensors, making it difficult to meet the demand for precise positioning. The present invention addresses the problem of reduced positioning accuracy due to insufficient feature points in weak texture environments, and proposes a visual odometry framework that combines point and line features. By improving the EDLines algorithm, gradient filtering and iterative weighting methods are introduced to improve the accuracy of line feature extraction; in response to the problem of reduced positioning accuracy due to loss of image details in dark light environments, a Zero-DyDSCU (Zero-DynamicDepth wise Separable Convolutions U-net) image preprocessing module is designed to improve the quality of input images; in response to the problem that pure visual odometry has a high dependence on environmental information and poor short-term stability, the present invention improves the positioning accuracy of the system in weak texture dark light environments by fusing inertial measurement unit (IMU) information. The present invention effectively overcomes the problem of significantly reduced feature discriminability and uniqueness of mobile robots in weak light environments, and improves the accuracy of scene matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is the overall flow chart of the method of the present invention;

[0017] Figure 2 This is a schematic diagram of the U-net network architecture;

[0018] Figure 3 This is the DSCU-net framework diagram;

[0019] Figure 4 This is the framework diagram of DyPL-VO;

[0020] Figure 5 It is the ineVec data structure;

[0021] Figure 6 This is the line feature matching result map;

[0022] Figure 7 Schematic diagram of sliding window algorithm;

[0023] Figure 8 This is a trajectory diagram of indoor weak texture experiment in a verification experiment;

[0024] Figure 9 This is a schematic diagram of the extraction position in a verification experiment;

[0025] Figure 10This is a verification experiment for the feature extraction effect;

[0026] Figure 11 This is the effect of extracting some point and line features in a verification experiment;

[0027] Figure 12 This is the trajectory and error diagram in a verification experiment;

[0028] Figure 13 is the heading angle and heading error in a verification experiment;

[0029] Figure 14 This is a schematic diagram of the indoor trajectory route in another verification experiment;

[0030] Figure 15 This is the trajectory diagram of the indoor dark light scene results in another verification experiment;

[0031] Figure 16 This is the indoor feature extraction situation in another verification experiment;

[0032] Figure 17 This is a graph showing the GPU usage in another verification experiment;

[0033] Figure 18 This is the outdoor scene trajectory and error map in another verification experiment;

[0034] Figure 19 This is the outdoor feature extraction situation in another verification experiment. DETAILED DESCRIPTION

[0035] In order to have a clearer understanding of the features and advantages of the technical solution of the present invention, the composition and implementation of the specific solution are explained below with reference to the accompanying drawings.

[0036] The present invention provides an inertial vision method and system for a mobile robot in a weak-texture dark light environment, aiming to achieve high-precision visual navigation and positioning of the mobile robot under dark light conditions using low-cost inertial navigation assistance. Figure 1The figure shows the overall flow chart of the method, which mainly includes three stages: image preprocessing, deep feature extraction and matching, and error modeling and optimization. Image preprocessing: In low-light environments (such as mines or dimly lit rooms), image contrast decreases, noise increases, and details become blurred, seriously affecting the robustness and positioning accuracy of the system. To improve the accuracy and real-time performance of image matching, this paper proposes an image enhancement model suitable for visual odometry, Zero-DSCU (Zero-Depthwise Separable Convolutions U-net). Based on the Zero-DCE theory, the U-net architecture and the depthwise separable convolution (DSC) module are introduced to improve model performance and real-time performance. Finally, a dynamic switching mechanism based on environmental perception is introduced to reduce computational redundancy, and the image enhancement preprocessing module Zero-DyDSCU (Zero-Dynamic Depthwise Separable Convolutions U-net) is designed for different scenarios. In the deep feature extraction and matching stage, to improve feature reliability and computational efficiency in weakly textured scenes, this paper proposes a dynamic point-line-visual odometer (DyPL-VO) based on the ORB-SLAM2 framework. This method retains the ORB-SLAM2 visual odometry backend architecture while reconstructing its front-end feature processing. First, line features are introduced based on the ORB point features. The EDLines algorithm is improved, using gradient filtering and iterative weighted optimization to improve the accuracy of the geometric description of line features, addressing the limitations of EDLines in complex edge detection. Secondly, a line feature approximation merging mechanism is used to address breakage and duplication issues. Finally, a dynamic thresholding strategy is introduced to dynamically adjust the line feature extraction threshold based on the point feature tracking status, suppressing redundant computations and achieving a coordinated optimization of feature quality and efficiency, thereby improving computational efficiency. In the error modeling and optimization stage, while the IMU has long-term cumulative errors, it can provide short-term high-precision pose data. Although the visual odometry has high positioning accuracy, it can suffer significant positioning deviations when the input image quality is poor, such as when rapid motion causes image blur. The high-precision pose information provided by the IMU can constrain this deviation. The present invention adopts a sliding window-based tight coupling method to fuse visual-inertial information, builds a graph optimization framework based on the G2O library (General Graph Optimization), performs optimization within a time window, and marginalizes information outside the window.

[0037] The whole process of the present invention will be described in detail below.

[0038] Step 1: Perform image preprocessing and propose an image enhancement model, Zero-DSCU, suitable for visual odometry. Building on the Zero-DCE theory, the model incorporates a U-net architecture and a depthwise separable convolution (DSC) module to improve performance and real-time capabilities. Finally, a dynamic switching mechanism based on environmental awareness is introduced to reduce computational redundancy, resulting in the design of the Zero-DyDSCU image enhancement preprocessing module, suitable for various scenarios.

[0039] (1) Light enhancement curve

[0040] The parameters of the light enhancement curve depend solely on the input image, automatically mapping the input image to the corresponding enhanced image. The curve considers three principles: 1) Boundedness: Ensures that the normalized pixel values are within [0, 1] to prevent overflow or truncation; 2) Monotonicity: Maintains monotonicity during image enhancement, ensuring that the relative size of pixel values in the enhanced image is consistent with that of the original image; 3) Differentiability: The curve is differentiable during gradient backpropagation, ensuring the stability and efficiency of the computational process. Therefore, to meet these three design principles, the quadratic curve equation is adopted:

[0041] LE(I(x);α)=I(x)+αI(x)(1-I(x)) (1)

[0042] Where x represents the pixel coordinate, I(x) is the input image, LE(I(x);α) is the enhanced image, and α is a trainable parameter of the curve.

[0043] The quadratic curve equation LE(I(x); α) satisfies the three principles mentioned above. The input image I(x) is mapped to the curve with parameter α to produce the output image LE(I(x); α). To further improve the generalization ability of the curve, the curve is iteratively adjusted to a higher-order curve to increase the dynamic adjustment range:

[0044] LE n (x) = LE n-1 (x)+α n LE n-1 (x)(1-LE n-1 (x)) (2)

[0045] Where n is the number of iterations, which controls the curvature of the curve. Higher-order curves have stronger adjustment capabilities, that is, greater curvature.

[0046] The generalization ability of higher-order curves allows for more flexible image adjustments. However, this is still a global adjustment, which may result in over- or under-enhancement of local areas. Therefore, α is adjusted as a pixel-by-pixel parameter. That is, for each pixel of a given input image, there is a corresponding best-fit curve, which outputs the corresponding enhancement parameter based on the illumination value of the pixel. Therefore, the equation can be restated as:

[0047] LE n (x) = LE n-1 (x)+A n LE n-1 (x)(1-LE n-1 (x)) (3)

[0048] Among them, A n is a parameter map of the same size as the given image. Pixel-by-pixel enhancement means that when the input image has non-uniform illumination, it can not only enhance the low-illuminance areas but also suppress the overexposed areas.

[0049] (2) Light enhancement curve estimation network based on U-net

[0050] U-Net is the mainstream network structure in current image enhancement tasks. Figure 2 As shown in the figure, U-Net employs a "U"-shaped network architecture. Its encoder and decoder are symmetrically designed, so the entire network can be viewed as consisting of symmetrical downsampling and upsampling modules. The downsampling module consists of two 3x3 convolutional layers and a 2x2 max pooling layer to reduce the dimensionality of the feature map. Reluctant Unit (ReLU) activation is used after the convolutional layers to facilitate backpropagation and weight updates. The upsampling module, consisting of a 2x2 deconvolutional layer and two 3x3 convolutional layers, upsamples and increases the dimensionality of the feature map. U-Net's skip connection mechanism enables direct transmission of low-level features from the encoder to the decoder, helping to better restore image details. This mechanism is achieved by concatenating the feature map generated by the encoder with the feature map of the corresponding decoder layer. Furthermore, a 1x1 convolutional layer is added to the concatenated feature map to fuse the low-level and high-level features, thereby preserving more spatial information. This characteristic gives U-Net a significant advantage over other models in image enhancement.

[0051] Thanks to the encoding and decoding structure and skip connection mechanism of U-net, the network has strong learning ability and can handle pixel-level tasks. The core task of the image enhancement model is to estimate the enhancement curve of each pixel, which is consistent with the characteristics of U-net. Therefore, in order to reasonably estimate the pixel-level transformation curve, the present invention designs a deep network DSCU-net based on the U-net architecture to estimate the pixel-by-pixel transformation curve. The network structure diagram is shown in the figure. Figure 3As shown in Figure 2. The encoder consists of three convolutional layers and a downsampling layer, and the decoder consists of three upsampling layers. They perform jump splicing operations to suppress gradient vanishing and preserve image details. The Tanh activation function is used to generate the output. The main reason for using the Tanh activation function is that according to Equation (1), the network output should be limited to the range of [-1, 1] to avoid excessive deformation of the enhancement curve, which may lead to excessive image enhancement or insufficient brightness. The convolutional layers in the DSCU-net architecture use depthwise separable convolution modules instead of traditional convolutions.

[0052] (3) Lightweight implementation based on depthwise separable convolution

[0053] In order to reduce the number of model parameters, the depthwise separable convolution module is introduced to replace the traditional convolution in DSCU-net for lightweight implementation. The basic idea of the depthwise separable convolution module is to split the convolution process into two independent sub-operations: depthwise convolution and pointwise convolution, so as to reduce the computational complexity of the model and the number of parameters while ensuring that the performance of the model does not deteriorate significantly. Its principle is as follows Figure 3 As shown:

[0054] The standard convolutional layer performs feature extraction and channel fusion simultaneously, and the parameter q1 is:

[0055] q1=D k ×D k ×C×N (4)

[0056] Among them D k is the convolution kernel size; C is the number of input channels; N is the number of output channels. The computational effort p1 is:

[0057] p1=D k ×D k ×C×N×H×W (5)

[0058] Where: H and W are the height and width of the output feature map respectively.

[0059] DSC decomposes the standard convolution process into feature extraction and channel fusion. In the feature extraction stage, each channel is independently convolved in depth. In the channel fusion stage, all channels are convolved point by point using a 1×1 convolution kernel. The parameter q2 is:

[0060] q2=D k ×D k ×C+1×1×C×N (6)

[0061] The calculation amount p2 is:

[0062] p2=D k ×Dk ×C×H×W+1×1×C×N×H×W (7)

[0063] Therefore, the comparison between the two in terms of parameter quantity and computational complexity is:

[0064]

[0065] As can be seen from the comparison, DSC uses far less computation than standard convolution. The introduction of the DSC module helps improve computational efficiency and achieve model lightweighting. Furthermore, because depthwise separable convolution uses fewer parameters for feature extraction and reduces information interleaving during convolution operations, it effectively prevents overfitting and improves model generalization.

[0066] (4) Dynamic switching strategy based on environmental perception

[0067] To further improve computational efficiency, the present invention uses a dynamic switching strategy based on environmental perception to dynamically control the image enhancement module. Using a neural network to enhance every input frame is unnecessary and redundant. A potential solution is "enhancement on demand," automatically turning on the network when the system is in poor condition and off when it is in good condition. To achieve this goal, the present invention first uses an illumination evaluation metric to measure the illumination of the input frame and then designs a switching strategy.

[0068] The average grayscale value of the frame is used to measure the illumination of the image. The calculation formula is as follows:

[0069]

[0070] Where x ij is the grayscale value of the i-th row and j-th column, H is the image height, W is the image width, and M is the image's average grayscale. When the average grayscale value of an image frame is less than 20, it is considered a dark scene. Therefore, a threshold of 20 is set. If the image's illumination index is less than the given threshold, the input image is enhanced by the image enhancement module before being fed into the algorithm's front-end for feature extraction. If it is greater than the threshold, feature extraction proceeds directly from the image.

[0071] Step 2: In the deep feature extraction and matching stage: To improve the feature reliability and computational efficiency in weak texture scenes, the present invention proposes a dynamic point-line fusion visual odometry DyPL-VO (Dynamic Point Line-Visual Odometer) based on the ORB-SLAM2 framework, retains the visual odometry backend architecture of ORB-SLAM2, and reconstructs its front-end feature processing module: first, line features are introduced based on ORB point features. The EDLines algorithm is selected for improvement, and the accuracy of the geometric description of line features is improved through gradient filtering and iterative weighted optimization to solve the problem of EDLines being limited in complex edge detection; secondly, the line feature approximation merging mechanism is used to solve the problems of breakage and repetition; finally, a dynamic threshold strategy is introduced to dynamically adjust the line feature extraction threshold according to the point feature tracking status, suppress redundant calculations, achieve coordinated optimization of feature quality and efficiency, and improve computational efficiency.

[0072] (1) DyPL-VO framework design

[0073] The binocular visual odometry that integrates improved line features and point features and introduces a dynamic threshold strategy is named DyPL-VO (Dynamic Point Line-Visual Odometer). The implementation framework of DyPL-VO is as follows: Figure 4 As shown in the figure, the initialization phase acquires data from the binoculars. Before calculating the trajectory, camera calibration is performed. This step corrects the inherent distortion in the camera lens and ensures accurate distance measurement in the captured image. Next, the initialization phase extracts key features from the initial image frames. These features, such as corners or points of high contrast, are used to enable environment tracking and mapping. By analyzing the motion of these features in subsequent frames, the system determines the initial camera pose.

[0074] After initialization, the point feature algorithm uses the ORB feature method to extract feature information, while the line feature algorithm dynamically adjusts the threshold based on the tracking of the point features to retain high-quality line feature information. Both point and line features are matched using the Hamming distance, selecting the matching pair with the minimum distance. After preliminary matching, RANSAC is used to filter feature pairs that meet geometric consistency and further eliminate false matches. If a match is successful, the PnP (Perspective n-Point) method is used to obtain a pose estimate.

[0075] Local optimization is then performed, refining the initial pose by tracking the local map. To ensure the real-time efficiency of the tracking thread, map points are set as fixed nodes during optimization, and only the pose of the latest frame is used as an optimization variable. Unqualified map points are filtered out using the results of single-frame optimization. The main task of the local mapping thread is to simultaneously optimize keyframes and map points in the map, using local bundle adjustment to further improve map accuracy and stability.

[0076] (2) LBD descriptor and Hamming matching

[0077] The extracted line features are described using the Local Binary Descriptor (LBD) algorithm. The LBD algorithm represents image features locally, encoding the spatial structure of these features by analyzing the pixel relationships in the area surrounding the line segment. It divides the image into a series of small circular or rectangular regions and analyzes the intensity variations and spatial relationships of the pixels within each region to generate a unique binary code.

[0078] For each pixel in the region, the LBD descriptor compares the intensity of the pixel with the intensity of the surrounding pixels and encodes the comparison result into binary bits. Each local region generates a unique binary code sequence as the descriptor of the region. In the present invention, the local region is a rectangular area, which is further divided into sub-regions (or "bands") of width w. The structure of each band is represented as a binary code sequence, which is formally defined as follows:

[0079]

[0080] in, and are the mean vector and standard deviation vector with m, respectively. To meet real-time requirements, the algorithm uses the BinaryDescriptor method in OpenCV to convert the 72-dimensional LBD descriptor into a 256-bit binary LBD. This is achieved by comparing each pair of values in the descriptor: if the i-th element is greater than the j-th element, the corresponding bit is set to 1; otherwise, it is set to 0. This binary encoding provides a compact and efficient method for representing line features.

[0081] When matching, first define a data structure called "LineVec", such as Figure 5 As shown in Figure 2, LineVec is a set of representations of a line in space at different scales in the image pyramid. When matching the two-dimensional line features of the query image and the reference image, we first use the LineVecs direction histogram (h r ,h q ) to estimate its overall rotation angle θ g , specifically expressed as:

[0082]

[0083] Among them, the subscripts r and q represent the reference image and query image, h r (x)-h q(x-θ) is the distance of the direction histogram after translation, which reflects the similarity between the two images under the rotation angle θ. If the direction angle ɑ between the two LineVecs and the minimum descriptor distance s meet the following conditions:

[0084] |α-θ g | <t θ And s <t s (13)

[0085] in, And t s =0.35. Then this pair of features can be considered as a candidate matching pair.

[0086] For the matching point pairs after rotation correction, the Hamming distance is further used to calculate the matching degree between binary descriptors. If the direction angle Δφ between the two LineVecs and the minimum Hamming distance d min The following conditions are met:

[0087]

[0088] Where, τ φ and τ d are the thresholds of direction angle and Hamming distance respectively, then the pair of features can be regarded as a candidate matching pair.

[0089] After establishing the candidate matching relationships, the adjacency matrix A containing the matching relationships can be constructed, where the matrix elements A ij Represents the matching similarity between candidate matching pairs. By maximizing the sum of the elements of the adjacency matrix, the best matching relationship between images can be determined. Line feature matching is performed using the Euroc dataset, and some results are shown below. Figure 6 As shown, the green line is the extracted line feature, and the red line connects the successfully matched line feature matching pairs.

[0090] (3) Anchor point selection based on gradient direction filtering

[0091] Gradient-directed filtering can smooth the image to a certain extent and suppress noise interference, thereby optimizing the anchor point selection process and improving overall stability and robustness.

[0092] In general, the absolute difference in the gradient direction of any two anchor points on the same line is within a certain range. If it exceeds this range, the point is considered an outlier or nonlinear point. To mitigate the impact of these points, the MAD (median absolute deviation) algorithm is introduced to remove outliers. The algorithm execution process is detailed as follows:

[0093] 1. Calculate the gradient values of all anchor points on the fitted line, sort them from large to small, and record the median value as G median .

[0094] 2. Calculate the absolute deviation between all anchor points and the median value according to formula (15), where G i Represents the gradient value of anchor point i:

[0095] σ i =|G i -G median | (15)

[0096] 3. Sort the absolute deviations from small to large to get the median of the absolute deviations G. MAD .

[0097] 4. Determine the parameter p and retain G i In the original estimate, G i Satisfying formula (16):

[0098] G mediam -p×G MAD <G i <G mediam +p×G MAD (16)

[0099] Based on the Lyddane–Shindo criterion, the parameter p is typically set to 3. Points whose gradient directions exceed the specified threshold are identified as nonlinear points and automatically removed to maintain the accuracy of the subsequent linear fit. This step is iterated until all anchor points in the image have been screened.

[0100] (4) Segment fitting optimization based on iterative weighting

[0101] In the EDLines method, the filtered anchor points are typically fitted using standard least-squares techniques. However, the least-squares method assigns equal weights to all points, ignoring their distance from the fitted line. While this method is computationally fast, it suffers from poor robustness and often results in suboptimal line segment parameters. To address these limitations, the IRLS (Iterative Reweighted Least Squares) method was introduced. The IRLS method enhances the accuracy and robustness of the fitting process by assigning different weights to points based on their distance from the fitted line, with points at greater distances receiving smaller weights. Furthermore, IRLS sets appropriate iterative termination criteria to conserve computational resources while maintaining fitting accuracy.

[0102] The IRLS method assigns a weight to each point according to its distance to the line, minimizing the parameter β = (β1, β2, ..., β n ) TDetermined by iteration. In each step, the following weighted least squares problem needs to be solved:

[0103]

[0104] Where L(k,b) is the linear estimation parameter at the t+1th iteration; W i (β (t) ) is the corresponding weight of each point; k represents the slope; n is the intercept.

[0105] Define the distance from a point to a line Variables as weight functions:

[0106]

[0107] There are many forms of weight functions, including Tukey, Geman-McClure, and Huber. The Huber weight function ensures low computational complexity while effectively suppressing the influence of outliers. Therefore, in this invention, the weight function corresponding to each point adopts the Huber weight function, which is defined as follows:

[0108]

[0109] Where u is the distance threshold. Next, we take the partial derivatives of k and b and obtain the following equations (20) and (21):

[0110]

[0111] Initially, all weights are set to 1, and equation (17) is substituted into equation (21), the weights are updated, and the slope k and intercept b are solved by multiple iterations. j and k j The expression of can be expressed by equations (22) and (23).

[0112]

[0113]

[0114] When the convergence criterion is met or the maximum number of iterations is reached, the iteration stops, and the final linear parameters are determined. The convergence conditions described can be summarized as follows (24):

[0115]

[0116] in is the i-th classification of the linear parameter quantity in the t-th iteration. When the convergence value is lower than the preset threshold ε, the algorithm is considered to have converged and the termination iteration can be extracted to reduce the computation time.

[0117] (5) Approximate line segment merging mechanism

[0118] For the initially extracted long line features, all line segments are grouped starting from the longest line segment. The longest line segment is chosen because longer line segments usually originate from image regions with continuous strong gradients, which makes them more reliable. Here, the line feature l is defined as i The two endpoints are (A i ,B i ), their coordinates are and The length of the line segment is len i , the angle with the horizontal direction is θ i ∈[0,π]. The algorithm uses three threshold conditions to group line segments and merge similar line segments. To illustrate this process, the following example uses a line segment set. Define the initial line segment set L1.

[0119] 1) Angle threshold filtering: Determine the angle θ1 of line segment l1 and any other line segment l i Angle θ i If the angle difference is less than the angle threshold θ min , then l i Add to the line segment set L2:

[0120]

[0121] 2) Endpoint-to-segment distance threshold filtering: Based on the angle restriction, the line segments are further filtered. In order to merge some redundantly extracted line features, the algorithm introduces an endpoint-to-segment distance threshold. This threshold calculates the distance from the endpoint of the line segment to other line segments in the set and determines whether the distance is less than the endpoint-to-segment distance threshold dep minf :

[0122]

[0123] in and is the distance from the endpoint to the line.

[0124] 3) Endpoint-to-endpoint distance threshold filtering: Experiments show that the two aforementioned filtering conditions can effectively merge most similar segments. However, some broken segments cannot be merged by these two conditions. To address this issue, the algorithm adds a third filtering condition. This condition calculates the distance between the four endpoints of two segments, selects the smallest distance, and determines whether the distance is less than the threshold dep. minpp If the condition is met, the line segment is added to the final set L4 for merging:

[0125]

[0126] 4) After the three threshold conditions are met, a set of line segments to be fitted is obtained. The algorithm uses a line segment fitting function to fit these segments into a straight line. Because the fitted segments extend infinitely in space, their endpoints must be determined. Depending on the angle of the fitted segment, either its horizontal or vertical coordinates can be used to determine the endpoints. If the angle of the fitted segment is less than 45°, the horizontal coordinates are used to determine the endpoints; if the angle is greater than 45°, the vertical coordinates are used to determine the endpoints. This ensures that the fitted segments are both long and accurate.

[0127] (6) Dynamic threshold strategy based on point feature tracking status

[0128] In texture-rich scenes, point features can fully describe the scene structure, thus supporting visual odometry pose calculation. However, in weakly textured scenes, since point features are difficult to track stably, appropriately increasing the extraction of line features can effectively improve positioning accuracy. In the EDLines algorithm, only pixel chains longer than L are identified as line segments, triggering the least squares method. To further optimize the improved EDLines algorithm, making it flexible and adaptable to different environments and reducing computational redundancy, a dynamic threshold strategy is proposed: the threshold L is dynamically adjusted based on the tracking of point features. In areas with rich point features, the introduction of line features is reduced to avoid unnecessary computation. In weakly textured scenes, the threshold L is appropriately lowered to enhance the participation of line features, thus achieving a balance between efficiency and accuracy. The details are as follows.

[0129] For point features, larger tracking error regions usually correspond to tracking losses caused by low texture or fast motion. Therefore, the tracking failure rate D is used p To represent the tracking status of point features. For the i-th frame image, It is defined as the average of the feature point tracking failure rates in the first ten frames of images. The specific calculation is as follows:

[0130]

[0131] in, is the number of feature points tracked in the jth frame; is the number of feature points continuously tracked in the jth frame. The value indicates that the proportion of feature points continuously tracked in the current frame is low, that is, the point feature has a high tracking failure rate, and line feature information should be added for constraint. As an evaluation indicator, the threshold L is dynamically adjusted to dynamically adjust the extraction of line features according to the quality of point feature tracking. p It is linearly mapped into the calculation formula of L:

[0132]

[0133] in, is a value between 0 and 1; L min is the lower limit of the line segment length. Too short a line segment may introduce unnecessary errors. max is the maximum threshold for long line segments, used for upper limit control and adjusted according to the system's working environment. On this basis, all detected line segments are sorted by length and the first N line segments are selected for matching.

[0134] Through the above dynamic threshold strategy, the improved EDLines algorithm can flexibly adjust the extraction of line features according to the tracking status of point features, thereby better adapting to the needs of different scenarios.

[0135] Step 3: During the error modeling and optimization phase: While the IMU has long-term cumulative errors, it can provide short-term, high-precision attitude data. Although the visual odometry has high positioning accuracy, it can exhibit significant positioning deviations when the input image quality is poor, such as when rapid motion causes image blur. The high-precision attitude information provided by the IMU can constrain this deviation. This paper fuses visual-inertial information using a tightly coupled sliding window approach, constructing a graph optimization framework based on the G2O library (General Graph Optimization), performing optimization within a time window and marginalizing information outside the window.

[0136] (1) Visual IMU joint initialization model

[0137] When a visual-inertial odometry system fuses visual and IMU sensor data, the initial values have a direct impact on the system's accuracy. Therefore, both visual initialization and combined visual-inertial initialization are necessary to obtain relatively accurate initial values and parameters for subsequent optimization. The following describes both pure visual initialization and combined visual-inertial initialization.

[0138] Pure vision initialization relies on the structure from motion (SFM) method to obtain pose estimates. It uses a sliding window to obtain a set of pose frames that observe the same feature point, and then establishes constraints between these sets to solve the position and pose of all frames. The main steps are as follows:

[0139] Step i: Feature extraction and matching: First, visual features are extracted from the image. These features represent key information points in the image. These features are then matched to find corresponding feature points in different images, which forms the basis for subsequent processing.

[0140] Step ii: Depth Information Acquisition and 3D Coordinate Calculation: After matching feature points, the depth information of the point features is obtained. Depth information can convert feature points in the 2D image into 3D coordinates, providing richer information for subsequent pose calculations.

[0141] Step iii: Optimization and Bundle Adjustment: Optimize the camera's position and pose, as well as the 3D coordinates of the feature points. For the image frames within the sliding window, calculate their positions and poses. Bundle adjustment is used here to minimize the reprojection error of all features.

[0142] The first frame image is used as the reference coordinate system c0. The poses of other frames are based on the changes of c0. At the same time, the pose information of the visual coordinate system and the IMU coordinate system needs to be converted, and the equation can be obtained:

[0143]

[0144] Among them, s represents the scale, Represents the rotation quaternion between coordinates. The external parameters between the camera and IMU are (p bc ,q bc ), the poses of all frames are represents the rotation matrix, Represents the multiplication of two quaternions.

[0145] In visual-inertial joint initialization, since the sampling frequencies of the camera and IMU are different, the first thing to solve is to align the image with the IMU pre-integrated data. This is done by matching the visual and IMU pre-integrated data without restoring the scale.

[0146] The gyroscope bias is estimated based on adjacent image keyframes and then linearized with the IMU pre-integration residual. The attitude-related estimation information is obtained through visual initialization. The cost function is constructed and minimized to obtain:

[0147]

[0148] Among them, B is the set of all key frames, Represents the first-order approximation of the deviation, and all pre-integration terms are recalculated according to the calculation results of equations (32) and (33). After completing the gyroscope bias initialization, the velocity, gravity vector and scale factor are initialized, considering the vector:

[0149]

[0150] in, Indicates the speed of the nth frame image in the world coordinate system, Represents the gravity vector. According to IMU pre-integration, we can get:

[0151]

[0152] It can be seen that the linear measurement model includes IMU measurements and visual measurements:

[0153]

[0154] Among them, χ I is the state variable to be estimated,

[0155]

[0156] in It is obtained by visual initialization, Δt is the time interval, and the velocity, gravity and scale factor can be obtained by solving the least squares problem:

[0157]

[0158] To minimize the error, the gravity acceleration result is refined based on the known gravity acceleration. By introducing the tangent space to optimize the two-dimensional error state, the solution process is simplified and the vector is redefined as shown in Equation (40):

[0159]

[0160] in, The direction of gravity points to the center of the earth; ‖g‖ is the magnitude of gravity; are two orthogonal bases of the tangent plane; ω1, ω2 are the variables to be optimized.

[0161] After gravity refinement is completed, the Z axis is aligned with the direction of gravity acceleration, and all variables are rotated to the world coordinate system. After the initialization work is completed, it is fed back to the backend for fusion.

[0162] (2) Sliding Window Marginalization Strategy

[0163] This paper builds a graph optimization framework based on the G2O library (General Graph Optimization) and uses a sliding window algorithm to implement a tightly coupled fusion model. It optimizes within a time window and marginalizes information outside the window. The specific strategy is as follows: if the penultimate frame of the sliding window (the frame before the last frame) is a key frame, then the key frame is retained and the earliest frame in the window and its related information are marginalized, such as Figure 7 As shown in (a); if the penultimate frame is a non-key frame, the relevant feature information is discarded and only the inertial navigation measurement data is retained, as shown in Figure 7The purpose of adopting this marginalization strategy is to ensure that the key frames obtained have sufficient parallax, avoid computational redundancy, and maintain the validity of accelerometer measurements.

[0164] (3) Vision IMU Tightly Coupled Model

[0165] IMUs have long-term cumulative errors, but they can provide short-term high-precision attitude data. Although visual odometry has high positioning accuracy, when the input image quality is poor, such as when rapid motion causes image blur, positioning will show significant deviations. In this case, the high-precision attitude information provided by the IMU can constrain this deviation. The present invention adopts a tight coupling method based on sliding windows to fuse visual-inertial information. The state vector to be optimized, χ, is expressed by equation (41):

[0166] χ=[x0,x1,…x n ,x bc ,λ0,λ1,…,λ m ,l0,l1,…l q ] (41)

[0167]

[0168] x bc =[p bc ,q bc ]

[0169] Among them, x k represents the IMU state corresponding to the kth frame, l is the orthogonal expression of the line feature, n represents the total number of key frames, x bc represents the relative pose between vision and IMU, λ represents the inverse depth of feature points, m represents the number of feature points, P, v, q represents the position, velocity and attitude information at the corresponding moment, b a and b g Represents zero bias. A tightly coupled optimization model is established, and the cost function of the system is obtained as:

[0170]

[0171] in are the IMU residuals and point-line reprojection residuals, respectively. ρ() is the Huber robust kernel function. B is the set of all pre-integration constraints within the sliding window, and F and L are the point-line features extracted within the sliding window, respectively. p -J p χ is the prior information obtained by marginalizing the keyframes, that is, when a new keyframe is inserted, the information of the oldest frame will be removed but the relevant constraints will remain.

[0172] In order to verify the performance of the method of the present invention, the inventors conducted experimental tests.

[0173] The first part tests the impact of point-line feature fusion on positioning accuracy in weak texture scenes, and the second part tests the impact of the image enhancement module on DyPL-VIO positioning accuracy in dark light environments.

[0174] 1) Experiments and analysis in weak texture environment

[0175] This experiment tested the positioning accuracy of DyPL-VIO in a realistic weak-texture scene. The experiment was divided into two groups: indoor and outdoor. The indoor experiment consisted of an L-shaped corridor with a total length of 50 meters. Labels were affixed to the corners, and the remote-controlled car was controlled to turn by passing through the labels. The corridor walls were largely white, and the floor was made of honed crystal stone tiles, creating a large area with simple and repetitive textures. Furthermore, the long corridor had sunlight entering at its end. The outdoor experiment consisted of a Z-shaped route, using Novate as the benchmark. The route was 284.8922 meters long and empty. With the exception of a few scenes, the route was mostly greenery and concrete, meeting the weak-texture criteria.

[0176] The comparison algorithms used were the binocular camera mode of ORB-SLAM2 and the DyPL-VO algorithm proposed in this paper. The algorithm details are shown in Table 1. The ORB-SLAM2 algorithm, as a comparison method for pure point feature visual odometry, does not introduce line features or inertial data. It is used to quantify the effect of line features on improving positioning accuracy in weakly textured scenes. The inertial-free point-line feature fusion DyPL-VO integrates improved line feature extraction and a dynamic thresholding strategy based on ORB-SLAM2 to verify the improvement in positioning accuracy achieved by point-line feature fusion. The inertial-based point-line feature fusion DyPL-VIO further integrates IMU pre-integration and tight coupling optimization in DyPL-VO to evaluate the optimization effect of inertial data on short-term stability and cumulative error suppression in dynamic scenes (such as turns and rapid motion).

[0177] Table 1 Algorithm details

[0178]

[0179] (1) Indoor experimental results

[0180] Positioning trajectory diagrams of ORB-SLAM2, DyPL-VO and DyPL-VIO, such as Figure 8 As shown in the figure, the total length of the trajectory is 50m. The starting coordinates are (0, 0) and the ending coordinates are (39, 11). The red trajectory is DyPL-VIO, the blue trajectory is ORB-SLAM2, and the green trajectory is DyPL-VO.

[0181] like Figure 9As shown in the figure, the extraction effect of DyPL-VIO on point and line features at four locations (a, b, c and corners) is shown as follows: Figure 10 As shown, Figure 10 (a), (b), (c), and (d) correspond to the extraction results at position a, the corner, position b, and position c, respectively. Red dot features and purple line features indicate long tracking times and are more robust, while blue dot features and green line features indicate short tracking times. Features tracked for a long time have higher matching accuracy between consecutive frames. Because these features are repeatedly observed in multiple frames, their descriptors are more discriminative and matching ambiguity is lower, thus reducing the probability of mismatches and being more robust. Features tracked for a short time may be lost due to sudden changes in brightness or angle changes, resulting in computational redundancy and matching failures.

[0182] At the corner (0,11) meters, the robot needs to complete a quick turn. Figure 8 As shown, the trajectories of ORB-SLAM2 and DyPL-VO deviate slightly here. This is because the external light at the end of the corridor causes local overexposure, and the feature gradient response decreases significantly. Compared with the features extracted at position a ( Figure 10 (a)), corner ( Figure 10 (b)) The number of blue point features (short tracking time) has increased significantly, and the number of line features has decreased significantly, indicating that the quality of feature extraction decreases when the image is blurred, and the matching success rate decreases, which in turn leads to cumulative errors in the heading angle. In addition, the pure vision methods ORB-SLAM2 and DyPL-VO rely on historical feature tracking. The camera motion blur exacerbates feature degradation during rapid turns. Although the line feature (DyPL-VO) improves continuity through an approximate merging mechanism, its geometric constraints are still not enough to completely suppress heading drift. In contrast, DyPL-VIO shows significant advantages after fusing inertial data. The IMU angular velocity integration provides high-frequency heading angle increments, maintains the continuity of the pose solution during the period of visual feature failure, and suppresses the heading angle. And as Figure 10 As shown in (c)-(d), DyPL-VIO can still extract long line features in areas with uneven illumination. By optimizing the pose estimation through reprojection constraints, its end point position error (0.64%) is reduced by 79.4% and 30.4% compared with ORB-SLAM2 (3.10%) and DyPL-VO (0.92%), respectively. The position errors are shown in Table 2.

[0183] Table 2 Weak texture indoor experimental positioning results

[0184]

[0185] DyPL-VO significantly reduces positioning error in weakly textured scenes (70.3% lower than ORB-SLAM2) through spatial constraints of long line features; DyPL-VIO compensates for visual degradation through IMU and further reduces the error by 30.4% in dynamic scenes, verifying the positioning advantage of tightly coupled data.

[0186] (2) Outdoor experimental results

[0187] The experimental results are as follows Figure 11 As shown, it shows that in the case of rich texture ( Figure 11 (a)), weak texture ( Figure 11 (b)) and turning ( Figure 11 (c)) feature extraction effect (purple line features and red point features are more robust features with longer tracking time, green line features and blue point features are feature information with shorter tracking time). Figure 11 (b) ORB-SLAM2’s blue point features (short tracking time) account for >50%, indicating that the quality of point features has seriously degraded (compared to Figure 11 (a) Compared); DyPL-VO extracts purple line features through gradient filtering (long tracking time) and provides stable geometric constraints. Figure 11 (c)) The rapid movement of the camera causes image blur, and ORB-SLAM2 can hardly extract valid point features; the proportion of green line features extracted by DyPL-VO (short tracking time) increases, and line features alone are not enough to suppress heading angle divergence.

[0188] Figure 12 The trajectory and error diagram of the outdoor weak texture experiment, where (a) is the trajectory diagram, (b) is the trajectory diagram of the east and north directions, and (c) is the trajectory error diagram. The trajectory root mean square error results are summarized in Table 3.

[0189] Table 3 Absolute trajectory positioning error of weak texture outdoor experiment (m)

[0190]

[0191] from Figure 12 Comparing the trajectories of (a) and 12(b), we can see that the robot motion can be divided into three stages. The first is the stable stage (0s–35s), where the robot moves in a straight line and the environment is rich in texture. The trajectories of ORB-SLAM2, DyPL-VO, and DyPL-VIO are highly consistent with the benchmark, indicating that all three algorithms have reliable positioning capabilities in conventional scenarios. In the weak texture area (35s–175s), the robot trajectory gradually deviates. ORB-SLAM2 deviates due to the degradation of point feature quality ( Figure 12(b) The blue point features account for >50%), the maximum error is 13.826m; DyPL-VO provides additional geometric constraints through line features ( Figure 12 (b) purple line feature), the maximum error is suppressed to 7.2154m, which is 47.78% lower than ORB-SLAM2 (13.816m); DyPL-VIO further integrates IMU angular velocity integral, and in the period of visual feature degradation (such as Figure 12 (c) High-frequency inertial data was used to suppress heading drift during a turn, further reducing the maximum error (3.7140m), a 48.52% improvement over DyPL-VO. Finally, during the straight-line walking phase (175s to the end point), the robot resumed linear motion, but the accumulated error from the previous phase was not eliminated. The root mean square error (RMS) of DyPL-VIO (2.1021m) was significantly lower than that of ORB-SLAM2 (8.1779m) and DyPL-VO (4.3050m), demonstrating the advantages of integrating point-line feature fusion with tight inertial coupling.

[0192] Further analysis of the heading angle and heading error, such as Figure 13 As shown in Table 4.

[0193] Table 4. Root mean square error of heading in weak texture outdoor experiment (°)

[0194]

[0195] In the weak texture stage of 35–175s, the maximum instantaneous error of ORB-SLAM2 exceeds 50°, which directly leads to trajectory deviation (maximum error 13.826m). This is due to the degradation of point feature quality in the weak texture area ( Figure 11 (b)), while turning increases image blur, reduces feature matching success rate, and accumulates heading angle errors. DyPL-VO extracts long line features through gradient filtering, iterative weighting and line segment merging ( Figure 11 (b) Purple line features), provide stable geometric constraints in weak texture areas, suppressing the heading root mean square error to 15.254°, which is 50.4% lower than ORB-SLAM2. However, when turning ( Figure 11 (c) The increasing proportion of green line features (short tracking time) indicates insufficient feature robustness, resulting in a still-large heading error. DyPL-VIO integrates IMU angular velocity data and uses inertial observations to maintain heading angle solution continuity during periods of visual feature failure (turning phases), reducing the RMS heading error to 0.523°.

[0196] 2) Experiment and analysis in dark light environment

[0197] In order to verify the effect of the dark light enhancement module on the positioning accuracy of the DyPL-VIO algorithm in actual scenes, this experiment is divided into two experiments, indoor and outdoor. The indoor experiment is conducted in an indoor "L" shaped corridor with a total length of 54 meters. Figure 14 As shown, there is a corridor L. Section a is brightly lit, with the average grayscale values of the images in section a exceeding 50, meeting normal lighting conditions. Section b is unlit, with only three emergency lights on. The average grayscale values of the images in section b are below 20, meeting low-light conditions. The outdoor experiment involved walking along an isosceles trapezoidal route at night, with a total length of 311.7742 meters.

[0198] To verify the effectiveness and real-time performance of the Zero-DyDSCU enhancement module proposed in this paper, the following three algorithms were selected for comparison. First, DyPL-VIO (without an enhancement module) was used as a baseline method to quantify the independent contribution of the image enhancement module to the positioning performance of low-light scenes. Second, histogram equalization enhancement + DyPL-VIO (hereinafter referred to as HE+DyPL-VIO) was used as a representative method of traditional image enhancement. Its high computational efficiency can verify the performance boundary of traditional enhancement strategies in positioning tasks. Finally, Zero-DyDSCU+DyPL-VIO (the method of this invention) integrates a self-developed lightweight enhancement module into DyPL-VIO to evaluate the comprehensive advantages of the proposed method in enhancement quality, robustness, and computational efficiency.

[0199] (1) Indoor experimental results

[0200] The experimental results are as follows Figure 15 As shown. The starting coordinate is (0, 0) and the end coordinate is (27, -27). Figure 15 A comparison of the trajectories shows that in low-light conditions (section b), the DyPL-VIO without an integrated image enhancement module loses its track (black track) due to feature extraction failure. However, the HE+DyPL-VIO (blue track) and Zero-DyDSCU+DyPL-VIO (red track), both with integrated enhancement modules, maintain stable tracking. The HE+DyPL-VIO endpoint coordinates are (27.4571, -26.4165), with an offset of 0.6760m and a position error of 1.24%. The Zero-DyDSCU+DyPL-VIO endpoint coordinates are (26.9719, -26.5466), with an offset of 0.4609m and an error of 0.85%, a 31.4% improvement compared to the HE+DyPL-VIO. This difference stems from the fundamental differences in how the two enhancement modules optimize image quality.

[0201] Figure 16This figure demonstrates the performance of different methods for feature extraction in the dark environment of segment b. Features with long tracking times are marked in purple, while those with short tracking times are marked in green. Features tracked for long periods of time achieve higher matching accuracy between consecutive frames. Because these features are repeatedly observed across multiple frames, their descriptors are more discriminative and matching ambiguity is lower, reducing the probability of mismatches and increasing robustness. Features tracked for short periods of time may be lost due to sudden changes in brightness or angle, resulting in computational redundancy and matching failures.

[0202] Original image ( Figure 16 The first column (a) in the figure shows that the number of features extracted is small, mainly green features (short tracking time), indicating that the feature quality is seriously degraded in low light environments. This phenomenon is usually caused by low light conditions that result in insufficient contrast and detail information in the image, resulting in the inability to stably extract and track feature points. HE enhanced image ( Figure 16 The number of features extracted in the second column (b) of the image has increased, significantly improving the overall brightness of the image. However, due to noise interference, the extracted line features are often broken and few in number. This is because the HE algorithm causes overexposure in some areas and underexposure in other areas during global equalization processing, introducing additional noise and affecting the quality of feature extraction and matching. Zero-DyDSCU enhanced image ( Figure 16 The third column (c) in the figure shows even better performance in terms of the number of extracted features, with not only a further increase in the number but also a higher proportion of purple features (long tracking time). This demonstrates that Zero-DyDSCU effectively enhances image brightness while preserving image details and extracting more stable feature points. Specifically, through its encoding and decoding structure and skip connection mechanism, Zero-DyDSCU effectively estimates pixel-by-pixel light enhancement parameters, achieving adaptive enhancement and ensuring a balanced brightness boost across all image regions.

[0203] Analyze the impact of dynamic switching strategy on computing efficiency. Figure 17 As shown, the GPU usage of the two algorithms during operation. It can be seen that Zero-DyDSCU+DyPL-VIO only turns on the image enhancement module after entering section b, that is, 110s, and the GPU usage increases. It is not turned on under normal light conditions. HE+DyPL-VIO, on the other hand, turns on the enhancement module throughout the entire process. The enhancement time of the HE algorithm is significantly lower than that of Zero-DyDSCU. Because of the introduction of the dynamic switching strategy, the final average GPU usage of Zero-DyDSCU+DyPL-VIO and HE+DyPL-VIO is 39.48% and 42.87%, respectively. The average GPU usage of the algorithm of the present invention is lower than that of HE+DyPL-VIO. The results show that in normal light scenes, image enhancement occupies computing resources. The introduction of a dynamic switching mechanism can selectively enhance images, which can help reduce GPU usage and improve computing efficiency.

[0204] (2) Outdoor experiment

[0205] Figure 18 is the outdoor dark light experiment trajectory and error diagram, where Figure 18 (a) shows the real-time motion trajectories of the baseline (groundtruth, dashed line), the DyPL-VIO algorithm without the enhancement module (black curve), the comparison algorithm HE+DyPL-VIO (blue curve), and the Zero-DyDSCU+DyPL-VIO algorithm with the enhancement module (red curve). Because the algorithm without the enhancement module loses trajectory, only the easting and northing trajectories and positioning error results of the HE+DyPL-VIO and Zero-DyDSCU+DyPL-VIO algorithms are shown: Figure 18 (b) The real-time trajectory changes of the easting coordinates and northing coordinates of the two algorithms and the benchmark are given respectively. Figure 18 (c) shows the real-time eastward trajectory error, northward trajectory error, and total trajectory error of the proposed algorithm and the comparison algorithm. The root mean square error of the positioning trajectory of the algorithm is summarized in Table 5.

[0206] Table 5 Algorithm error of outdoor dark light experiment (m)

[0207]

[0208] from Figure 18 Comparing the trajectories of (a) and (b), we can see that the algorithm without the integrated image enhancement module loses its trajectory in low-light environments, validating the necessity of the image enhancement module in low-light scenes. In contrast, the algorithms integrating the HE and Zero-DyDSCU enhancement modules both stably track the trajectory, but there are significant differences in their positioning accuracy. The HE+DyPL-VIO algorithm has a maximum error of 4.1367m and a root mean square error of 1.8781m, indicating significant error fluctuations in its positioning process. The Zero-DyDSCU+DyPL-VIO algorithm has a maximum error of 2.6814m and a root mean square error of 1.2466m, which are 35.18% and 33.62% lower than the HE+DyPL-VIO, respectively.

[0209] This performance gap can be attributed to the essential differences in how the two types of enhancement algorithms optimize image quality. Figure 19Figure 3. Features extracted by DyPL-VIO on different enhanced images. (a), (b), and (c) correspond to the original image, HE enhancement, and Zero-DyDSCU enhancement, respectively. Purple indicates a long tracking time for the feature. Long-tracked features achieve higher matching accuracy between consecutive frames. Because these features are repeatedly observed across multiple frames, their descriptors are more discriminative and matching ambiguity is lower, reducing the probability of mismatches and making the algorithm more robust. Green indicates a short tracking time for the feature. Short-term features may be lost due to sudden changes in brightness or angle, resulting in computational redundancy and matching failure.

[0210] The HE algorithm causes some local overexposure due to global histogram stretching ( Figure 19 (b) Insufficient restoration of some illumination levels leads to a decrease in contrast, which results in a decrease in the number of reliable new features extracted by the front-end (fewer purple line features) and limited geometric constraint capabilities. Misdetected features in noise areas introduced by uneven exposure introduce abnormal constraints during back-end optimization, leading to trajectory deviation. Zero-DyDSCU uses a decoding and encoding mechanism and jump connections to adaptively enhance local areas of different illumination levels in the image. This results in better illumination restoration, significantly improved contrast, and reduced noise while restoring image details. The quality of extracted line features is better ( Figure 19 (c) The purple line has more features). High-quality feature input leads to higher positioning accuracy.

[0211] Based on the same technical concept as the method embodiment, the present invention also provides an inertial vision positioning system for a mobile robot in a weak-texture dark light environment, comprising: an image preprocessing module for acquiring image frames, performing illumination value judgment on the image frames, and if the illumination value of the image frame is lower than a specified threshold, performing image enhancement processing on the image before entering the front-end processing module; otherwise, directly entering the front-end processing module;

[0212] A front-end processing module is configured to extract point features based on image frames using the ORB algorithm, extract line features using the improved EDLines algorithm, match point and line features using the Hamming distance based on a reference image, and obtain an initial pose estimate using the PnP method if a match is successful. The improved EDLines algorithm filters anchor points based on gradient directional filtering, performs line segment fitting on the filtered anchor points using an iterative weighted method, merges similar line segments, and dynamically adjusts the pixel chain threshold of the EDLines algorithm based on the tracking of point features. The module also pre-integrates IMU measurement data to obtain the pose relationship between adjacent key frames, and performs visual initialization and visual-inertial joint initialization.

[0213] The back-end optimization module is used to fuse visual and inertial information in a tightly coupled sliding window manner, build a system residual model for local optimization, and output position and attitude information.

[0214] It should be understood that the inertial vision positioning system for a mobile robot in a weak-texture dark light environment in the embodiment of the present invention can implement all the technical solutions in the above-mentioned method embodiment, and the functions of its various functional modules can be specifically implemented according to the method in the above-mentioned method embodiment. The specific implementation process can refer to the relevant description in the above-mentioned embodiment, and will not be repeated here.

[0215] The present invention also provides a computer device comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the inertial vision positioning method for a mobile robot in a weak-texture dark light environment as described above are implemented.

[0216] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the inertial vision positioning method for a mobile robot in a weak-texture dark light environment as described above are implemented.

Claims

1. A method for inertial vision positioning of a mobile robot in a weak texture dark light environment, characterized by: include: Step S1, acquiring an image frame and determining the illumination value of the image frame. If the illumination value of the image frame is lower than a specified threshold, the image is enhanced and the process proceeds to step S2; otherwise, the process proceeds directly to step S2; Step S2: extracting point features using the ORB algorithm based on the image frame, extracting line features using the improved EDLines algorithm, matching point and line features using the Hamming distance based on the reference image, and obtaining an initial pose estimate using the PnP method if the match is successful. The improved EDLines algorithm filters anchor points based on gradient directional filtering, performs line segment fitting on the filtered anchor points using an iterative weighted method, merges similar line segments, and dynamically adjusts the pixel chain threshold of the EDLines algorithm based on the tracking of the point features; pre-integrates the inertial measurement unit (IMU) measurement data to obtain the pose relationship between adjacent key frames; and performs visual initialization and visual-inertial joint initialization. In step S3, the visual-inertial information is fused in a tightly coupled manner based on a sliding window, a system residual model is constructed for local optimization, and position and attitude information is output.

2. The method according to claim 1, characterized in that Perform image enhancement, including: The image is enhanced using DSCU-net based on the U-Net network. The DSCU-net replaces the traditional convolution in the U-Net network with a depth-separable convolution operation. The encoder consists of three depth-separable convolution layers and a downsampling layer, and the decoder consists of three upsampling layers. It performs skip splicing operations and uses the Tanh activation function to generate output.

3. The method according to claim 1, characterized in that Filter the anchor points based on gradient direction filtering, including: Calculate the gradient values of all anchor points on the fitted line, sort them from large to small, and obtain the median value; calculate the absolute deviations between all anchor points and the median value, sort the absolute deviations from small to large, and obtain the median of the absolute deviations; determine the threshold range based on the median of the absolute deviations and the median value of the gradient, and identify points whose gradient directions exceed the specified threshold range as nonlinear points and automatically remove them.

4. The method according to claim 1, wherein Line segments are fitted to the filtered anchor points using an iterative weighting method, including: Calculate the distance from each anchor point to the fitted line, use the distance as the variable of the weight function, and solve the weighted least squares problem: Determine the final linear parameters, where L(k,b) is the linear estimation parameter at the t+1th iteration; W i (β (t) ) is the corresponding weight for each point; β (t) is the minimization parameter at the tth iteration; k is the slope; b is the intercept; num is the number of anchor points involved in the calculation, x i ,y i is the coordinate of anchor point i.

5. The method according to claim 1, wherein Merge similar line segments, including: Define line features i The angle with the horizontal direction is θ i ∈[0,π], define the initial line segment set as L1; Determine the angle θ1 between the specified line segment l1 and any other line segment l in L1 i Angle θ i If the angle difference is less than the angle threshold, l i Add to the line segment set L2; Calculate the line segment l in L2 i The distance from the endpoint to other line segments in the set, and determine whether the distance is less than the distance threshold from the endpoint to the line segment. If it is less than, l i Add to the line segment set L3; Calculate the line segment l in L3 i The distance between the two endpoints of and the two endpoints of other line segments, select the minimum distance, and determine whether the minimum distance is less than the specified endpoint distance threshold. If it is less than, l i Add to line segment set L4; The line segment set L4 is a set of line segments to be fitted, and these line segments are fitted into a straight line using a line segment fitting function.

6. The method according to claim 1, characterized in that Dynamically adjust the pixel chain threshold of the EDLines algorithm based on the tracking of point features, including: Use tracking failure rate D p To represent the tracking status of point features, for the i-th frame image, It is defined as the average of the feature point tracking failure rates in the previous several frames of images. The specific calculation is as follows: in, is the number of feature points tracked in the jth frame; is the number of feature points continuously tracked in the jth frame; M is the number of frames involved in the calculation; Will Linear mapping to the calculation formula of threshold L: in, is a value between 0 and 1; L min is the lower limit of the line segment length; L max is the maximum threshold of the long line segment; Sort all detected line segments by length and select the first N line segments for matching.

7. The method according to claim 1, characterized in that The tightly coupled sliding window-based method fuses visual-inertial information, including: The optimization is performed within a time window, and the information outside the window is marginalized. If the penultimate frame of the sliding window is a key frame, the key frame is retained, and the earliest frame in the window and its related information are marginalized. If the penultimate frame is a non-key frame, the related feature information is discarded, and only the inertial navigation measurement data is retained.

8. An inertial vision positioning system for a mobile robot in a weak texture dark light environment, characterized by: include: The image preprocessing module is used to obtain image frames and judge the illumination value of the image frames. If the illumination value of the image frame is lower than the specified threshold, the image is enhanced and then enters the front-end processing module; otherwise, it directly enters the front-end processing module; A front-end processing module is configured to extract point features based on image frames using the ORB algorithm, extract line features using the improved EDLines algorithm, match point and line features using the Hamming distance based on a reference image, and obtain an initial pose estimate using the PnP method if a match is successful. The improved EDLines algorithm filters anchor points based on gradient directional filtering, performs line segment fitting on the filtered anchor points using an iterative weighted method, merges similar line segments, and dynamically adjusts the pixel chain threshold of the EDLines algorithm based on the tracking of point features. The module also pre-integrates IMU measurement data to obtain the pose relationship between adjacent key frames, and performs visual initialization and visual-inertial joint initialization. The back-end optimization module is used to fuse visual and inertial information in a tightly coupled sliding window manner, build a system residual model for local optimization, and output position and attitude information.

9. A computer device, characterized in that: include: one or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the inertial vision positioning method for a mobile robot in a weak-texture dark light environment as described in any one of claims 1-7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the inertial vision positioning method for a mobile robot in a weak-texture dark light environment are implemented as described in any one of claims 1 to 7.

Images