A visual positioning and mapping method based on feature tracking and matching

Abstract

The present invention discloses a visual positioning and mapping method based on feature tracking and matching, comprising: quantifying the illumination and texture intensity of an image captured by a camera to obtain illumination quantization results and texture quantization results; using the obtained illumination quantization results and texture quantization results through MobileNetV4 to obtain the credibility of a direct method and the credibility of a feature point method; inputting the credibility of the direct method and the credibility of the feature point method into a fusion module to obtain a weighted ratio of the results of the direct method and the feature point method; and inputting the weighted ratio of the results of the direct method and the feature point method into a state estimation module to perform state estimation. The present invention fully considers the impact of low-light and low-texture environments on algorithm performance, establishes a relationship between environmental factors and algorithm performance through MobileNetV4, and utilizes feature descriptors to achieve closed-loop detection, thereby improving the overall performance of the system and having the advantages of high trajectory accuracy and good robustness.

Description

Technical Field

[0001] The present invention belongs to the technical field of simultaneous positioning and mapping, and in particular relates to a visual positioning and mapping method based on feature tracking and matching. Background Art

[0002] Accurate and robust state estimation is crucial for autonomous navigation of mobile robots. The direct and feature point methods used in traditional vision / inertial SLAM systems each have their own unique advantages, but also inherent limitations. Effectively integrating these two approaches to leverage their complementary strengths in diverse environments and enhance system performance in complex scenarios remains a key research topic.

[0003] In terms of existing research, SVO achieves feature tracking and pose estimation by minimizing photometric errors, but lacks a closed-loop detection mechanism. LDSO introduces feature descriptors to achieve closed-loop detection, but feature point information is not used for state estimation. Other studies have attempted to fuse the two methods through different fusion mechanisms, such as switching mechanisms based on the number of features, error-weighted fusion, etc., but there is still room for improvement in accuracy and computational efficiency. However, there are currently two major problems with the system. One is how to make full use of environmental factors to guide the fusion of the direct method and the feature point method under limited computing resources, and the other is how to effectively correct the accumulated errors. Therefore, a new technical solution is needed to solve these problems. Summary of the Invention

[0004] Purpose of the invention: In order to overcome the deficiencies in the prior art, a visual positioning and mapping method based on feature tracking and matching is provided.

[0005] Technical Solution: To achieve the above objectives, the present invention provides a visual positioning and mapping method based on feature tracking and matching, comprising the following steps:

[0006] S1: quantify the illumination and texture intensity of the image captured by the camera to obtain illumination quantization results and texture quantization results;

[0007] S2: The obtained illumination quantization results and texture quantization results are passed through MobileNetV4 to obtain the direct method credibility and feature point method credibility;

[0008] S3: Input the credibility of the direct method and the credibility of the feature point method into the fusion module to obtain the weight ratio of the results of the direct method and the feature point method;

[0009] S4: Input the weight ratio of the results of the direct method and the feature point method into the state estimation module to perform state estimation.

[0010] Furthermore, the step S1 includes:

[0011] A1: A data set is acquired through camera capture, and the data set acquired by the camera is processed to obtain a processed data set;

[0012] A2: Process the data set processed in step A1 using an illumination quantization algorithm to obtain an illumination quantization result;

[0013] A3: Processing the data set processed in step A1 by a texture quantization algorithm to obtain a texture quantization result.

[0014] Furthermore, step A1 specifically includes: performing feature tracking and matching on the images in the data set using the feature point method and the direct method, obtaining the pixel position of the feature point by the feature point method, and obtaining the gradient of the tracking feature by the direct method; processing one frame of the image, manually adjusting the pixel brightness or dynamic blur, and performing feature tracking and matching on the processed image again, recording the pixel position changes of the matching features by the feature point method and the gradient changes of the tracking features by the direct method, acquiring all images in the data set by the camera, and repeating the above steps to form a data set.

[0015] Furthermore, the step A2 specifically includes:

[0016] A2-1: Establish the current frame With reference frame The feature point method between the two methods obtains the corresponding relationship of the feature points. Grayscale features of feature points Matching is performed, and the similarity between the descriptors of the feature points is calculated by the Hamming distance calculation method in the feature point method, and the most similar feature point is found as Corresponding feature points Grayscale features , These are the grayscale features of the 1st, 2nd…, nth feature points respectively;

[0017] A2-2: Initial illumination quantification results , Used to accumulate the grayscale difference of pixels in grayscale features; traverse through the for loop Grayscale features of feature points and Each corresponding feature point Grayscale features , execute the formula:

[0018]

[0019] That is, calculate the absolute value of the pixel grayscale difference between the matching feature points in two adjacent frames of images and add it to the degree of illumination change. ,in ( ) indicates that in the image Corresponding grayscale features The pixel gray value, ( ) indicates that in the image Middle feature point The pixel gray value;

[0020] A2-3: When the image All After all feature points have completed the operation, the loop ends; the accumulated Divide by the total number of feature points , that is, execute , get the illumination quantization result and output it.

[0021] Furthermore, the step A3 specifically includes:

[0022] A3-1: For images , set the image pixel gray level k, step size ,direction ;

[0023] For images Processing , and initialize the contrast parameter value ;

[0024] A3-2: Traversing directions through for loop , calculate the gray level co-occurrence matrix for each direction , in step length and direction Under the condition, the starting pixel gray level is , the ending pixel gray level is The probability of occurrence is recorded as , when the image The grayscale value is divided into levels and determine the step size and direction spatial relationship; is the row index, is the column index, each row of the matrix corresponds to a starting pixel gray level, and each column corresponds to an ending pixel gray level; the probability value is used as the matrix element to construct a The matrix:

[0025]

[0026] A3-3: Calculate the contrast parameter value in this direction based on the matrix ; The calculation formula is as follows:

[0027]

[0028] The contrast parameter value calculated in the current direction Added to the contrast parameter cumulative value When all the specified directions are traversed, the loop ends; the contrast parameter is accumulated. Divide by the number of directions , get the final texture quantization result .

[0029] Furthermore, the step S2 specifically includes: using the MobileNetV4 neural network to perform credibility regression, and forming a credibility prediction model with a three-layer fully connected network; the network input layer dimension is 2, the activation function is Relu; the number of hidden layer nodes is 256, the activation function is Relu; the number of output layer nodes is 2, and there is no activation function; the network takes the illumination change L and the texture intensity T as input, and outputs the direct method credibility and the credibility of the feature point method .

[0030] Furthermore, the step S3 specifically includes:

[0031] B1: Analyze the reliability of the direct method:

[0032] In normal adjacent frames and In the example, there are features of feature point matching obtained according to the direct method. and ,Change The brightness or blur of the frame gets the corresponding frame ,use right Match the features in the ,in and Because the environmental factors are not exactly the same, the calculation and The gradient difference of the corresponding features in and the arithmetic mean ;

[0033] B2: Analyze the credibility of the feature point method:

[0034] In normal adjacent frames and There are matching features in and ,Change The brightness or blur of the frame gets the corresponding frame ,use right Match the features in the ,in and Because the environmental factors are not exactly the same, the calculation and The pixel position difference of the corresponding features in and the arithmetic mean ;

[0035] B3: Weight ratio of the results of direct method and feature point method and Defined as follows:

[0036] .

[0037] Furthermore, the step S4 specifically includes:

[0038] C1: The relative pose between adjacent nodes is calculated as follows:

[0039]

[0040] in, It is Frame relative to Frame, in The position in the frame coordinate system is obtained by the position of the two frames in the world coordinate system. 、 and The rotation matrix of the frame Transformation is obtained; and They are the posture related quantities of the two frames, and the difference between the two is the relative posture ;

[0041] C2: After a loop occurs, the following constraints are established for the keyframe poses between the loop frames:

[0042]

[0043] in, is the error term, which includes position error and attitude error. and Indicates different frame numbers; It is The inverse matrix of the frame rotation matrix is used to convert the position in the world coordinate system 、 Transform to Frame coordinate system and relative position Calculate the position error; the attitude error is calculated by the attitude difference between loop frames.

[0044] Furthermore, the goal of pose graph optimization in step C2 is:

[0045]

[0046] in, is the set of all consecutive frames, is the set of all closed-loop frames, represents the set of variables to be optimized, is the kernel function, is the error term The above closed-loop detection and optimization further enhance the role of feature point descriptors, reduce the cumulative error of historical key frames, and fully improve the positioning accuracy of the system.

[0047] Beneficial effects: Compared with the existing technology, the present invention proposes a new adaptive fusion algorithm, which integrates the visual / inertial SLAM of the direct method and the feature point method. Different from the existing switching mechanism, the present invention fully considers the impact of the weak light and weak texture environment on the algorithm performance, establishes the relationship between environmental factors and algorithm performance through MobileNetV4, and uses feature descriptors to realize closed-loop detection, thereby improving the overall performance of the system, and has the advantages of high trajectory accuracy and good robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a process framework diagram of the method of the present invention;

[0049] Figure 2 It is a flow chart for making a data set in the present invention;

[0050] Figure 3 It is an image of simulated illumination changes during the data set preparation process of the present invention;

[0051] Figure 4 It is a simulated dynamic blurred image during the data set preparation process of the present invention;

[0052] Figure 5 This is a simplified structural diagram of MobileNetV4 of the present invention;

[0053] Figure 6 This is a schematic diagram of the situation where environmental factors change feature matching in the present invention;

[0054] Figure 7 Schematic diagram of closed-loop constraints in the system loop of the present invention;

[0055] Figure 8 These are the trajectory visualization results of the three algorithms in some test sequences. DETAILED DESCRIPTION

[0056] The present invention is further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0057] Example 1:

[0058] like Figure 1 As shown, this embodiment provides a visual positioning and mapping method based on feature tracking and matching, including the following steps:

[0059] S1: quantify the illumination and texture intensity of the image captured by the camera to obtain illumination quantization results and texture quantization results;

[0060] Step S1 includes:

[0061] A1: A data set is acquired through camera capture, and the data set acquired by the camera is processed to obtain a processed data set;

[0062] Figure 2 This is the dataset preparation process provided by this embodiment, which performs feature tracking and matching on the screened image pairs, and records the pixel positions of the matching features using the feature point method and the gradients of the tracking features using the direct method.

[0063] Process one of the frames and adjust pixel brightness or motion blur. Figure 3 For part of the dataset after brightness adjustment, brightness adjustment is performed on the image pairs selected from the dataset to simulate the light changes in real scenes. Figure 3 (a) is the original image of the dataset. Figure 3 (b) in the middle is the image after increasing the brightness by 40%. Figure 3 The image in (c) is reduced by 40% in brightness. Motion blur is added to simulate the weak texture scenes in real scenes.

[0064] Figure 4 (a) is the original image of the dataset. Figure 4 (b) is the image after dynamic blurring of 4 pixels. Figure 4 Image (c) shows the image after dynamic blurring by 8 pixels. Feature tracking and matching are performed again on the processed image pair. The pixel positions of the matched features using the feature point method and the gradients of the tracked features using the direct method are recorded to verify whether they meet the initial screening criteria. Using the normal frame as the intermediate frame, the same features in the three frames are connected and the reliability of the direct method and feature point method is calculated. Repeat these steps to record all the data to form a dataset.

[0065] A2: Process the data set processed in step A1 using an illumination quantization algorithm to obtain an illumination quantization result;

[0066] Step A2 specifically includes:

[0067] A2-1: Establish the current frame With reference frame The feature point method between the two methods obtains the corresponding relationship of the feature points. Grayscale features of feature points Matching is performed, and the similarity between the descriptors of the feature points is calculated by the Hamming distance calculation method in the feature point method, and the most similar feature point is found as Corresponding feature points Grayscale features , These are the grayscale features of the 1st, 2nd…, nth feature points respectively;

[0068] A2-2: Initial illumination quantification results , Used to accumulate the grayscale difference of pixels in grayscale features; traverse through the for loop Grayscale features of feature points and Each corresponding feature point Grayscale features , execute the formula:

[0069]

[0070] That is, calculate the absolute value of the pixel grayscale difference between the matching feature points in two adjacent frames of images and add it to the degree of illumination change. ,in ( ) indicates that in the image Corresponding grayscale features The pixel gray value, ( ) indicates that in the image Middle feature point The pixel gray value;

[0071] A2-3: When the image All After all feature points have completed the operation, the loop ends; the accumulated Divide by the total number of feature points , that is, execute , get the illumination quantization result and output it.

[0072] A3: Processing the data set processed in step A1 by a texture quantization algorithm to obtain a texture quantization result;

[0073] Step A3 specifically includes:

[0074] A3-1: For images In this embodiment, the gray level of the image pixel is set , step length ,direction ;

[0075] step length When calculating the gray-level co-occurrence matrix, the spatial distance between pixels is defined; direction The set of directions considered when calculating the gray-level co-occurrence matrix;

[0076] For images Processing , change the grayscale value range of the pixels in the original image to the set new range; at the same time initialize the contrast parameter value ;

[0077] A3-2: Traversing directions through for loop , calculate the gray level co-occurrence matrix for each direction , in step length and direction Under the condition, the starting pixel gray level is , the ending pixel gray level is The probability of occurrence is recorded as , when the image The grayscale value is divided into levels and determine the step size and direction spatial relationship; is the row index, is the column index, and each row of the matrix corresponds to a starting pixel gray level (using express, From 1 to value), each column corresponds to a termination pixel gray level (using express, From 1 to Take the probability value); use the probability value as the matrix element to construct a The matrix:

[0078] ;

[0079] A3-3: Calculate the contrast parameter value in this direction based on the matrix ; The calculation formula is as follows:

[0080]

[0081] The contrast parameter value calculated in the current direction Added to the contrast parameter cumulative value When all the specified directions are traversed, the loop ends; the contrast parameter is accumulated. Divide by the number of directions , get the final texture quantization result .

[0082] S2: The obtained illumination quantization results and texture quantization results are passed through MobileNetV4 to obtain the direct method credibility and the credibility of the feature point method ;

[0083] In this embodiment, the MobileNetV4 neural network is used for credibility regression, such as Figure 5 As shown in Figure 1, a three-layer fully connected network is used to construct the credibility prediction model, which consists of input layer, hidden layer and output layer. The network input layer dimension is 2, and the activation function is Relu; the number of nodes in the hidden layer is 256, and the activation function is Relu; the number of nodes in the output layer is 2, and there is no activation function. The network takes illumination change L and texture intensity T as input and outputs direct method credibility and the credibility of the feature point method .

[0084] In this embodiment, the training set is ,in, Represents input data, Represents the training label. And the mean square error of the training labels constitutes the loss function:

[0085] ,

[0086] After obtaining the loss function, gradient descent is performed based on MobileNetV4 to update the network parameters.

[0087] S3: Output direct method credibility and the credibility of the feature point method Input into the fusion module to obtain the weight ratio of the results of the direct method and the feature point method;

[0088] Due to changes in environmental factors, errors may occur in feature tracking or matching using the direct method and feature point method. For example, under normal circumstances, the feature matching results may be as follows: Figure 6 As shown in (a) in the figure, when the environmental factors change, the matching results are as follows Figure 6 As shown in (b) of Figure 1, due to changes in environmental factors, features in the image may be mismatched. Therefore, it is necessary to quantitatively evaluate the performance of the direct method and the feature point method in different environments.

[0089] Step S3 specifically includes:

[0090] B1: Analyze the reliability of the direct method:

[0091] In normal adjacent frames and In the example, there are features of feature point matching obtained according to the direct method. and ,Change The brightness or blur of the frame gets the corresponding frame ,use right Match the features in the ,in and Because the environmental factors are not exactly the same, the calculation and The gradient difference of the corresponding features in and the arithmetic mean ;

[0092] B2: Analyze the credibility of the feature point method:

[0093] In normal adjacent frames and There are matching features in and ,Change The brightness or blur of the frame gets the corresponding frame ,use right Match the features in the ,in and Because the environmental factors are not exactly the same, the calculation and The pixel position difference of the corresponding features in and the arithmetic mean ;

[0094] B3: Direct Method Credibility Credibility of feature point method They cannot be directly used as weights in the fusion algorithm. This is because these two values are used as quantities to evaluate the performance of the direct method and the feature point method. As weights, participating in the system state estimation may make the visual information account for too large a proportion in the overall optimization, limiting the influence of other quantities. Both the direct method and the feature point method use visual information to provide visual constraints for the system. Therefore, in state estimation, their importance should be consistent with other residual terms. Therefore, the weight of the results of the direct method and the feature point method is proportional to the weight of the system state. and Defined as follows:

[0095] ;

[0096] S4: Input the weight ratio of the results of the direct method and the feature point method into the state estimation module to perform state estimation.

[0097] In this embodiment, Figure 1 The pre-integration part can be implemented by referring to the following literature:

[0098] [C. Campos, R. Elvira, JJG Rodríguez, JMM Montiel, and JD Tardós, “ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM,” IEEE Transactions on Robotics, vol. 37, no. 6, pp. 1874–1890, Dec. 2021, doi: 10.1109 / TRO.2021.3075644.】

[0099] In odometry, the feature point method can use the calculated descriptors to achieve closed loop detection. Usually, the loop is determined by comparing the similarity between the current scene and the map. When a loop is detected, the system will update the positioning based on this result to correct the accumulated error. In vision / inertial SLAM, inertial measurement contains gravity information. After the system is initialized, the magnitude and direction of gravity are already known to the system. When a loop occurs, the six-degree-of-freedom posture of the system will be simplified to a four-degree-of-freedom correction, optimizing the position and yaw angle while keeping the roll angle and pitch angle unchanged. The establishment of the loop will introduce closed loop constraints. For example Figure 7 As shown in Figure 2, when a loop occurs in the system, a closed-loop constraint will be established.

[0100] Step S4 specifically includes:

[0101] C1: In odometry, feature point methods can utilize calculated descriptors to achieve loop closure detection. Typically, loop closure is determined by comparing the current scene with the map. When a loop is detected, the system updates the positioning based on the result, correcting any accumulated errors. The relative pose of adjacent nodes between frames is calculated using the following formula:

[0102]

[0103] in, It is Frame relative to Frame, in The position in the frame coordinate system is obtained by the position of the two frames in the world coordinate system. 、 and The rotation matrix of the frame Transformation is obtained; and They are the posture related quantities of the two frames, and the difference between the two is the relative posture ;

[0104] C2: After a loop occurs, the following constraints are established for the keyframe poses between the loop frames:

[0105]

[0106] in, is the error term, which includes position error and attitude error. and Indicates different frame numbers; It is The inverse matrix of the frame rotation matrix is used to convert the position in the world coordinate system 、 Transform to Frame coordinate system and relative position Calculate the position error; the attitude error is calculated by the attitude difference between loop frames.

[0107] The goal of pose graph optimization is:

[0108]

[0109] in, is the set of all consecutive frames, is the set of all closed-loop frames, Represents the set of variables to be optimized; It is a kernel function, which is introduced in the optimization The main purpose is to deal with outliers and reduce the weight of data points with large errors to avoid these outliers from having too much negative impact on the optimization results; when the error is small, the kernel function The role of is similar to the identity function; when the error is large, the kernel function will reduce the weight of the error, making the optimization process more robust. is the error term The squared binorm of quantifies the error. In the optimization objective, minimizing this value reduces the pose error. This closed-loop detection and optimization further leverages the feature point descriptor, reducing the accumulated error of historical keyframes and significantly improving the system's positioning accuracy.

[0110] Example 2:

[0111] To verify the effectiveness and effect of the present invention, this example runs the algorithm provided by the present invention on the public dataset EuRoc. The experimental results are compared with those of other algorithms to verify the good performance of the present invention in low-light and low-texture environments. The details are as follows:

[0112] For the experimental comparison, ORB-SLAM3 and VI-DSO were selected. Both algorithms employ sliding window optimization, but their state estimation is optimized for geometric and photometric errors, respectively. To ensure objectivity and comparability, the sliding window parameter for all algorithms was set to 10 frames.

[0113] The experimental evaluation was carried out based on 11 sequences of the EuRoc dataset. Each algorithm was tested 5 times on each sequence, with a cumulative test time of more than 300 minutes. The total test time for a single algorithm exceeded 100 minutes. In order to quantitatively evaluate the system performance, the root mean square error (RMSE) of the trajectory translation was used as the evaluation criterion for positioning accuracy. The performance of the three algorithms, ODF-SLAM (the algorithm of this invention), ORB-SLAM3, and VI-DSO, on each sequence was listed in detail. Figure 8 The trajectory visualization results of the three algorithms in some test sequences are given, where the dotted line represents the true trajectory, which is the accurate trajectory used as a reference; "VI-DSO" and "ORB-SLAM3" are comparison algorithms, and "ODF-SLAM" is the algorithm of the present invention. Figure 8 (a) is the test result in the dataset scene named MH_01. It can be seen that the trajectory (red line) generated by the ODF-SLAM algorithm closely fits the real trajectory (dashed line). Compared with the two comparison algorithms VI-DSO (green line) and ORB-SLAM3 (blue line), ODF-SLAM can more accurately follow the twists and turns of the real trajectory and show better positioning accuracy in complex paths, indicating that it is more effective in extracting and utilizing environmental features in this scenario and more accurate in pose estimation. Figure 8 (b), (c), and (d) are the test results under the datasets named MH_03, V1_01, and V1_03 respectively. It can be seen that the test results are also similar to those in Figure 8 The situation is similar to that in (a), which shows that in these test scenarios, the ODF-SLAM algorithm outperforms other comparison algorithms in trajectory estimation accuracy and has better performance.

Claims

1. A visual positioning and mapping method based on feature tracking and matching, characterized in that: The steps include: S1: Perform quantitative calculations on the illumination and texture intensity of the image captured by the camera to obtain illumination quantization results and texture quantization results; S2: The obtained illumination quantization results and texture quantization results are passed through MobileNetV4 to obtain the direct method credibility and feature point method credibility; S3: Input the credibility of the direct method and the credibility of the feature point method into the fusion module to obtain the weight ratio of the results of the direct method and the feature point method; S4: Input the weight ratio of the results of the direct method and the feature point method into the state estimation module to perform state estimation; Step S3 specifically includes: B1: Analyze the reliability of the direct method: In normal adjacent frames I i and I i+1 In the example, there are features of feature point matching obtained according to the direct method. and Change I i+1 The brightness or blur of the frame gets the corresponding frame Use D i right Match the features in the Among them D i+1 and Because the environmental factors are not exactly the same, calculate D i+1 and The gradient difference of the corresponding features and the arithmetic mean RD; B2: Analyze the credibility of the feature point method: In normal adjacent frames I i and I i+1 There are matching features in and Change I i+1 The brightness or blur of the frame gets the corresponding frame Use F i right Match the features in the Among them F i+1 and Because the environmental factors are not exactly the same, calculate F i+1 and The pixel position difference of the corresponding feature in the and the arithmetic mean RF; B3: The weight ratio of the results of the direct method and the feature point method ω d and ω f Defined as follows: Step S4 specifically includes: C1: The relative pose between adjacent nodes is calculated as follows: in, It is the position of the i+1th frame relative to the ith frame in the ith frame coordinate system, through the positions of the two frames in the world coordinate system And the rotation matrix of the i-th frame Transformation is obtained; and They are the posture related quantities of the two frames, and the difference between the two is the relative posture C2: After a loop occurs, the following constraints are established for the keyframe poses between the loop frames: Among them, e i,j is the error term, which includes position error and attitude error. i and j represent different frame numbers. It is the inverse matrix of the rotation matrix of the i-th frame, which is used to convert the position in the world coordinate system Transform to the i-th frame coordinate system and the relative position P i,i+1 Calculate the position error; the attitude error is calculated by the attitude difference between loop frames; The goal of pose graph optimization in step C2 is: Among them, S is the set of all continuous frames, L is the set of all closed-loop frames, P represents the set of variables to be optimized, ρ is the kernel function, is the error term e i,j The squared norm of .

2. The visual positioning and mapping method based on feature tracking and matching according to claim 1, characterized in that: The step S1 comprises: A1: A data set is acquired through camera capture, and the data set acquired by the camera is processed to obtain a processed data set; A2: Process the data set processed in step A1 using an illumination quantization algorithm to obtain an illumination quantization result; A3: Processing the data set processed in step A1 by a texture quantization algorithm to obtain a texture quantization result.

3. The visual positioning and mapping method based on feature tracking and matching according to claim 2, characterized in that: The step A1 specifically includes: using the feature point method and the direct method to track and match features on the images in the data set, obtaining the pixel position of the feature point by the feature point method, and obtaining the gradient of the tracked feature by the direct method; processing one frame of the image, manually adjusting the pixel brightness or dynamic blur, and again tracking and matching the processed image, recording the pixel position changes of the matching features of the feature point method and the gradient changes of the tracking features of the direct method, acquiring all the images in the data set by the camera, and repeating the above steps to form a data set.

4. The method for visual positioning and mapping based on feature tracking and matching according to claim 2, wherein: The step A2 specifically includes: A2-1: Establish current frame I i With reference frame I i+1 The feature point method between the two obtains the corresponding relationship of the feature points. i The grayscale feature of the feature point f={f1,f2…f n } to match, calculate the similarity between the descriptors of the feature points by the Hamming distance calculation method in the feature point method, and find the most similar feature point as I i+1 The corresponding feature points f1,f2…f n Grayscale features f1,f2…f n These are the grayscale features of the 1st, 2nd…, nth feature points respectively; A2-2: The initial illumination quantization result L = 0, L is used to accumulate the pixel grayscale difference of the grayscale feature; through the for loop traversal I i Grayscale features f and I of the feature points in i+1 Each corresponding feature point f1,f2…f n Grayscale features Execute the formula: That is, calculate the absolute value of the pixel grayscale difference between the matching feature points in two adjacent frames of images and add it to the illumination change degree L + ,in Indicated in image I i+1 Corresponding grayscale features The pixel gray value, I i (f) represents the image I i The pixel gray value of the feature point f; A2-3: When the image I i After all n feature points in the loop have completed the operation, the loop ends; the accumulated L + Divide by the total number of feature points n, that is, execute L=L + / n, get the lighting quantization result and output it.

5. The method for visual positioning and mapping based on feature tracking and matching according to claim 2, wherein: The step A3 specifically includes: A3-1: For image I i , set the image pixel gray level k, step size d, and direction θ; For image I i Process I = [I*k / 256] and initialize the contrast parameter value CON i =0; A3-2: Use a for loop to traverse the direction θ and calculate the gray level co-occurrence matrix G(a,b) in each direction. Under the conditions of step size d and direction θ, the probability of the starting pixel gray level being a and the ending pixel gray level being b is recorded as Count(a,b). When the image I i The grayscale value of is divided into N levels, and the spatial relationship between the step size d and the direction θ is determined; with a as the row index and b as the column index, each row of the matrix corresponds to a starting pixel grayscale level, and each column corresponds to an ending pixel grayscale level; the probability value is used as the matrix element to construct an N*N matrix: A3-3: Calculate the contrast parameter value CON in this direction based on the matrix i ;CON i The calculation formula is as follows: The contrast parameter value CON calculated in the current direction i Added to the contrast parameter accumulated value CON + When all the specified directions are traversed, the loop ends; the contrast parameter is accumulated to CON + Divide by the number of directions v to get the final texture quantization result T i =Con + / v.

6. The visual positioning and mapping method based on feature tracking and matching according to claim 1, characterized in that: The step S2 specifically includes: using the MobileNetV4 neural network to perform credibility regression, and forming a credibility prediction model with a three-layer fully connected network; the network input layer dimension is 2, and the activation function is Relu; the number of hidden layer nodes is 256, and the activation function is Relu; the number of output layer nodes is 2, and there is no activation function; the network takes illumination change L and texture intensity T as input, and outputs direct method credibility RF and feature point method credibility RD.

Images