A DDPG visual SLAM parameter adaptation algorithm integrating attention mechanism

Abstract

This application discloses a DDPG visual SLAM parameter adaptation algorithm that integrates an attention mechanism in the technical fields of computer vision, image processing, mixed reality, etc. The algorithm uses video frames to construct a state space as the input of the algorithm; selects three parameters of ORB‑SLAM3: nearest neighbor threshold, number of pyramid layers, and scale factor, and uses them to construct a continuous action space; calculates posture uncertainty through the RMSE of posture, and constructs a reward function, and finally obtains adaptive adjustment parameters by improving the DDPG network, effectively improving the posture accuracy. The algorithm integrates the convolution module, GRU module, and Transformer module to achieve the optimization of DDPG. The algorithm solves the problem of visual SLAM parameter adaptation and effectively improves the accuracy of posture estimation. The invention has broad application prospects in the fields of robotics, mixed reality, navigation and positioning.

Description

Technical Field

[0001] The present invention relates to technical fields such as computer vision, image processing, and mixed reality, and in particular to a DDPG visual SLAM parameter adaptation algorithm integrating an attention mechanism. Background Art

[0002] SLAM technology is based on a variety of sensors. The most common ones are: Laser SLAM, which uses LiDAR as the main sensor, builds an environmental map by measuring the distance between the laser and the surface of the object; the other is visual SLAM, which uses a camera as the main sensor, and estimates the camera's motion and builds the three-dimensional structure of the environment by analyzing image sequences. Visual SLAM uses a camera as a sensor. Compared with laser SLAM, it is low-cost and easy to popularize. Because the camera is small and light, it is suitable for installation on devices of various sizes.

[0003] The visual SLAM process mainly includes sensor information reading, front-end visual odometer, back-end nonlinear optimization, loop detection and mapping. These steps work together to achieve simultaneous positioning and map construction of the environment. The performance of the entire system depends on the feature extraction and matching capabilities of the front-end, the optimization algorithm of the back-end, the accuracy of loop detection, and the robustness of map construction. In the development or application process of the entire visual SLAM algorithm, the adjustment of algorithm parameters is an indispensable step. The choice of parameters directly affects the accuracy of visual SLAM positioning. However, the parameters used by most current visual SLAM systems are usually empirical values ​​obtained through multiple experiments, and fixed configurations are used in practical applications. When faced with changes in sensors or scenes, the parameters need to be readjusted, which may be a relatively time-consuming and difficult task for non-professional users. Summary of the invention

[0004] The present invention aims to provide a DDPG visual SLAM parameter adaptation algorithm integrating an attention mechanism to solve the problem that when visual SLAM works in a new environment, parameters need to be reconfigured, but visual SLAM cannot adaptively configure parameters and needs to be manually reconfigured.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: a DDPG visual SLAM parameter adaptation algorithm integrating attention mechanism, comprising the following steps:

[0006] S1. Select three parameters of ORB-SLAM3 and set the value range to construct the action space;

[0007] S2. Use ORB-SLAM3 to estimate the mean square error of the pose and construct the reward function.

[0008] S3, input the video frame into the convolutional neural network (CNN) of DDPG, extract features from the video frame, and use it to construct the state space;

[0009] S4, input the feature information extracted in step S3 into a gated recurrent neural network (GRU) to extract timing information and context information;

[0010] S5, input the information output from step S4 into Transformer to obtain the long-distance dependency of the information;

[0011] S6. Use a fully connected neural network to calculate the content of step S5 to obtain the action, that is, the parameters for adaptive adjustment.

[0012] Furthermore, the three parameters of ORB-SLAM3 selected in step S1 are optimization parameters including a scale factor (M1), a number of pyramid levels (M2) and a nearest neighbor threshold (M3), and the value ranges of these three parameters are M1∈{1.10, ..., 1.40}, M2∈{2, ..., 8} and M3∈{0.60, ..., 0.90}, respectively.

[0013] Furthermore, the step S2 constructs a reward function: the root mean square error of the pose calculated for each image frame is used as the signal e returned by the environment to the agent during each system operation, the reward value is r, and the reward function is designed as follows: r=1-e.

[0014] Furthermore, the step S3 uses a convolutional neural network (CNN) to extract features from the image: input the 84×84 video frame into the convolution, that is, A∈R 84×84×C Where C is the number of input channels, the first layer of convolution is shown in formula (1), where Conv(·) represents the convolution operation, W1 is the weight of the first layer of convolution, b1 is the bias term, and the output size is A1∈R 20×20×32 ; The second layer of convolution formula (2) is shown, where W2 is the weight of the second layer of convolution, b2 is the bias term, and the output size is A2∈R 9×9×64 ; The third layer of convolution is shown in formula (3), where W3 is the weight of the third layer of convolution, b3 is the bias term, and the output size is A3∈R 7 ×7×64 Finally, the output of the convolutional layer is reshaped into a one-dimensional vector, as shown in formula (4), where Reshape(·) represents the reshape operation.

[0015] A1=ReLU(Conv(A,W1)+b1) (1)

[0016] A2=ReLU(Conv(A1,W2)+b2) (2)

[0017] A3=ReLU(Conv(A2,W3)+b3) (3)

[0018] A reshaped =Reshape(A3) (4).

[0019] Furthermore, step S4 extracts timing information and context information: these feature sequences are input into the gated recurrent unit (GRU) network, and the reshaped results are input into the double-layer GRU. Assume that the hidden state of the GRU is H (1) and H (2) , then the update rules of GRU are shown in formulas (5) and (6), A reshaped,t is the tth element of the reshaped vector, H t-1 is the hidden state at the previous time step,

[0020] H t (1) =GRU(A reshaped,t ,H t-1 (1) ) (5)

[0021] H t (2) =GRU(H t (1) ,H t-1 (2) ) (6).

[0022] Furthermore, the step S5 obtains the long-distance dependency of information: the data processed by the GRU network is input into the Transformer model, the output of the double-layer GRU is used as the input of the Transformer, and the output of the Transformer is Y. Then the update rule of the Transformer is shown in (7). The Transformer operation is represented by a function Transformer(·),

[0023] Y=Transformer(H (2) ) (7).

[0024] Furthermore, the step S6 obtains the action of passing the output of the Transformer to a fully connected neural network (FCNN), and the weight matrix W of the fully connected layer fc , the bias term is b fc , the final output is shown in formula (8):

[0025] Output = FC (Y, W fc )+b fc (8).

[0026] Further, after step S4 is completed, the loss function optimization is performed, and the loss function optimization includes the loss functions of the Critic network and the Actor network. The loss function of the Critic network usually uses the mean square error (MSE) to measure the difference between the Q value (action value) predicted by the Critic network and the target Q value. The target Q value is calculated according to the Bellman equation, where y i is the target Q value, r i is the immediate reward, γ is the discount factor, Q′ is the target Critic network, μ′ is the target Actor network, s i+1 is the next state, θ Q′ and θ μ′ are the parameters of the target Critic and target Actor networks, respectively, expressed as:

[0027] y i =r i +γQ′(s i+1 ,μ′(s i+1 |θ μ′ )|θ Q′ ) (9)

[0028] The loss function L of the Critic network can be expressed as:

[0029]

[0030] The parameter update of the Actor network is to maximize the Q value of the Critic network output through gradient ascent, where μ(s i |θ μ ) is the state s of the Actor network at time i i The action output, θ μ is the parameter of the Actor network, and the Actor's loss function J can be expressed as:

[0031]

[0032] The beneficial effects of the present invention are as follows: (1) The existence of the loss function enables the model to optimize parameters through continuous iteration, so as to better fit the training data and improve the accuracy of prediction. It can also effectively constrain the complexity of the model and prevent the model from overfitting on the training data. Selecting different loss functions can make the model better adapt to the needs of specific tasks; (2) The algorithm adaptively configures parameters through reinforcement learning and combines multiple advanced deep learning technologies to achieve accurate extraction and processing of image features, thereby improving learning performance and efficiency; (3) The algorithm can more accurately adapt the parameters of visual SLAM and improve trajectory accuracy without the help of auxiliary equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of a DDPG visual SLAM parameter adaptation algorithm integrating an attention mechanism of the present invention;

[0034] Figure 2 for Figure 1 Schematic diagram of the mean square error of pose estimation;

[0035] Figure 3 A schematic diagram of the DDPG network structure of a DDPG visual SLAM parameter adaptation algorithm integrating an attention mechanism in the present invention;

[0036] Figure 4 This is the fixed scene for initializing MH01 in Example 1;

[0037] Figure 5 It is the trajectory diagram completed by the interaction between DDPG and ORB-SLAM3 in Example 1;

[0038] Figure 6 A comparison chart of the trajectory data and the original data in Example 1;

[0039] Figure 7 This is the fixed scene for initializing room5 in Example 2;

[0040] Figure 8 It is the trajectory diagram completed by the interaction between DDPG and ORB-SLAM3 in Example 2;

[0041] Fig. 9 A comparison chart of the trajectory data and the original data in Example 2;

[0042] Fig.10 The pose trajectory obtained using the present invention on the Euroc MAV dataset;

[0043] Fig.11 The pose trajectory obtained using the present invention on the TUM-VI dataset. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention.

[0046] The present invention is basically as follows Figure 1-Figure 11 As shown: The present invention provides a DDPG visual SLAM parameter adaptation algorithm integrating an attention mechanism: comprising the following steps:

[0047] S1. Select three parameters of ORB-SLAM3 and set the value range. The optimization parameters include scale factor (M1), number of pyramid layers (M2) and nearest neighbor threshold (M3). The value ranges of these three parameters are M1∈{1.10, ..., 1.40}, M2∈{2, ..., 8} and M3∈{0.60, ..., 0.90} respectively.

[0048] S2. Use ORB-SLAM3 to estimate the mean square error of the pose, and use the root mean square error (RMSE) of the pose calculated for each image frame as the signal e returned by the environment to the agent during each system operation. The reward value is r, and the reward function is designed as follows: r = 1-e, such as Figure 2 As shown;

[0049] S3. Input the video frame into the convolutional neural network (CNN) of DDPG. The DDPG network structure is as follows: Figure 3 As shown, the video frame is feature extracted and the 84×84 video frame is input into the convolution, that is, A∈R 84×84×C Where C is the number of input channels, the first layer of convolution is shown in formula (1), where Conv(·) represents the convolution operation, W1 is the weight of the first layer of convolution, b1 is the bias term, and the output size is A1∈R 20×20×32 ; The second layer of convolution is shown in formula (2), where W2 is the weight of the second layer of convolution, b2 is the bias term, and the output size is A2∈R 9×9×64 ; The third layer of convolution is shown in formula (3), where W3 is the weight of the third layer of convolution, b3 is the bias term, and the output size is A3∈R 7×7×64 Finally, the output of the convolutional layer is reshaped into a one-dimensional vector, as shown in formula (4), where Reshape(·) represents the reshape operation.

[0050] A1=ReLU(Conv(A,W1)+b1) (1)

[0051] A2=ReLU(Conv(A1,W2)+b2) (2)

[0052] A3=ReLU(Conv(A2,W3)+b3) (3)

[0053] Areshaped =Reshape(A3) (4);

[0054] S4: Input the feature information extracted from step S3 into the gated recurrent unit (GRU) network, and input the reshaped result into the double-layer GRU. Assume that the hidden state of GRU is H (1) and H (2) , then the update rules of GRU are shown in formulas (5) and (6), A reshaped,t is the tth element of the reshaped vector, H t-1 is the hidden state of the previous time step. It controls the flow of information through a gating mechanism and effectively captures the temporal relationship and contextual information between features.

[0055] H t (1) =GRU(A reshaped,t ,H t-1 (1) ) (5)

[0056] H t (2) =GRU(H t (1) ,H t-1 (2) ) (6);

[0057] S5. Input the data processed by the GRU network into the Transformer model. The output of the double-layer GRU is used as the input of the Transformer. The output of the Transformer is Y. The update rule of the Transformer is shown in (7). The Transformer operation is represented by a function Transformer(·). Transformer is a model based on the self-attention mechanism. It can globally model the entire sequence data. The self-attention mechanism allows the model to consider all elements in the sequence when processing each element of the sequence, so that it can capture long-distance dependencies.

[0058] Y=Transformer(H (2) ) (7);

[0059] S6. Pass the output of Transformer to a fully connected neural network (FCNN). The weight matrix W of the fully connected layer fc , the bias term is b fc , the final output is shown in formula (8). This network consists of multiple fully connected layers, which are used to integrate the features extracted and processed previously to perform the final prediction task.

[0060] Output = FC (Y, W fc )+b fc (8).

[0061] Embodiment 1:

[0062] The MH01 sequence in the Euroc MAV data set is used as the initial data of Example 1, which includes the following steps:

[0063] (1) Enter the command to call the MH01 sequence in the Ubuntu system terminal;

[0064] (2) Initialize MH01, the scene will be fixed as follows Figure 4 As shown, after waiting for DDPG to start, the interaction between DDPG and ORB-SLAM3 begins;

[0065] (3) After DDPG and ORB-SLAM3 have interacted, you can see the following Figure 5 trajectories;

[0066] (4) Use the EVO evaluation tool to Figure 5 The running trajectory data is evaluated with the original data, and the results are as follows Figure 6 shown.

[0067] The experimental results using the Euroc MAV and dataset of the present invention are compared with those of the ORB-SLAM3, DSO, and DSM methods, and the results are shown in the following table. DSO refers to Direct Sparse Odometry, which is a technology used for real-time camera positioning; DSM refers to Direct Sparse Mapping, which directly uses the original pixel intensity values ​​of the image to estimate camera motion and reconstruct the environment.

[0068] Table 1 Root mean square error of Euroc MAV dataset, RMSE / m

[0069]

[0070] From the experimental results, we can see that our method has the lowest calculated mean square error and the best effect.

[0071] Embodiment 2:

[0072] The room5 sequence in the TUM-VI data set is used as the initial data of Example 2, which includes the following steps:

[0073] (1) Enter the command to call the room5 sequence in the Ubuntu system terminal;

[0074] (2) Initialize room5, the scene will be fixed as follows Figure 7 As shown, after waiting for DDPG to start, the interaction between DDPG and ORB-SLAM3 begins;

[0075] (3) After DDPG and ORB-SLAM3 have interacted, you can see the following Figure 8 trajectories;

[0076] (4) Use the EVO evaluation tool to Figure 8 The running trajectory data is evaluated with the original data, and the results are as follows Fig. 9 As shown;

[0077] The experimental results of the TUM-VI dataset of the present invention are compared with the root mean square error results of the ORB-SLAM3, ROVIO, and VINS-Mono methods. ROVIO (Robust Visual Inertial Odometry) is a VIO system developed by the Robotic Systems Laboratory of the Swiss Federal Institute of Technology in Zurich (ETH Zurich); the VINS-Mono system focuses on using monocular camera information to achieve navigation and positioning.

[0078] Table 2 Root mean square error of TUM-VI dataset, RMSE / m

[0079]

[0080] From the analysis of experimental results and trajectories, the method in this paper outperforms other methods on the TUM-VI dataset sequence.

[0081] The pose trajectories obtained by the present invention on the challenging Euroc MAV and TUM-VI datasets are as follows: Fig.10 and Fig.11 As shown, it can be seen that the present invention can adapt the parameters of visual SLAM more accurately and improve the trajectory accuracy without the help of auxiliary equipment.

Claims

1. A DDPG visual SLAM parameter adaptation algorithm integrating attention mechanism, characterized by: The following steps are involved: S1. Select three parameters of ORB-SLAM3 and set the value range to construct the action space; S2. Use ORB-SLAM3 to estimate the mean square error of the pose and construct the reward function. S3, input the video frame into the convolutional neural network (CNN) of DDPG, extract features from the video frame, and use it to construct the state space; S4, input the feature information extracted in step S3 into a gated recurrent neural network (GRU) to extract timing information and context information; S5, input the information output from step S4 into the Transformer module to obtain the long-distance dependency of the information; S6. Use a fully connected neural network to calculate the content of step S5 to obtain the action, that is, the parameters for adaptive adjustment.

2. A DDPG visual SLAM parameter adaptation algorithm integrating attention mechanism according to claim 1, characterized in that: The three parameters of ORB-SLAM3 selected in the step S1 are optimization parameters including a scale factor (M1), a number of pyramid layers (M2) and a nearest neighbor threshold (M3), and the value ranges of these three parameters are M1∈{1.10, ..., 1.40}, M2∈{2, ..., 8} and M3∈{0.60, ..., 0.90}, respectively.

3. A DDPG visual SLAM parameter adaptation algorithm integrating attention mechanism according to claim 2, characterized in that: The step S2 constructs a reward function: the root mean square error (RMSE) of the pose calculated for each image frame is used as the signal e returned by the environment to the agent during each system operation, the reward value is r, and the reward function is designed as follows: r = 1-e.

4. A DDPG visual SLAM parameter adaptation algorithm integrating attention mechanism according to claim 3, characterized in that: The step S3 uses a convolutional neural network (CNN) to extract features from the image: input a 84×84 video frame into the convolution, i.e., A∈R 84×84×C Where C is the number of input channels, the first layer of convolution is shown in formula (1), where Conv(·) represents the convolution operation, W1 is the weight of the first layer of convolution, b1 is the bias term, and the output size is A1∈R 20×20×32 ; The second layer of convolution is shown in formula (2), where W2 is the weight of the second layer of convolution, b2 is the bias term, and the output size is A2∈R 9×9×64 ; The third layer of convolution is shown in formula (3), where W3 is the weight of the third layer of convolution, b3 is the bias term, and the output size is A3∈R 7×7×64 Finally, the output of the convolutional layer is reshaped into a one-dimensional vector, as shown in formula (4), where Reshape(·) represents the reshape operation. A1=ReLU(Conv(A,W1)+b1) (1) A2=ReLU(Conv(A1,W2)+b2) (2) A3=ReLU(Conv(A2,W3)+b3) (3) A reshaped =Reshape(A3) (4)。 5. A DDPG visual SLAM parameter adaptation algorithm integrating attention mechanism according to claim 4, characterized in that: The step S4 extracts the timing information and context information: these feature sequences are input into the gated recurrent unit (GRU) network, and the reshaped results are input into the double-layer GRU. Assume that the hidden state of the GRU is H (1) and H (2) , then the update rules of GRU are shown in formulas (5) and (6), A reshaped,t is the tth element of the reshaped vector, H t-1 is the hidden state at the previous time step, H t (1) =GRU(A reshaped,t ,H t-1 (1) ) (5) H t (2) =GRU(H t (1) ,H t-1 (2) ) (6)。 6. A DDPG visual SLAM parameter adaptation algorithm integrating attention mechanism according to claim 5, characterized in that: The step S5 obtains the long-distance dependency of information: the data processed by the GRU network is input into the Transformer model, the output of the double-layer GRU is used as the input of the Transformer, and the output of the Transformer is Y. Then the update rule of the Transformer is shown in (7). The Transformer operation is represented by a function Transformer(·), Y=Transformer(H (2) ) (7)。 7. A DDPG visual SLAM parameter adaptation algorithm integrating attention mechanism according to claim 6, characterized in that: The step S6 obtains the action of passing the output of the Transformer to a fully connected neural network (FCNN), and the weight matrix W of the fully connected layer fc , the bias term is b fc , the final output is shown in formula (8): Output=FC(Y,W fc )+b fc (8)。 8. A DDPG visual SLAM parameter adaptation algorithm integrating attention mechanism according to claim 7, characterized in that: After step S4, loss function optimization is performed. The loss function optimization includes the loss functions of the Critic network and the Actor network. The loss function of the Critic network usually uses the mean squared error (MSE) to measure the difference between the Q value (action value) predicted by the Critic network and the target Q value. The target Q value is calculated according to the Bellman equation, where y i is the target Q value, r i is the immediate reward, γ is the discount factor, Q′ is the target Critic network, μ′ is the target Actor network, s i+1 is the next state, θ Q′ and θ μ′ are the parameters of the target Critic and target Actor networks, respectively, expressed as: y i =r i +γQ′(s i+1 ,μ′(s i+1 |θ μ′ )|θ Q′ ) (9) The loss function L of the Critic network can be expressed as: The parameter update of the Actor network is to maximize the Q value of the Critic network output through gradient ascent, where: μ(s i |θ μ ) is the state s of the Actor network at time i i The action output, θ μ is the parameter of the Actor network, and the Actor's loss function J can be expressed as:

Images