A UAV visual odometry navigation and positioning method based on attention mechanism

Abstract

The present invention belongs to the field of navigation and positioning technology, and relates to a method for navigation and positioning of a drone visual odometry based on an attention mechanism. The navigation and positioning method relies on the GoogLeNet convolutional neural network model. First, a vehicle-mounted visual odometry model based on the GoogLeNet convolutional layer structure is trained based on an existing public dataset of urban and suburban roads on vehicles. Then, a special mission target model is constructed through a direct transfer learning method. The drone target dataset is introduced for network model training. Finally, a relationship model between image features and drone posture in a drone mission environment is established, completing the positioning of the drone visual odometry based on the GoogLeNet convolutional layer structure. The method can still achieve relatively ideal results even with only a small amount of drone image data.

Description

Technical Field

[0001] The present invention belongs to the field of navigation and positioning technology, and relates to an unmanned aerial vehicle visual odometer navigation and positioning method based on an attention mechanism. Background Art

[0002] Part 1: When a moving object, such as an unmanned platform or robot, enters an unknown environment, it has no awareness of its position or surroundings and can only obtain information through various sensors. Humans acquire a vast amount of information through their visual system. Moving objects are assumed to observe and understand the surrounding objects and environment through their eyes, just like humans do, knowing their location and exploring the world. With the development of computer vision (CV), egomotion estimation using rich information obtained from attached visual sensors has gradually become possible in recent decades. Visual sensors provide the richest, most comprehensive, and most intuitive information, are relatively low-cost, and have a wide range of detection applications. The process of estimating camera egomotion using visual sensors is called visual odometry (VO). VO has been widely used in many types of unmanned platforms and robots, in the air, on land, and underwater.

[0003] The current mainstream visual odometry navigation and positioning methods rely on the geometric relationships of images for calculations. They require stable, textured features, are easily affected by changes in light and weather, and are very sensitive to camera parameters. This makes them unsuitable for drones operating outdoors for extended periods of time and over large areas. Visual odometry solutions based on convolutional neural network structures are mostly targeted at ground-based vehicular environments. However, drone images are mostly downward-facing, covering a wide range of content, with a high density of diverse categories. Their flight paths are also more flexible than those of cars on urban roads. This makes traditional visual odometry navigation and positioning solutions based on ground-based vehicular environments less suitable for drones. Summary of the Invention

[0004] The purpose of the present invention is to address the technical defects of poor applicability of traditional visual odometry navigation and positioning methods for ground vehicle environments in drone mission scenarios, and propose a drone visual odometry navigation and positioning method based on an attention mechanism. Relying on the GoogLeNet convolutional neural network model, firstly, a vehicle-mounted visual odometry model based on the GoogLeNet convolutional layer structure is trained based on a relatively mature and complete vehicle-mounted public data set, and then a drone visual odometry model is constructed by a direct push transfer learning method, and a drone target data set is introduced for network model training. Finally, a relationship model between image features and drone postures in a drone mission environment is established, and the positioning of the drone visual odometry based on the GoogLeNet convolutional layer structure is completed. The method optimizes the convolutional neural network based on the attention mechanism, adopts the vehicle-mounted visual odometry and combines it with transfer learning to migrate it to the drone mission scenario, so that a relatively ideal effect can still be achieved even with only a small amount of drone image data.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The method for visual odometry navigation and positioning of a UAV based on an attention mechanism, relying on a convolutional layer structure based on GoogLeNet, comprises the following steps:

[0007] S1. Collect and create a UAV vision-pose dataset, specifically:

[0008] S11. Use drones to collect videos of the ground at different altitudes, speeds, and trajectories;

[0009] S12, using a structure-from-motion estimation algorithm to offline solve the camera pose and mileage motion information to obtain solved pose sequence data;

[0010] S13. Create a FLYING dataset containing time-series images and pose sequence data, and save the images and pose labels separately for model training.

[0011] S2. Obtain the vehicle environment dataset and execute S12 and S13 to transform the public dataset into a form that is homogeneous with the FLYING dataset;

[0012] S3. Build the overall network architecture of the on-board visual odometry model, including designing the convolutional layer structure of the encoding layer, image position encoding, attention mechanism to solve the relationship between sequences, and setting the loss function;

[0013] S4, model training, specifically: setting the sample sequence length and the sample overlap value, and repeating the image acquisition one at a time in order to produce the next sample;

[0014] When S4 is implemented, the sample sequence length is set to 5 or 6, that is, each sample consists of 5 or 6 sequence images; the sample overlap sampling value is set to 1; the S4 is adapted to the recurrent convolutional neural network structure and is within the memory range of the desktop computer;

[0015] S5. Test and evaluate the vehicle visual odometry model based on the vehicle environment dataset. Test the model saved during training on the test dataset to evaluate model performance. Select the model with the best test results for subsequent transfer learning.

[0016] S6. Use the direct transfer learning method to train the UAV visual odometry model and obtain the UAV visual odometry network structure model based on the attention mechanism.

[0017] The drone collection described in S11 is specifically: collecting straight, turning, and lifting movement picture data respectively and moving in the scene for as long as possible to ensure that the collected original data is obtained by traversing the drone's visual posture.

[0018] The first three digits of the calculated data in S12 represent the position coordinates, and the last four digits represent the rotation quaternion.

[0019] The vehicle-mounted environment dataset described in S2 is a public vehicle-mounted urban and suburban road dataset.

[0020] The overall network architecture described in S3 includes CNN, a multi-head attention unit and a forward transmission unit; the CNN described in S3 includes a convolutional layer; the convolutional layer includes a coding layer, and the coding layer is a coding layer of a flownet structure.

[0021] The convolutional layers in the overall network architecture described in S3 are used for feature extraction, learning the relationship between sequences and balancing the weight distribution; the relationship between sequences is no longer learned in sequence according to time labels, but the temporal relationship is identified as position encoding before learning, which is specifically implemented through the CNN-attention module. The structure of each encoder layer is the same, but the weights are not shared; each encoder layer consists of a multi-head attention module and a feed forward module, and each module is connected to add&norm; each feature vector extracted by the convolutional layer will be input into the stacked multi-head attention module and feed forward module to obtain an output.

[0022] The vehicle-mounted visual odometry model described in S3 relies on a convolutional layer structure based on GoogLeNet.

[0023] The loss function described in S3 is set based on the minimum mean square error criterion between the ground truth pose information and the predicted information.

[0024] S6 describes a direct transfer learning method for training the UAV visual odometry model. Specifically, a portion of the images in the FLYING dataset are selected as the validation set, a portion of the images are selected as the test set, and the remaining scene sequences are used as the training set. Based on the model and parameters saved by the unmanned vehicle visual odometry model, the same configuration file and network structure are used. After modifying the dataset path and some hyperparameters, training is carried out to obtain a UAV visual odometry network structure model based on the attention mechanism.

[0025] Beneficial effects

[0026] The present invention proposes a UAV visual odometry navigation and positioning method based on the attention mechanism, which has the following advantages compared with the existing technology:

[0027] 1. The attention-based visual odometry navigation and positioning method relies on a network structure model, Attention VO, which takes a sequence of images as input, calculates the camera pose between them, extracts feature vectors between images using a convolutional neural network, and connects the attention mechanism in series to learn the constraints between sequences.

[0028] 2. The attention-based visual odometry navigation and positioning method trains and tests the network model. The displacement estimation accuracy is much higher than the classic visual odometry method VISO2-M, but lower than ORB SLAM with optimization and detection. It does not require camera parameterization, feature matching, or map alignment, and does not suffer from feature loss when scene transformations accelerate.

[0029] 3. The attention-based visual odometry navigation and positioning method uses transfer learning to transfer the visual odometry model to UAV mission scenarios, achieving relatively ideal results even with limited UAV image data. In practical applications, the vehicle environment dataset model can be pre-trained as a general model, and then transfer learning can be performed using collected UAV aerial images to specifically train the UAV visual odometry model for the desired mission scenario.

[0030] 4. The visual odometry navigation and positioning method based on the attention mechanism has the advantages of low computational complexity and high speed. It is superior to VISO2M and ORB SLAM in terms of algorithm time consumption. When the drone captures images at a frequency of 20Hz or lower, the model can meet the real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is an example of the pose calculation result of 6 adjacent frames of images in the present invention;

[0032] Figure 2 The overall network architecture of the vehicle-mounted visual odometry model is built for S3 of the present invention;

[0033] Figure 3 Build the convolutional layer structure in the vehicle-mounted visual odometry model for S3 of the present invention;

[0034] Figure 4 Build the CNN-attention in the vehicle-mounted visual odometry model for S3 of the present invention to solve the relationship between sequences;

[0035] Figure 5 The calculation process of the multi-head attention mechanism in the vehicle visual odometry model built for S3 of the present invention;

[0036] Figure 6 This is the direct push transfer learning training process of the present invention;

[0037] Figure 7 Some video screenshots obtained during the specific implementation of the present invention;

[0038] Figure 8 The output content corresponding to the third step of data preprocessing of the present invention;

[0039] Figure 9 This is a diagram showing the composition of the data set of the present invention;

[0040] Figure 10 This is a training error curve during the specific implementation of the present invention. DETAILED DESCRIPTION

[0041] The following is a further explanation and detailed description of the UAV visual odometry positioning method based on the attention mechanism of the present invention with reference to the accompanying drawings and embodiments.

[0042] Example 1

[0043] The described method for unmanned aerial vehicle (UAV) visual odometry navigation and positioning based on an attention mechanism relies on a GoogLeNet convolutional neural network model and combines an existing online vehicle-mounted public urban and suburban road dataset (including 11 scenes with a total of more than 20,000 images) with a self-collected UAV aerial photography dataset (including 15 scenes with a total of 8,740 images). First, a vehicle-mounted visual odometry model based on a GoogLeNet convolutional layer structure is trained based on a relatively mature and complete vehicle-mounted public dataset. Then, a UAV visual odometry model is constructed through a direct transfer learning method. A UAV target dataset is introduced for network model training. Finally, a relationship model between image features and UAV posture in a UAV mission environment is established to complete the positioning of the UAV visual odometry based on the GoogLeNet convolutional layer structure.

[0044] A drone visual odometry navigation and positioning method based on an attention mechanism, relying on a convolutional layer structure based on GoogLeNet, includes the following steps:

[0045] S1. Collect and create a UAV vision-pose dataset, specifically:

[0046] S11. Use drones to collect videos of the ground at different altitudes, speeds, and trajectories;

[0047] S12, using a structure-from-motion estimation algorithm to offline solve the camera pose and mileage motion information to obtain solved pose sequence data;

[0048] S13. Create a FLYING dataset containing time-series images and pose sequence data, and save the images and pose labels separately for model training.

[0049] S2. Obtain the vehicle environment dataset and execute S12 and S13 to transform the public dataset into a form that is homogeneous with the FLYING dataset;

[0050] S3. Build an on-board visual odometry model, including designing the convolutional layer structure of the encoding layer, image position encoding, attention mechanism to solve the relationship between sequences, and setting the loss function;

[0051] S4, model training, specifically: setting the sample sequence length and the sample overlap value, and repeating the image acquisition one at a time in order to produce the next sample;

[0052] When S4 is implemented, the sample sequence length is set to 5 or 6, that is, each sample consists of 5 or 6 sequence images; the sample overlap sampling value is set to 1; the S4 is adapted to the recurrent convolutional neural network structure and is within the memory range of the desktop computer;

[0053] S5. Test and evaluate the vehicle visual odometry model based on the vehicle environment dataset. Test the model saved during training on the test dataset to evaluate model performance. Select the model with the best test results for subsequent transfer learning.

[0054] S6. Use the direct transfer learning method to train the UAV visual odometry model and obtain the UAV visual odometry network structure model based on the attention mechanism.

[0055] The drone collection described in S11 is specifically: collecting straight, turning, and lifting movement picture data respectively and moving in the scene for as long as possible to ensure that the collected original data is obtained by traversing the drone's visual posture.

[0056] The first three digits of the calculated data in S12 represent the position coordinates, and the last four digits represent the rotation quaternion.

[0057] The vehicle-mounted environment dataset described in S2 is a public vehicle-mounted urban and suburban road dataset.

[0058] The overall network architecture described in S3 includes CNN, a multi-head attention unit and a forward transmission unit; the CNN described in S3 includes a convolutional layer; the convolutional layer includes a coding layer, and the coding layer is a coding layer of a flownet structure.

[0059] The convolutional layer in the overall network architecture described in S3 is used for feature extraction, learning the relationship between sequences and balancing the weight distribution; the relationship between sequences is no longer learned in sequence according to time labels, but the temporal relationship is identified as a position code and then learned.

[0060] The vehicle-mounted visual odometry model described in S3 relies on a convolutional layer structure based on GoogLeNet.

[0061] The loss function described in S3 is set based on the minimum mean square error criterion between the ground truth pose information and the predicted information.

[0062] S6 describes a direct transfer learning method for training the UAV visual odometry model. Specifically, a portion of the images in the FLYING dataset are selected as the validation set, a portion of the images are selected as the test set, and the remaining scene sequences are used as the training set. Based on the model and parameters saved by the unmanned vehicle visual odometry model, the same configuration file and network structure are used. After modifying the dataset path and some hyperparameters, training is carried out to obtain a UAV visual odometry network structure model based on the attention mechanism.

[0063] In specific implementation, the drone visual odometry navigation and positioning method based on the attention mechanism is based on the GoogLeNet convolutional layer structure and includes the following steps:

[0064] Step 1: Collect and create UAV vision-pose dataset;

[0065] Step 1.1: Data Collection: Use a drone to capture ground-level video at different altitudes, speeds, and trajectories. The drone's gimbal maintains a fixed angle relative to the drone during capture. The scene includes forests, buildings of various shapes, vehicles, and moving objects. Some scenes also experience changes in lighting. Data is collected for straight driving, turning, and ascending and descending movements. The drone's movement within the scene is maintained for as long as possible to ensure the richness of the raw data.

[0066] Step 1.2: Use the motion estimation structure algorithm to calculate the camera pose and mileage information offline. The first three digits of the data represent the position coordinates, and the last four digits represent the rotation quaternion q = q0 + q1i + q2j + q3k. The result of the pose solution of the adjacent 6 frames is shown as follows: Figure 1 As shown:

[0067] Step 1.3: Dataset creation: Create a dataset named FLYING and create an image sequence folder "Images" and a pose folder "Poses" in it to save the corresponding images and converted pose labels for model training.

[0068] Step 2: Download the KITTI dataset (including 11 scenes and more than 30,000 images) for urban and suburban roads. Similar to steps 1.2 and 1.3, convert the KITTI dataset into a form that is homogeneous with the FLYING dataset.

[0069] Step 3: Network design;

[0070] Overall architecture: The overall architecture of the network is as follows Figure 2 As shown, the input data consists of sample time-series images. Before entering the network structure, two adjacent images are stacked to form a tensor. First, a convolutional layer extracts features, reducing the features of the high-dimensional image. The feature vector is then fed into an attention mechanism to learn sequential constraints. The output is the camera's pose information with seven degrees of freedom (DOF). The first three degrees of freedom represent displacement, and the last four represent the attitude change as a rotational quaternion. The network is based on a self-attention mechanism and incorporates a multi-head attention mechanism to balance weight distribution. Instead of performing computations sequentially based on time labels, the model identifies temporal relationships as positional encodings, thereby learning relationships between sequences.

[0071] Step 3.1, convolutional layer structure design: Figure 3 As shown in the figure, the encoding layer of the FlowNet architecture is used as the convolutional layer in this architecture. The CNN parameters are shown in the figure above. The size of the convolution kernel is gradually reduced from 7×7, 5×5, to 3×3 to capture small local features in the image. The reduction in feature map size indicates the collection of more abstract features. The number of convolution kernels is gradually increased from 128, 512, to 1024. The increase in the number of feature maps after the convolution layer indicates the collection of richer features. Two images are stacked and input into the CNN in sequence to obtain a feature vector representing the geometric relationship between the images. This feature vector is then stretched one-dimensionally and input into the subsequent attention mechanism.

[0072] Step 3.2, Image Position Encoding: Since the attention mechanism itself does not consider the order of elements, position encoding is required to add additional information to each element in the sequence so that the model can utilize the position information of the elements. In the visual odometry task, the images extracted from the video stream have a time series relationship. Preserving the mutual sequence relationship between images is the process of encoding features. The images are input into the network structure in sequence to obtain their corresponding relative camera poses. The image order is identified by encoding the image position information:

[0073]

[0074] Where angle is the angle calculation, hid_idx is the serial position of the image, and d_hid is the number of dimensions after encoding. The sin and cos values ​​of the angle are solved in each image and combined as the position code of the image.

[0075] Step 3.3, attention mechanism solves the relationship between sequences: Figure 4 As shown in the figure, the encoded features of adjacent images, along with the positional encoding information, are fed into the attention mechanism to learn constraints between sequences. The attention layer consists of four stacked components called encoders. Each encoder layer has the same structure, but the weights are not shared. Each encoder layer consists of a multi-head attention module and a feed-forward module, each followed by an add & norm function. Each feature vector extracted by the convolutional layer is fed into the stacked multi-head attention module and feed-forward module to produce an output.

[0076] In the multi-head attention mechanism, there are multiple query, key, and value weight matrices. These matrices are randomly initialized and map the encoded input elements to different subspaces. The calculation process of the multi-head attention mechanism is as follows Figure 5 As shown, n_head is set to 4. Each header in the mechanism has an independent query, key, and value weight matrix. The input is copied into n_head parts and fed into different self-attention mechanisms. The matrices obtained from the self-attention output are concatenated and multiplied by the weight matrix ω0 to obtain the output vector z, which is then transmitted backward. The code for the forward transmission process is excerpted as follows:

[0077]

[0078]

[0079] After the multi-head attention operation, it passes through the add&norm module. The output of this module when inputted is x is:

[0080] y = LayerNorm(x + SubLayer(x))

[0081] The image is then fed into a fully connected feedforward network and the add&norm module for dimensionality reduction. Each element in the input sequence undergoes the same feedforward neural network structure, but independently. Finally, the output dimension is mapped to 7 dimensions, representing the change in camera pose between images.

[0082] Step 3.4, loss function setting: decompose the pose information Y into displacement and rotation The predicted pose obtained by the neural network is The loss function for M sequences of length T is calculated by the mean square error:

[0083]

[0084] Among them, ||·||2 is the 2-norm, and β is the proportional factor of the balance weight. The neural network parameters are calculated to minimize the error between the ground truth pose information and the predicted information, thereby training to obtain the optimal parameters w *

[0085] In step 4, model training, set the sample sequence length to 5 or 6, meaning each sample consists of 5 or 6 sequential images. Set the sample overlap value to 1, and repeat the sampling sequence one image at a time to produce the next sample. To adapt to the recurrent convolutional neural network structure and operate normally within the memory range of a desktop computer, use center cropping (CenterCrop) to process the images, reducing the GPU load and significantly improving the computation speed. Use the Adam optimizer with a learning rate of 0.00001, n-head of 4, the number of encoder layers of 4, the value and key dimensions of 256, and the coupling coefficient in the loss function of 100. The CNN parameters are initialized using the trained flownet model parameters, while the remaining weights are randomly initialized, and the bias is initialized from 0.

[0086] Step 5: Model testing and evaluation based on the vehicle environment dataset. The models saved during training are tested on the test set to evaluate model performance, and the models with better test results are selected for subsequent transfer learning. Set the length of the image sequence processed each time during testing to 6, the overlap value to 5, and place the samples into the trained network model in order. Take the first five images of each image sequence and align them with the last five images. Use the neural network model to predict the relative position of the camera. For example, for the image sequence 0, 1, 2, 3, 4, 5, convert the images into 0, 1, 2, 3, 4 and 1, 2, 3, 4, 5 according to the method defined in the data processing file, and combine them into image pairs 0-1, 1-2, 2-3, 3-4, and 4-5.

[0087] Step 6: Figure 6We used a direct transfer learning approach to train a UAV visual odometry model. We selected a subset of images from the FLYING dataset as the validation set, a subset as the test set, and the remaining scene sequences as the training set. Based on the model and parameters saved in the unmanned vehicle visual odometry model, we used the same configuration file and network structure, modified the dataset path and some hyperparameters, and then trained the model. This resulted in a new attention-based UAV visual odometry network model, called attention VO.

[0088] 1. Data collection: The drone data collected by this patent is DJI's Wuyi, which has a load camera resolution of 12.76 million pixels. The drone captures videos on the ground while traveling at different altitudes and speeds along different trajectories. The scenes include trees, buildings of various shapes, vehicles, moving objects, and even flocks of birds that occupy a large part of the image. At the same time, the acquisition process of some scenes includes changes in light. Some screenshots of the video obtained are as follows: Figure 7 shown.

[0089] 2. Data preprocessing: Refer to the data preprocessing file in the previous article and modify preprocess.py for the FLYING dataset. Preprocess the data to lay the foundation for network training in the following text. The first step is to clear unused images; the second step is to convert the representation of the pose into three-dimensional position and Euler angle; the third step is to calculate the mean and variance of the image. Due to the large number of scene sequences, Figure 8 This is the output content corresponding to the third step of data preprocessing.

[0090] like Figure 9 We constructed the target FLYING dataset and selected scene sequences 40 and 47 from the FLYING dataset as the validation set, 32, 45, and 47 as the test set, and the remaining scene sequences as the training set, totaling approximately 8,000 images. The training sample sets for both models were generated using the same method: 5 or 6 images were sampled sequentially, one at a time. Pose labels were converted to relative camera pose information between images. To ensure proper operation within the experimental platform, center-cropped images were used for training both models.

[0091] 3. Transfer learning: We selected scenes 40 and 42 from the FLYING dataset as validation sets, scenes 32 and 45 as test sets, and the remaining scene sequences as training sets. Based on the model and parameters saved in the unmanned vehicle visual odometry model, we continued to use the Adam optimization algorithm, modified the dataset path and some hyperparameters, and then started training to obtain the attention mechanism-based drone visual odometry network structure model, attention VO. During the training process, it takes about 200 epochs for the model's training loss and validation loss to converge to an appropriate range. When the model converges well, we save the corresponding network and parameters. The decline curve of the training loss during training is recorded as follows: Figure 10 As shown in the figure, curve A represents the training loss and curve B represents the validation loss. The iterative process is accompanied by a small range of loss fluctuations. This is because some scene pictures are similar to the data in the previous article.

[0092] 5. Data evaluation: The average displacement, rotation error, and processing time of the visual odometry based on the UAV test dataset are compared with the results of the traditional VISO2M model, as shown in Tables 1 and 2.

[0093] Table 1 Average displacement and rotation errors

[0094]

[0095] Table 2 Comparison of test time based on FLYING dataset (s)

[0096]

[0097] Data analysis shows that the error for scene sequence 47 is lower than for the other test scene sequences, consistent with intuitive analysis of the trajectory graph. This is primarily due to the fact that the drone's altitude changes less in scene 47. Since the KITTI dataset is a vehicle-based environment dataset and does not experience altitude variations, the visual odometry model trained using transfer learning on this dataset has poor adaptability to changes in altitude.

[0098] In terms of model testing time, the attention mechanism effectively improves the network's parallel computing capabilities and has obvious advantages in time performance. When the frequency is adjusted to around 20Hz, the attention VO model can still meet real-time requirements.

[0099] This embodiment describes in detail the data acquisition equipment, training method, and test results of a drone visual odometry design based on the attention mechanism of the present invention.

[0100] The above description is only a preferred embodiment of the present invention, and the present invention should not be limited to the contents disclosed in the embodiment and the accompanying drawings. Any equivalent or modification completed without departing from the spirit disclosed in the present invention shall fall within the scope of protection of the present invention.

Claims

1. A UAV visual odometry navigation and positioning method based on attention mechanism, characterized in that: Relying on the convolutional layer structure based on GoogLeNet, the following steps are included: S1. Collect and create a UAV vision-pose dataset, specifically: S11. Use drones to collect videos of the ground at different altitudes, speeds, and trajectories; S12, using a structure-from-motion estimation algorithm to offline solve the camera pose and mileage motion information to obtain solved pose sequence data; S13. Create a FLYING dataset containing time-series images and pose sequence data, and save the images and pose labels separately for model training. S2. Obtain the vehicle environment dataset and execute S12 and S13 to transform the public dataset into a form that is homogeneous with the FLYING dataset; S3. Build the overall network architecture of the on-board visual odometry model, including designing the convolutional layer structure of the encoding layer, image position encoding, attention mechanism to solve the relationship between sequences, and setting the loss function; S4, model training, specifically: setting the sample sequence length and the sample overlap value, and repeating the image acquisition one at a time in order to produce the next sample; S5. Test and evaluate the vehicle visual odometry model based on the vehicle environment dataset. Test the model saved during training on the test dataset to evaluate model performance. Select the model with the best test results for subsequent transfer learning. S6. Use the direct transfer learning method to train the UAV visual odometry model and obtain the UAV visual odometry network structure model based on the attention mechanism.

2. The method for visual odometry navigation and positioning of a UAV based on an attention mechanism according to claim 1, characterized in that: The drone collection described in S11 is specifically: collecting straight, turning, and lifting movement picture data respectively and moving in the scene for as long as possible to ensure that the collected original data is obtained by traversing the drone's visual posture.

3. The method for visual odometry navigation and positioning of a UAV based on an attention mechanism according to claim 1, characterized in that: The first three digits of the calculated data in S12 represent the position coordinates, and the last four digits represent the rotation quaternion.

4. The method for visual odometry navigation and positioning of a UAV based on an attention mechanism according to claim 1, characterized in that: The vehicle-mounted environment dataset described in S2 is a public vehicle-mounted urban and suburban road dataset.

5. The method for visual odometry navigation and positioning of an unmanned aerial vehicle based on an attention mechanism according to claim 1, characterized in that: The overall network architecture described in S3 includes a CNN-attention module and a forward transmission unit; the CNN-attention module described in S3 includes a convolutional layer and several encoder layers; the convolutional layer further includes an encoding layer, which is an encoding layer of a flownet structure; the number of encoder layers is greater than or equal to 2.

6. The method for visual odometry navigation and positioning of a UAV based on an attention mechanism according to claim 5, characterized in that: The convolutional layer in the overall network architecture of S3 is used for feature extraction, learning the relationship between sequences and balancing the weight distribution; the relationship between sequences is no longer learned in sequence according to the time label, but the temporal relationship is identified as a position code and then learned, which is specifically implemented through the CNN-attention module. The structure of each encoder layer is the same, but the weights are not shared; each encoder layer consists of a multi-head attention module and a feed forward module, and each module is connected to add&norm; each feature vector extracted by the convolutional layer will be input into the stacked multi-head The attention module and feed forward module obtain an output.

7. The method for visual odometry navigation and positioning of a UAV based on an attention mechanism according to claim 6, characterized in that: The vehicle-mounted visual odometry model described in S3 relies on a convolutional layer structure based on GoogLeNet.

8. The method for visual odometry navigation and positioning of a UAV based on an attention mechanism according to claim 1, characterized in that: The loss function described in S3 is set based on the minimum mean square error criterion between the ground truth pose information and the predicted information.

9. The method for visual odometry navigation and positioning of a UAV based on an attention mechanism according to claim 1, characterized in that: S6 describes a direct transfer learning method for training the UAV visual odometry model. Specifically, a portion of the images in the FLYING dataset are selected as the validation set, a portion of the images are selected as the test set, and the remaining scene sequences are used as the training set. Based on the model and parameters saved by the unmanned vehicle visual odometry model, the same configuration file and network structure are used. After modifying the dataset path and some hyperparameters, training is carried out to obtain a UAV visual odometry network structure model based on the attention mechanism.

Images