A visual position recognition method, system and device based on attention compression coding features

Abstract

The application provides a visual position recognition method based on attention compression coding features. A hierarchical database is established based on a scene three-dimensional map; and global features are extracted by using an attention mechanism and codebook compression coding based on the hierarchical database to realize online position recognition. The application is used to reduce the calculation overhead of feature extraction in offline and online stages, improve global recall accuracy, and thus guarantee the service quality of user positioning.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of visual positioning, and particularly relates to a visual position recognition method, system and device based on attention compression coding features. BACKGROUND

[0002] In recent years, with the vigorous development of microelectronics, automation technology and artificial intelligence technology, the research and application of unmanned intelligent systems have been rapidly promoted. Unmanned systems such as sweeping robots, unmanned aerial vehicles, unmanned vehicles, unmanned ships and rail transit automatic driving play an increasingly important role in military, science and technology, economy and social life, but also put forward higher requirements for the positioning, perception, planning and control of unmanned intelligent systems. Autonomous positioning, as a key technology of unmanned intelligent systems, solves the problem of "where am I" and is an essential basis and prerequisite for subsequent planning. Most of the currently widely used autonomous positioning technologies need to rely on external facilities such as Global Positioning System (GPS), magnetic nails and magnetic strips. However, GPS can only provide meter-level positioning information in open environments and is difficult to popularize to urban areas with large-scale greening or closed indoor scenes. The positioning based on magnetic nails and magnetic strips needs to modify the application scene and limits the flexibility of robot movement, and the maintenance cost also limits its wide application. Therefore, the self-positioning technology based on scene environment information has attracted widespread attention and research. According to the sensor type, it can be divided into two categories: laser radar positioning and visual positioning. Laser radar can stably perceive the three-dimensional structure information of the surrounding environment, and the laser radar positioning system has realized high-precision real-time positioning in various application scenes. However, laser radar has high cost and requires a certain layout of sensors, which restricts its further popularization and application. Compared with laser radar positioning, visual positioning technology has relatively low precision, but it is low in cost and rich in perception information, so visual sensors have become the standard configuration of many intelligent devices, and visual positioning systems have been widely used in practice.

[0003] Visual Place Recognition (VPR), as a key technology in visual positioning systems, aims to speed up the positioning in large-scale scenes. Specifically, VPR determines whether the current view of a robot or mobile device comes from a place or location that has been visited in the past. Visual place recognition technology can be considered as an image retrieval technology, that is, matching the currently observed image with the image observed at a place visited in the past, and thereby identifying and positioning the place. Obviously, the related research on visual position recognition has important significance. In the above background, how to improve the existing visual position recognition method or propose a new and more efficient visual position recognition method has become a problem to be solved and has attracted the attention of many scholars. SUMMARY

[0004] The application provides a visual position recognition method, system and device based on attention compression coding features, which reduces the calculation overhead of feature extraction in offline and online stages, improves global recall accuracy, and guarantees the service quality of user positioning.

[0005] The application is realized by the following technical solutions:

[0006] A visual position recognition method based on attention compression coding features, characterized in that it comprises

[0007] A hierarchical database is established based on a scene three-dimensional map;

[0008] Based on the hierarchical database, a global feature is extracted by using an attention mechanism and codebook compression coding to realize online position recognition.

[0009] Further, the hierarchical database is established based on a scene three-dimensional map, and specifically,

[0010] Step S1: According to the environment to be positioned, a coordinate origin is selected, and a two-dimensional rectangular coordinate system is established as a two-dimensional map;

[0011] Step S2: According to the environment to be positioned, a path is planned to cover the scene to be positioned;

[0012] Step S3: A camera is used to shoot reference images of the scene to be positioned along the trajectory;

[0013] Step S4: Point local features are extracted from the reference images, and an incremental SfM algorithm is used to establish a scene three-dimensional map;

[0014] Step S5: The mapping coefficient between the position of the reference image in the three-dimensional map and the position in the two-dimensional map is adjusted according to the trajectory, so as to automatically obtain the position of the reference image in the two-dimensional map, and the association between the reference image and the scene position is established;

[0015] Step S6: Based on the attention mechanism and codebook compression coding principle, a neural network is used to extract global features from the reference images;

[0016] Step S7: The two-dimensional position of the reference image, the global features and the point local features are stored in the database, and the establishment of the scene database is completed.

[0017] Further, the extraction of global features in step S6 is specifically,

[0018] Step S6.1: Block local features of the image are extracted by using a feature extraction module;

[0019] Step S6.2: The posterior probability of classifying the block local features to a certain vocabulary is calculated according to the correlation between the block local features and the codebook vocabulary.

[0020] Step S6.3: Calculate the posterior mean of the block feature in each class according to the posterior probability;

[0021] Step S6.4: Subtract the posterior mean from the prior mean to obtain the representation of the image in the class;

[0022] Step S6.5: Normalize or normalize and weight the representations of each class, and then normalize the vector obtained by splicing to obtain the global feature of the image.

[0023] A location recognition method based on a hierarchical database, characterized in that the location recognition method uses the hierarchical database established according to any one of claims 1-5, and the location recognition method specifically comprises the following steps:

[0024] Step R1: A user uses a device with a camera to take an image of a scene to be positioned and uploads it to a cloud server, and the cloud server extracts the global feature of the image;

[0025] Step R2: Calculate the cosine of the angle between the global feature of the user's image and the global feature of the reference image, and sort the cosine values, and select the reference images corresponding to the top N cosine values as candidate results;

[0026] Step R3: Extract the point local feature of the user's image, match it with the point local feature of each candidate reference image, and use the random sample consensus algorithm to verify the spatial consistency, and calculate the inlier ratio as the score;

[0027] Step R4: Select the two-dimensional map location of the candidate reference image with the highest score as the positioning result, and return it to the user's device.

[0028] Further, the cloud server comprises a local feature extraction module and a feature aggregation module; the cloud server uses a convolutional neural network combined with an attention mechanism;

[0029] The local feature extraction module performs convolution and pooling on the grayscale image to obtain the point feature or block feature of the image. The convolution layer refers to using a learnable convolution kernel to perform spatial filtering on the input two-dimensional image, and different convolution kernels are used to extract different local features. By splicing multiple local features, a better local representation can be obtained. Since convolutional neural networks are often used to process two-dimensional images, the amount of data generated after convolution is large, which is not conducive to operation and storage, so a pooling layer is usually used to downsample the convolution features in space. After pooling, on the one hand, the data volume can be reduced and the model size can be reduced, and on the other hand, overfitting can be prevented.

[0030] Further, the local feature extraction module is a nonlinear mapping function, and the input image is The local feature is obtained after the feature extraction module

[0031] For block feature, N q determined by the feature extraction network structure and the input image size; for point feature, N q determined by the number of sampled key points.

[0032] Further, the feature aggregation module uses an attention mechanism, which is essentially to take a weighted mean of the input vectors after linear transformation according to the correlation of the input vectors in the transformation domain, and its expression is

[0033]

[0034]

[0035] where f(·,·) is an attention function used to describe the similarity between two feature vectors. i , k j , v j are obtained by transforming the input features through a learnable mapping matrix, respectively q i = W q x i , k i = W k x i , v i = W v x i .

[0036] The number of output features depends on the number of q i , and the dimension of the output feature depends on v j , so the number of features can be compressed by adjusting q i , and the dimension of the feature can be compressed by adjusting v j .

[0037] Further, the feature aggregation module first calculates the classification posterior probability of the local feature according to the codebook vocabulary, and then estimates the class mean of the input feature according to the posterior probability; the classification posterior probability is estimated using an attention function, and its expression is

[0038]

[0039] where α is a trainable scaling coefficient.

[0040] After obtaining the posterior probability, the class mean of the input feature can be estimated as the first-order statistic of the input feature, which can be regarded as the representation of the input feature in this class.

[0041]

[0042] Using residual form, the aggregate representation of the feature on a certain class is obtained as:

[0043] v k = c' k - c k

[0044] Cosine similarity is used as the similarity measure of the aggregate representation, so v k needs to be L2 normalized, so the final local feature on a certain class is normalized aggregate representation:

[0045]

[0046] Assuming that the contribution of each class representation to similarity is the same, the normalized similarity of the two images is:

[0047]

[0048] Therefore, use as the final aggregate representation of the image;

[0049] If it is assumed that the contribution of each class representation to similarity is different and the composition vector a = [a1 a2…a K ] T , then the weighted normalized similarity is

[0050]

[0051] Therefore, use as the final aggregate representation of the image.

[0052] A visual position recognition system based on attention compression coding features, comprising

[0053] A hierarchical database establishment unit establishes a hierarchical database based on a scene three-dimensional map;

[0054] An online position recognition unit extracts global features based on the hierarchical database using an attention mechanism and codebook compression coding to realize online position recognition.

[0055] A visual position recognition device based on attention compression coding features, comprising

[0056] A hierarchical database establishment module establishes a hierarchical database based on a scene three-dimensional map;

[0057] An online position recognition module extracts global features based on the hierarchical database using an attention mechanism and codebook compression coding to realize online position recognition.

[0058] The beneficial effects of the present application are:

[0059] The global feature extraction network of the application has high training efficiency, relatively less occupied memory resources, and does not need expensive GPU equipment to train the network, thereby reducing the application cost.

[0060] The feature aggregation module of the application has high flexibility, and can adapt to different feature distributions by modifying the similarity measurement function and the probabilistic function, thereby obtaining more accurate codebook vocabulary.

[0061] The application replaces VGG16 with EfficientNet B3 as the backbone network, so the global feature extraction speed is higher than that of all global feature extraction methods using VGG16 as the backbone network, thereby improving the real-time performance of online positioning result feedback.

[0062] The global feature of the application has higher precision than most one-stage global recall methods that require high complexity operations, and can provide more accurate global recall results while meeting real-time requirements.

[0063] The application generates a database image three-dimensional motion trajectory while modeling the scene, and realizes the mapping of the three-dimensional map to the plane map through the trajectory alignment method, thereby reducing the workload of position labeling. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 is a flowchart of the application.

[0065] Figure 2 is a schematic diagram of selecting a coordinate origin and establishing a coordinate system on the map according to the application.

[0066] Figure 3 is a schematic diagram of the reference image shooting trajectory according to the application.

[0067] Figure 4 is a feature extraction network structure according to the application.

[0068] Figure 5 is the precision test result of the feature extraction network on the public data set according to the application. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0070] A visual position recognition method based on attention compression coding features, comprising

[0071] A hierarchical database is established based on the three-dimensional scene map;

[0072] Based on the hierarchical database, global features are extracted by using an attention mechanism and codebook compression coding to realize online location recognition.

[0073] Further, the hierarchical database based on the three-dimensional scene map is specifically,

[0074] Step S1: According to the environment to be positioned, a coordinate origin is selected, and a two-dimensional rectangular coordinate system is established as a two-dimensional map, as shown in Figure 2

[0075] Step S2: According to the environment to be positioned, a path is planned to cover the scene to be positioned, as shown in Figure 3

[0076] Step S3: A camera is used to shoot reference images (which can be obtained by video decoding) of the scene to be positioned along the trajectory;

[0077] Step S4: Local features of the reference images are extracted, and an incremental SfM algorithm is used to establish a three-dimensional scene map;

[0078] Step S5: The mapping coefficient between the position of the reference images in the three-dimensional map and the position in the two-dimensional map is adjusted according to the trajectory, so as to automatically obtain the position of the reference images in the two-dimensional map, and the association between the reference images and the scene position is established;

[0079] Step S6: Based on the attention mechanism and codebook compression coding principle, a neural network is used to extract global features of the reference images;

[0080] Step S7: The two-dimensional position of the reference images, the global features, and the local features are stored in the database, and the establishment of the scene database is completed.

[0081] Further, the extraction of global features in step S6 is specifically,

[0082] Step S6.1: Block local features of the image are extracted using a feature extraction module;

[0083] Step S6.2: The posterior probability of classifying the block local features to a certain vocabulary is calculated according to the relevance of the block local features and the codebook vocabulary;

[0084] Step S6.3: The posterior mean of the block features in each class is calculated according to the posterior probability;

[0085] Step S6.4: The posterior mean and the prior mean (codebook vocabulary) are subtracted to obtain the representation of the image in the class;

[0086] ​​Step S6.5: normalize each type of feature or normalize and combine weighted, then normalize after splicing into a vector as the global feature of the image.

[0087] A location recognition method based on a hierarchical database, the location recognition method using the hierarchical database as established, the location recognition method specifically comprising the following steps:

[0088] Step R1: a user uses a device with a camera to take an image of a scene to be located and upload it to a cloud server, the cloud server extracting global features from the image;

[0089] Step R2: calculate the cosine of the angle between the user's image and the global features of the database reference images and sort them, and select the reference images corresponding to the top N cosine values as candidate results;

[0090] Step R3: extract the point local features of the user's image, match them with the point local features of each candidate reference image, and use the random sample consensus algorithm for spatial consistency verification, and calculate the inlier ratio as the score;

[0091] Step R4: select the two-dimensional map location of the candidate reference image with the highest score as the positioning result and return it to the user's device.

[0092] Further, the cloud server includes a local feature extraction module and a feature aggregation module; the cloud server uses a convolutional neural network (CNN) combined with an attention mechanism;

[0093] The local feature extraction module convolves and pools the grayscale image to obtain the point features or block features of the image. The convolution layer refers to using a learnable convolution kernel to perform spatial filtering on the input two-dimensional image, and different convolution kernels are used to extract different local features. By splicing multiple local features, better local representation can be obtained. Since convolutional neural networks are commonly used to process two-dimensional images, the amount of data generated after convolution is large, which is not conducive to operation and storage, so a pooling layer is usually used to downsample the convolution features in space. After pooling, on the one hand, the data volume can be reduced and the model size can be reduced, and on the other hand, overfitting can be prevented.

[0094] Further, the local feature extraction module is a nonlinear mapping function, and the input image is The local features are obtained after the feature extraction module

[0095] For block features, N q Determined by the feature extraction network structure and the input image size; for point features, N q Determined by the number of key points sampled.

[0096] Further, the feature aggregation module uses an attention mechanism, which is essentially to take a weighted mean of the linearly transformed input vectors according to the correlation of the input vectors in the transformation domain, and its expression is

[0097]

[0098]

[0099] where f(·,·) is an attention function used to describe the similarity between two feature vectors. i , k j , v j are obtained by transforming the input features through a learnable mapping matrix, respectively q i = W q x i , k i = W k x i , v i = W v x i ;

[0100] As can be seen from the expression, the number of output features depends on the number of q i , and the dimension of the output feature depends on v j , so the number of features can be compressed by adjusting q i , and the dimension of the feature can be compressed by adjusting v j .

[0101] Further, after obtaining the local features of the image, it is necessary to compress and encode them for easy calculation and storage. Based on the idea of codebook compression and encoding, the global clustering center is used as the codebook vocabulary. The input features are modeled as Gaussian mixture distribution, and the feature aggregation module first calculates the classification posterior probability of the local features according to the codebook vocabulary, and then estimates the mean value of the input features of each class according to the posterior probability; the classification posterior probability is estimated using an attention function, and its expression is:

[0102]

[0103] where a is a trainable scaling coefficient;

[0104] After obtaining the posterior probability, the class mean of the input feature can be estimated, which can be regarded as a representation of the input feature in the class as a first-order statistic of the input feature;

[0105]

[0106] To improve the discrimination of the compressed and encoded features, a residual form is used, and the aggregated representation of the features in a class is obtained as:

[0107] v k = c' k - c k

[0108] To improve the speed of operation, cosine similarity is usually used as the similarity measure of the aggregated representation, so v k is L2 normalized, so the final local feature's normalized aggregated representation on a class is:

[0109]

[0110] To improve the accuracy, consider combining multi-class aggregated representation. Assuming that the contribution of each class representation to the similarity is the same, the normalized similarity of two images is:

[0111]

[0112] Therefore, use as the final aggregated representation of the image;

[0113] If it is assumed that the contribution of each class representation to the similarity is different and the composition vector a = [a1 a2…a K ] T , the weighted normalized similarity is

[0114]

[0115] At this time, use as the final aggregated representation of the image, and these coefficients can be learned in the network.

[0116] A visual location recognition system based on attention compression coding features, comprising

[0117] A hierarchical database establishment unit establishes a hierarchical database based on a scene three-dimensional map;

[0118] An online location recognition unit extracts global features based on the hierarchical database using an attention mechanism and codebook compression coding to realize online location recognition.

[0119] A visual location recognition device based on attention compression coding features, comprising

[0120] A hierarchical database establishment module establishes a hierarchical database based on a scene three-dimensional map;

[0121] An online location recognition module extracts global features based on the hierarchical database using an attention mechanism and codebook compression coding to realize online location recognition.

[0122] According to the class representation calculated by the proposed network;

[0123] L2-normalize each type of representation;

[0124] Multiply each type of representation by a class weight coefficient (optional);

[0125] Concatenate each type of representation into a vector-form descriptor;

[0126] L2-normalize the vector-form descriptor;

[0127] Apply PCA dimensionality reduction to the resulting descriptor (optional).

Claims

1. A visual location recognition method based on attention-compressed encoded features, characterized in that, Comprising a hierarchical database is established based on a scene three-dimensional map; global features are extracted based on the hierarchical database using an attention mechanism and codebook compression coding to realize online position recognition; the hierarchical database is established based on the scene three-dimensional map, and specifically, Step S1: According to the environment to be positioned, a coordinate origin is selected, and a two-dimensional rectangular coordinate system is established as a two-dimensional map; Step S2: According to the environment to be positioned, a path is planned to cover the scene to be positioned; Step S3: A reference image of the scene to be positioned is captured along the trajectory using a camera; Step S4: Local features of the reference image are extracted, and an incremental SfM algorithm is used to establish a scene three-dimensional map; Step S5: The mapping coefficient between the position of the reference image in the three-dimensional map and the position in the two-dimensional map is adjusted according to the trajectory, so that the position of the reference image in the two-dimensional map is automatically obtained, and the association between the reference image and the scene position is established; Step S6: Based on the principle of attention mechanism and codebook compression coding, a neural network is used to extract global features from the reference image; Step S7: The two-dimensional position of the reference image, the global features and the point local features are stored in the database, and the establishment of the scene database is completed; the extraction of global features in step S6 is specifically, Step S6.1: The block local features of the image are extracted using a feature extraction module; Step S6.2: The posterior probability of classifying the block local features to a certain vocabulary is calculated according to the relevance of the block local features and the codebook vocabulary; Step S6.3: The posterior mean of the block features in each class is calculated according to the posterior probability; Step S6.4: The posterior mean and the prior mean are subtracted to obtain the representation of the image in the class; Step S6.5: The representations of each class are normalized or normalized and combined with weighting, and then normalized after being spliced into a vector as the global features of the image.

2. A hierarchical database-based location recognition method, characterized by, A hierarchical database is established in the visual position recognition method based on attention compression coding features according to claim 1, and a position recognition method based on the hierarchical database specifically includes the following steps: Step R1: The user uses a device with a camera to capture an image in the scene to be positioned and uploads it to the cloud server, and the cloud server extracts global features from the image; Step R2: The cosine value of the angle between the global features of the user image and the reference image in the database is calculated and sorted, and the reference image corresponding to the top N cosine values is selected as the candidate result; Step R3: The point local features of the user image are extracted, matched with the point local features of each candidate reference image, and verified for spatial consistency using the random sample consensus algorithm, and the inner point ratio is calculated as the score; Step R4: The two-dimensional map position of the candidate reference image with the highest score is selected as the positioning result, and returned to the user device.

3. The hierarchical database-based location identification method of claim 2, wherein, The cloud server includes a local feature extraction module and a feature aggregation module; the cloud server uses a convolutional neural network combined with an attention mechanism; The local feature extraction module convolves and pools the grayscale image to obtain point features or block features of the image; wherein the convolution layer refers to using a learnable convolution kernel to perform spatial filtering on the input two-dimensional image, different convolution kernels are used to extract different local features, and better local representation is obtained by splicing multiple local features; since the convolutional neural network is often used to process two-dimensional images, the data volume generated after the convolution operation is large, which is not conducive to operation and storage, therefore, the pooling layer is used to perform spatial down-sampling on the convolution features, after pooling, on the one hand, the data volume can be reduced, and the model size can be reduced, on the other hand, overfitting can also be prevented.

4. The hierarchical database-based location identification method of claim 3, wherein, The local feature extraction module is a nonlinear mapping function, wherein an input image is , and local features are obtained after the feature extraction module. For block features, N q determined by the feature extraction network structure and the input image size; for point features, N q determined by the number of sampled key points.

5. The hierarchical database-based location identification method of claim 4, wherein, The feature aggregation module uses an attention mechanism, which is essentially to take a weighted mean of the input vector after linear transformation according to the correlation of the input vector in the transformation domain, and its expression is wherein, is an attention function used to describe the similarity between two feature vectors, , , is the input feature after transformation by a learnable mapping matrix, respectively , , ; The number of output features depends on The dimension of the output features depends on Thus, the number of features Compression of the number of features is achieved, the dimension of the output features is adjusted Compression of the dimension of the features is achieved.

6. The hierarchical database-based location identification method of claim 5, wherein, The feature aggregation module first calculates the classification posterior probability of the local feature according to the codebook vocabulary, and then estimates the class mean of the input feature according to the posterior probability; the classification posterior probability is estimated using an attention function, and its expression is: wherein is a trainable scaling factor; After obtaining the posterior probability, the class mean of the input feature can be estimated, which can be regarded as the representation of the input feature in this class as the first-order statistic of the input feature; Using the residual form, the aggregation representation of the feature in a certain class is: Cosine similarity is used as the similarity measure for the aggregated representation, so we need to normalize L2-normalization is performed, so the final local feature's normalized aggregated representation on a class is: Assuming that the contribution of each class representation to the similarity is the same, the normalized similarity of the two images is: Thus, with as the final aggregate representation of the image; If it is assumed that the contribution of each type of representation to similarity is different and the composition vector The weighted normalized similarity is: Then the image is used as the final aggregate representation as the final aggregate representation of the image.

7. A visual position recognition system based on attention-compressed encoded features, characterized in that, It comprises A hierarchical database establishment unit establishes a hierarchical database based on a scene three-dimensional map; An online position recognition unit extracts global features by using an attention mechanism and codebook compression coding based on the hierarchical database to realize online position recognition; The hierarchical database is established based on the scene three-dimensional map, and specifically, Step S1: according to the environment to be positioned, a coordinate origin is selected, and a two-dimensional rectangular coordinate system is established as a two-dimensional map; Step S2: according to the environment to be positioned, a path is planned to cover the scene to be positioned; Step S3: a reference image of the scene to be positioned is taken along the trajectory using a camera; Step S4: point local features are extracted from the reference image, and an incremental SfM algorithm is used to establish a scene three-dimensional map; Step S5: the mapping coefficient between the position of the reference image in the three-dimensional map and the position in the two-dimensional map is adjusted according to the trajectory, so that the position of the reference image in the two-dimensional map is automatically obtained, and the association between the reference image and the scene position is established; Step S6: based on the attention mechanism and codebook compression coding principle, global features are extracted from the reference image using a neural network; Step S7: the two-dimensional position of the reference image, the global features and the point local features are stored in the database, and the establishment of the scene database is completed; The extraction of global features in step S6 is specifically, Step S6.1: block local features of the image are extracted using a feature extraction module; Step S6.2: the posterior probability of classifying the block local features to a certain vocabulary is calculated according to the correlation between the block local features and the codebook vocabulary; Step S6.3: the posterior mean of the block features in each class is calculated according to the posterior probability; Step S6.4: the posterior mean and the prior mean are subtracted to obtain the representation of the image in the class. Step S6.5: normalize or normalize combined with weighted, spliced into a vector after normalization, as the global features of the image.

8. An attention-based feature compression encoding visual position recognition device, characterized by, Comprise The hierarchical database establishment module establishes a hierarchical database based on the three-dimensional map of the scene; The online location recognition module extracts global features based on the hierarchical database using attention mechanism and codebook compression coding to realize online location recognition; The hierarchical database is established based on the three-dimensional map of the scene, specifically, Step S1: according to the environment to be positioned, select the coordinate origin, establish a two-dimensional rectangular coordinate system as a two-dimensional map; Step S2: according to the environment to be positioned, plan a path to cover the scene to be positioned; Step S3: use the camera to shoot the reference image of the scene to be positioned along the trajectory; Step S4: extract point local features from the reference image, and use the incremental SfM algorithm to establish a three-dimensional map of the scene; Step S5: adjust the mapping coefficient between the position of the reference image in the three-dimensional map and the position in the two-dimensional map according to the trajectory, so as to automatically obtain the position of the reference image in the two-dimensional map, and establish the association between the reference image and the scene position; Step S6: based on the principle of attention mechanism and codebook compression coding, use neural network to extract global features from the reference image; Step S7: store the two-dimensional position of the reference image, global features and point local features into the database, and complete the establishment of the scene database; The global feature extraction of step S6 is specifically, Step S6.1: use the feature extraction module to extract the block local features of the image; Step S6.2: calculate the posterior probability of block local features classified into a certain vocabulary according to the relevance of block local features and codebook vocabulary; Step S6.3: calculate the posterior mean of block features in each category according to the posterior probability; Step S6.4: subtract the posterior mean from the prior mean to obtain the representation of the image in the category; Step S6.5: normalize or normalize combined with weighted, spliced into a vector after normalization, as the global features of the image.

Images