Visual positioning method and device based on sparse infrared self-learning features

Abstract

The invention discloses a visual positioning method and device based on sparse infrared self-learning features, and the method comprises the steps: obtaining an image data set which comprises an infrared visual image and a homographic change image; constructing a visual feature extraction module, performing visual feature extraction on the infrared visual image and the homography image to obtain an original image position vector, a homography image position vector, an original image descriptor vector and a homography image descriptor vector, and constructing a loss function visual feature extraction module to perform model independent element learning training; and integrating the trained visual feature extraction module into the visual positioning method, and deploying the method on a carrier to complete real-time prediction of carrier positioning information. The method is applied to the field of navigation, and can effectively realize navigation and positioning of unmanned platforms such as unmanned vehicles, unmanned aerial vehicles, mobile robots and the like at night and in a weak light environment by constructing the visual feature extraction module with an infrared feature detection capability.

Description

Technical Field

[0001] This invention relates to the fields of unmanned systems and navigation technology, specifically a visual positioning method and device based on sparse infrared self-learning features. Background Technology

[0002] In navigation tasks of unmanned platforms such as autonomous vehicles, drones, and mobile robots, visual odometry is used to obtain the platform's position and attitude. Effective visual feature tracking and vehicle localization in nighttime and low-light environments is an extremely challenging task. Infrared cameras use thermal sensors to capture external temperature differences and convert this information into visual signals to achieve nighttime visual imaging. Using infrared images as an information source provides a new approach for visual localization in nighttime and low-light environments. Existing model-based visual localization methods mostly rely on traditional hand-designed operators for feature extraction and tracking. However, hand-designed operators have poor feature point detection performance in texture-deficient images such as infrared images, making effective feature tracking impossible.

[0003] In recent years, deep learning has made significant strides in image processing, developing various task-specific deep learning neural networks using convolutional neural networks, such as image classification, object recognition, and feature point detection. Deep learning-based feature point detection algorithms use pseudo-labels for self-supervised training, constructed from infrared images. After multiple rounds of training, these algorithms acquire a certain level of feature point detection capability. However, limited by the theoretical foundations of deep learning, these algorithms exhibit poor transferability across different environments; without extensive data for transfer learning, significant performance degradation may occur. Summary of the Invention

[0004] To address the shortcomings of the existing technologies, this invention provides a visual positioning method and apparatus based on sparse infrared self-learning features. By constructing a visual feature extraction module with infrared feature detection capabilities, it can effectively achieve navigation and positioning of unmanned platforms such as unmanned vehicles, drones, and mobile robots in nighttime and low-light environments.

[0005] To achieve the above objectives, the present invention provides a visual localization method based on sparse infrared self-learning features, comprising the following steps: Step 1: Obtain an image dataset, which includes several infrared visual images and homography change images corresponding to each infrared visual image; Step 2: Construct a visual feature extraction module and process a set of corresponding infrared visual images and homography change images as follows: Visual features were extracted from infrared visual images and homography change images respectively to obtain the visual features of the original image and the visual features of the homography image. The original image visual features and the homography image visual features are respectively converted into a first position vector and a second position vector; The original image visual features and the homography image visual features are respectively converted into a first descriptor vector and a second descriptor vector; Step 3: Construct a first feature point prediction error based on the first position vector, construct a second feature point prediction error based on the second position vector, construct a descriptor prediction error based on the first descriptor vector and the second descriptor vector, and construct the loss function of the visual feature extraction module based on the first feature point prediction error, the second feature point prediction error and the descriptor prediction error. Step 4: Based on the loss function and the image dataset, perform model-independent meta-learning training on the visual feature extraction module; Step 5: Integrate the trained visual feature extraction module into the visual positioning method, deploy the method on the carrier, and input the infrared visual images acquired by the carrier in real time into the trained visual feature extraction module to complete the real-time prediction of the carrier's positioning information.

[0006] In one embodiment, in step 3, the prediction error of the first feature point is:

[0007] in, This indicates the prediction error of the first feature point. This represents 1 / 8 of the height resolution of infrared visual images compared to homography images. This represents 1 / 8 of the width resolution of infrared visual images and homography variation images. , Represents the summation parameters. Represents the true value of the feature point location in an infrared visual image. This represents the predicted position value of the feature point corresponding to the first position vector. This represents the cross-entropy loss function.

[0008] In one embodiment, in step 3, the prediction error of the second feature point is:

[0009] in, This indicates the prediction error of the second feature point. This represents 1 / 8 of the height resolution of infrared visual images compared to homography images. This represents 1 / 8 of the width resolution of infrared visual images and homography variation images. , Represents the summation parameters. This represents the true value of the feature point location in a homography image. This represents the predicted value of the feature point position corresponding to the second position vector. This represents the cross-entropy loss function.

[0010] In one embodiment, in step 3, the descriptor prediction error is:

[0011] in, Indicates the descriptor prediction error This represents the descriptors in the first descriptor vector. This represents the descriptors in the second descriptor vector. Descriptor With descriptor Does it match? This represents the formula for calculating the error of sparse descriptors.

[0012] In one embodiment, when the descriptor With descriptor During matching, When the descriptor With descriptor When there is a mismatch, ; In one embodiment, in step 3, the loss function is:

[0013] in, Represents the loss function. This indicates the prediction error of the first feature point. This indicates the prediction error of the second feature point. This represents the descriptor prediction error. , These are the weighting coefficients.

[0014] In one embodiment, step 4 involves training the visual feature extraction module using model-independent meta-learning based on the loss function and the image dataset, including: Step 4.1: Divide the image dataset into a support set and a query set. Both the support set and the query set include several corresponding infrared visual images and homography change images. Step 4.2: Extract the original model parameters from the visual feature extraction module. Copy the model to obtain its parameters. ; Step 4.3, adjust the model parameters based on the support set. The corresponding visual feature extraction module undergoes inner loop training to optimize model parameters. conduct The inner loop model parameters are obtained through the next iteration. ; Step 4.4, based on the parameters of the inner loop model The training error is obtained by combining the query set, and then used to adjust the model parameters. The corresponding visual feature extraction module is trained in the outer loop to obtain the final model parameters. ; Step 4.5, set the final model parameters Substitute these parameters into the next round of model-independent meta-learning training, using them as initial model parameters, and then... Through iterative training, the model-independent meta-learning training of the visual positioning module is completed.

[0015] To achieve the above objectives, the present invention also provides a visual positioning device based on sparse infrared self-learning features, which performs visual positioning using the above method. The visual positioning device includes: A data acquisition unit is used to acquire an image dataset, wherein the image dataset includes several infrared visual images and homography change images corresponding one-to-one with the infrared visual images; The visual feature extraction module includes: The feature extraction submodule is used to extract visual features from infrared visual images and homography change images respectively, to obtain the visual features of the original image and the visual features of the homography image. The feature point location prediction submodule is used to convert the original image visual features and the homography image visual features into a first position vector and a second position vector, respectively, to complete the real-time perception of infrared scene features. The feature point descriptor prediction submodule is used to convert the original image visual features and the homography image visual features into a first descriptor vector and a second descriptor vector, respectively. The loss function construction unit is used to construct a first feature point prediction error based on the first position vector, construct a second feature point prediction error based on the second position vector, construct a descriptor prediction error based on the first descriptor vector and the second descriptor vector, and construct the loss function of the visual feature extraction module based on the first feature point prediction error, the second feature point prediction error and the descriptor prediction error. The model training unit is used to perform model-independent meta-learning training on the visual feature extraction module based on the loss function and the image dataset. The carrier positioning unit is used to integrate the trained visual feature extraction module into the visual positioning method, deploy the method on the carrier, and input the infrared visual images acquired by the carrier in real time into the trained visual feature extraction module to complete the real-time prediction of the carrier positioning information.

[0016] Compared with the prior art, the present invention has the following beneficial technical effects: This invention provides a visual positioning method and apparatus based on sparse infrared self-learning features. By extracting visual features of feature points from infrared visual images and homography change images respectively, and then obtaining the position vector and descriptor vector of the feature points based on the visual features, a loss function is constructed from multiple perspectives based on the feature point prediction error and the descriptor prediction error. The visual feature extraction module is then trained using model-independent meta-learning based on the loss function, so that the visual feature extraction module has relatively stable performance in new data environments. This enables effective navigation and positioning of unmanned platforms such as unmanned vehicles, drones and mobile robots in nighttime and low-light environments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the visual positioning method in Embodiment 1 of the present invention; Figure 2 This is a structural block diagram of the visual positioning device in Embodiment 2 of the present invention.

[0019] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0021] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0022] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0023] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two elements or the interaction between two elements, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0024] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0025] Example 1 like Figure 1 The image shows a visual localization method based on sparse infrared self-learning features disclosed in this embodiment, which mainly includes the following steps: Step 1: Obtain the image dataset, which includes several infrared visual images and homography change images that correspond one-to-one with the infrared visual images; Step 2: Construct a visual feature extraction module and process a set of corresponding infrared visual images and homography change images as follows: Visual features were extracted from infrared visual images and homography change images respectively to obtain the visual features of the original image and the visual features of the homography image. The visual features of the original image and the visual features of the homography image are respectively transformed into a first position vector and a second position vector; The visual features of the original image and the visual features of the homography image are respectively transformed into a first descriptor vector and a second descriptor vector; Step 3: Construct a first feature point prediction error based on the first position vector, construct a second feature point prediction error based on the second position vector, construct a descriptor prediction error based on the first descriptor vector and the second descriptor vector, and construct a loss function for the visual feature extraction module based on the first feature point prediction error, the second feature point prediction error and the descriptor prediction error. Step 4: Based on the loss function and the image dataset, perform model-independent meta-learning training on the visual feature extraction module; Step 5: Integrate the trained visual feature extraction module into the visual positioning method, deploy the method on the carrier, and input the infrared visual images acquired by the carrier in real time into the trained visual feature extraction module to complete the real-time prediction of the carrier's positioning information.

[0026] In the specific implementation of step 1, an infrared camera can be mounted on a carrier such as an unmanned vehicle or drone. Several infrared visual images can be collected based on the infrared camera during the operation of the carrier. Homography can be performed on the collected infrared visual images to obtain homography images that correspond one-to-one with the infrared visual images, thus completing the acquisition of the image dataset.

[0027] It is worth noting that during the acquisition of image datasets, the carrier can be controlled to run in multiple different scenarios, such as daytime scene in city A, nighttime scene in city A, daytime scene in city B, nighttime scene in city B, daytime scene in suburbs, nighttime scene in suburbs, daytime scene in mountainous areas, and nighttime scene in mountainous areas.

[0028] In the specific implementation of step 2, the visual feature extraction module includes a feature extraction submodule composed of a typical convolutional neural network, a feature point location prediction submodule composed of a multi-layer convolutional neural network, and a feature point descriptor prediction submodule composed of a multi-layer convolutional neural network. Specifically: the input layer channel of the feature extraction submodule is set to 3, receiving the infrared visual image and homography change image of each frame on the time axis. After processing by the visual feature extraction module, the infrared visual image and homography change image are used to obtain infrared hidden layer visual features with a size of 1 / 8 of the original size; the feature point location prediction submodule is used to convert the infrared hidden layer visual features extracted by the feature extraction submodule into feature point location vectors; the feature point descriptor prediction submodule is used to convert the infrared hidden layer visual features extracted by the feature extraction submodule into feature point descriptor vectors.

[0029] In this embodiment, the visual feature extraction module operates as follows during the training process: First, a set of corresponding infrared visual images Image of homography Input visual feature extraction module; Then, based on the feature extraction submodule, the infrared visual images are processed respectively. Image of homography Visual features were extracted separately to obtain infrared visual images. Original image visual features of feature points and homography change images Homography visual features of feature points ,for:

[0030]

[0031] in, This indicates the feature extraction submodule; Then, based on the feature point location prediction submodule, the visual features of the original image are respectively... visual features of homography images Converted into infrared visual images The first position vector of the feature point Image of homography The second position vector of the feature point ,for;

[0032]

[0033] in, This represents the feature point location prediction submodule; Meanwhile, the feature point descriptor prediction submodule will respectively convert the visual features of the original image visual features of homography images Converted into infrared visual images The first descriptor vector of the feature points Image of homography The second descriptor vector of the feature points ,for;

[0034]

[0035] in, This represents the feature point descriptor prediction submodule.

[0036] In the specific implementation of steps 3 and 4, the visual feature extraction module is trained using a self-supervised training dataset (i.e., the image dataset mentioned above). The dataset includes the original infrared visual image, the homography image, and the corresponding homography matrix. The original infrared visual image and the homography image are input into the visual feature extraction module to obtain the corresponding feature point positions and descriptor prediction values ​​(i.e., the first position vector, second position vector, first descriptor vector, and second descriptor vector mentioned above). Furthermore, the pre-labeled feature point positions are used as pseudo-true value position labels, and the homography matrix is ​​used as constraint signals.

[0037] In step 3, the loss function of the visual feature extraction module includes three parts: the first feature point prediction error, the second feature point prediction error, and the descriptor prediction error.

[0038] The prediction error of the first feature point is obtained using the cross-entropy loss function, based on the first position vector, and is as follows:

[0039]

[0040] in, This indicates the prediction error of the first feature point. This represents 1 / 8 of the height resolution of infrared visual images compared to homography images. This represents 1 / 8 of the width resolution of infrared visual images and homography variation images. , Represents the summation parameters. Represents the true value of the feature point location in an infrared visual image. This represents the predicted position of the feature point corresponding to the first position vector. This represents the cross-entropy function.

[0041] The prediction error for the second feature point also uses the cross-entropy loss function, obtained based on the second position vector, as follows:

[0042]

[0043] in, This indicates the prediction error of the second feature point. , Represents the summation parameters. This represents the true value of the feature point location in a homography image. This represents the predicted position of the feature point corresponding to the second position vector.

[0044] The descriptor prediction error is calculated from the first descriptor vector and the second descriptor vector, and is as follows:

[0045] in, This represents the descriptor prediction error. This represents the descriptors in the first descriptor vector. This represents the descriptors in the second descriptor vector. This represents the formula for calculating the error of sparse descriptors, namely the Euclidean distance of eigenvectors in the form of the 2-norm. Descriptor With descriptor Whether they match depends on the homography transformation relationship between the corresponding infrared visual image and the homography change image. Therefore, the correspondence between their descriptive subunits can be expressed as follows:

[0046] That is, when the descriptor With descriptor During matching, When the descriptor With descriptor When there is a mismatch, .when When, then determine the descriptor With descriptor If it matches, then it does not match. This is the homography matrix between the infrared visual image and the homography variation image. For descriptor Corresponding feature point location estimation, For descriptor Corresponding feature point location estimation, To ensure accuracy, this embodiment takes... .

[0047] After constructing the prediction errors for the first and second feature points, as well as the descriptor, the final loss function can be built.

[0048] in, Represents the loss function. , These are the weighting coefficients.

[0049] In step 4, the visual feature extraction module is trained using model-independent meta-learning based on the loss function and the image dataset. The main idea of ​​model-independent meta-learning training is to divide the training steps into an inner loop and an outer loop. First, the original visual feature extractor model parameters are... Copy to obtain parameters In the inner loop training, the support set is used for error calculation and in the parameters. Calculate the gradient and apply it to the parameters. Update. (Inner loop) The parameters of the inner loop model are obtained in the next iteration. Then, in the outer loop, the query set is used in combination with the parameters. Calculate the gradient, the gradient with respect to the original model parameters. Update. Through multiple iterations of "inner loop + outer loop," network parameters with good generalization ability to new data are finally obtained. The specific implementation process includes: Step 4.1: Divide the image dataset into a support set and a query set. Both the support set and the query set include several corresponding infrared visual images and homography change images. Step 4.2: Extract the original model parameters from the visual feature extraction module. Copy the model to obtain its parameters. ; Step 4.3, based on the support set, pair the model parameters. The corresponding visual feature extraction module undergoes inner loop training to optimize model parameters. conduct The inner loop model parameters are obtained through the next iteration. ; Step 4.4, based on the parameters of the inner loop model The training error is obtained by combining the query set, and then used to adjust the model parameters. The corresponding visual feature extraction module is trained in the outer loop to obtain the final model parameters. ; Step 4.5, set the final model parameters Substitute these parameters into the next round of model-independent meta-learning training, using them as initial model parameters, and then... Through rounds of iterative training, the model-independent meta-learning training of the visual localization module is completed.

[0050] In step 5, the trained visual feature extraction module is integrated into the visual positioning method, and this method is deployed on the carrier. The infrared visual images acquired by the carrier in real time are input into the trained visual feature extraction module to complete the real-time prediction of the carrier's positioning information. The specific implementation process is as follows: For the first time, real-time infrared visual images collected during the operation of the carrier are input into the trained visual feature extraction module; Next, the visual feature extraction module extracts features from the real-time infrared visual image to obtain the real-time visual features of the feature points in the visual feature extraction module. Then, the feature point position prediction submodule in the visual feature extraction module converts the real-time visual features of the real-time infrared visual image into a real-time position vector. Finally, the real-time position vector predicted by the visual feature extraction module is matched with the real-time position vector obtained from the previous frame of real-time infrared visual image, thus enabling real-time prediction of the carrier's current frame positioning information.

[0051] Furthermore, based on the trained visual feature extraction module, the feature descriptor subset corresponding to the image dataset can be obtained using the visual feature extraction module. .gather Each feature descriptor in The numerical distribution is in Between. For sets The feature descriptors in the data are binarized, with feature descriptors greater than or equal to 0 set to 1 and feature descriptors less than 0 set to 0, resulting in:

[0052] The binarized feature descriptors are used to form a feature descriptor set. , will set The input is fed into the iBow (Incremental Bags of Binary Words) algorithm to train a bag-of-words tree model based on learned feature descriptors. Assuming the goal is to obtain a bag-of-words tree model with depth M1 and number of branches M2, the iBow algorithm trains the model as follows: First, the original feature descriptor subset Consider it as the 0th level of the bag-of-words tree; Subsequently, the feature descriptor subset K-means clustering is used, and K cluster centers are randomly selected to represent the original feature descriptor subset. It is divided into K subsets, which serve as the first level of the bag-of-words tree model; Then, by repeating the K-means clustering operation within each subset, we obtain the second level of the bag-of-words tree; Then, repeat the above clustering operation within each set in the layers after the second layer, and finally obtain a bag-of-words tree model with a depth of M1 and a number of branches of M2. In this tree, level 0 is the root node, and the level furthest from the root node is the leaf node, also known as a word in the bag-of-words tree. Each word is assigned a weight based on its frequency of occurrence in the original feature descriptor set. The more frequently a word appears, the more common and less representative the feature descriptor is, and the smaller its weight is.

[0053] Example 2 Based on the visual positioning method in Embodiment 1, this embodiment discloses a visual positioning device based on sparse infrared self-learning features. (Reference) Figure 2The visual positioning device includes a data acquisition unit, a visual feature extraction module, a loss function construction unit, a model training unit, and a carrier positioning unit. The visual feature extraction module consists of a feature extraction submodule, a feature point position prediction submodule, and a feature point descriptor prediction submodule. This visual positioning device is used to execute some or all of the steps of the visual positioning method in Embodiment 1 to achieve visual positioning and navigation of unmanned platforms such as unmanned vehicles, drones, and mobile robots during operation. Specifically: The data acquisition unit is used to acquire an image dataset, which includes several infrared visual images and homography change images that correspond one-to-one with the infrared visual images; The feature extraction submodule is used to extract visual features from infrared visual images and homography change images respectively, to obtain the visual features of the original image and the visual features of the homography image. The feature point location prediction submodule is used to convert the visual features of the original image and the visual features of the homography image into a first position vector and a second position vector, respectively, to complete the real-time perception of infrared scene features. The feature point descriptor prediction submodule is used to convert the original image visual features and the homography image visual features into a first descriptor vector and a second descriptor vector, respectively; The loss function construction unit is used to construct a first feature point prediction error based on the first position vector, construct a second feature point prediction error based on the second position vector, construct a descriptor prediction error based on the first descriptor vector and the second descriptor vector, and construct the loss function of the visual feature extraction module based on the first feature point prediction error, the second feature point prediction error and the descriptor prediction error. The model training unit performs model-independent meta-learning training on the visual feature extraction module based on the loss function and the image dataset; The carrier localization unit is used to integrate the trained visual feature extraction module into the visual localization method, deploy the method on the carrier, and input the infrared visual images acquired by the carrier in real time into the trained visual feature extraction module to complete the real-time prediction of the carrier localization information.

[0054] In this embodiment, the specific working process and working principle of the data acquisition unit, visual feature extraction module, loss function construction unit, model training unit and carrier localization unit are the same as those in Embodiment 1, so they will not be described again in this embodiment.

[0055] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A visual positioning method based on sparse infrared self-learning features, characterized in that, The method comprises the following steps: Step 1, obtaining an image data set comprising a plurality of infrared vision images and a plurality of homographic change images corresponding to the infrared vision images respectively; Step 2, constructing a visual feature extraction module, and processing a group of corresponding infrared vision images and homographic change images as follows: respectively extracting visual features from the infrared vision images and the homographic change images to obtain original image visual features and homographic image visual features; respectively converting the original image visual features and the homographic image visual features into first position vectors and second position vectors; respectively converting the original image visual features and the homographic image visual features into first descriptor vectors and second descriptor vectors; Step 3, constructing a first feature point prediction error based on the first position vectors, constructing a second feature point prediction error based on the second position vectors, constructing a descriptor prediction error based on the first descriptor vectors and the second descriptor vectors, and constructing a loss function of the visual feature extraction module based on the first feature point prediction error, the second feature point prediction error and the descriptor prediction error; Step 4, performing model-agnostic meta-learning training on the visual feature extraction module based on the loss function and the image data set; Step 5, integrating the trained visual feature extraction module into a visual positioning method, deploying the method on a carrier, and inputting infrared vision images collected by the carrier in real time into the trained visual feature extraction module to complete real-time prediction of carrier positioning information.

2. The method of visual localization based on sparse infrared self-learning features according to claim 1, characterized in that, In step 3, the first feature point prediction error is: wherein, denotes the first feature point prediction error, denotes the infrared vision image and the homography changed image height resolution 1 / 8, denotes the infrared vision image and the homography changed image width resolution 1 / 8, , denotes the sum parameter, denotes the feature point position true value in the infrared vision image, denotes the feature point position predicted value corresponding to the first position vector, denotes the cross-entropy loss function.

3. The method of visual localization based on sparse infrared self-learning features according to claim 1, characterized in that, In step 3, the second feature point prediction error is: wherein, represents the second feature point prediction error, represents the infrared vision image and the homographic changed image height resolution 1 / 8, represents the infrared vision image and the homographic changed image width resolution 1 / 8, , represents the sum parameter, represents the feature point position true value in the homographic changed image, represents the feature point position predicted value corresponding to the second position vector, represents the cross-entropy loss function.

4. The method of claim 1, wherein, In step 3, the descriptor prediction error is: wherein, denotes a descriptor prediction error, denotes a descriptor in the first descriptor vector, denotes a descriptor in the second descriptor vector, denotes a descriptor matches a descriptor or not, denotes a sparse descriptor error computation formula.

5. The method of visual localization based on sparse infrared self-learning features according to claim 4, characterized in that, When descriptor With descriptor During matching, When the descriptor With descriptor When there is a mismatch, .

6. The visual positioning method based on sparse infrared self-learning features according to any one of claims 1 to 5, characterized in that, In step 3, the loss function is: wherein, represents a loss function, represents a first feature point prediction error, represents a second feature point prediction error, represents a descriptor prediction error, , is a weight coefficient.

7. The visual positioning method based on sparse infrared self-learning features according to any one of claims 1 to 5, characterized in that, In step 4, performing model-agnostic meta-learning training on the visual feature extraction module based on the loss function and the image data set comprises: Step 4.1, dividing the image data set into a support set and a query set, wherein the support set and the query set each comprise a plurality of corresponding infrared vision images and homographic change images; Step 4.2, extract the original model parameters of the visual feature module Copy to get the model parameters ; Step 4.3, updating the model parameters based on the support set Inner loop training is performed on the corresponding visual feature extraction module, and the model parameters are updated to obtain the inner loop model parameters ; Step 4.4, based on the inner loop model parameters The training error is obtained in combination with the query set, and the training error is used to update the model parameters The corresponding visual feature extraction module is trained in an outer loop to obtain the final model parameters ; Step 4.5, the final model parameters of the visual positioning module are obtained are substituted into the next round of model-agnostic meta-learning training, which is taken as the initial model parameters, and are trained for rounds of iterative training, to complete the model-agnostic meta-learning training of the visual positioning module.

8. A visual positioning device based on sparse infrared self-learning features, characterized by, The visual positioning method of any one of claims 1-7, wherein the visual positioning device comprises: a data acquisition unit configured to acquire an image data set, wherein the image data set comprises a plurality of infrared vision images and a plurality of homographic change images corresponding to the infrared vision images respectively; a visual feature extraction module comprising: a feature extraction sub-module configured to extract visual features from the infrared vision images and the homographic change images respectively to obtain original image visual features and homographic image visual features; a feature point position prediction sub-module configured to convert the original image visual features and the homographic image visual features into first position vectors and second position vectors respectively to complete real-time perception of infrared scene features; a feature point descriptor prediction sub-module configured to convert the original image visual features and the homographic image visual features into first descriptor vectors and second descriptor vectors respectively; a loss function construction unit configured to construct a first feature point prediction error according to the first position vector, a second feature point prediction error according to the second position vector, a descriptor prediction error according to the first descriptor vector and the second descriptor vector, and a loss function of the visual feature extraction module based on the first feature point prediction error, the second feature point prediction error and the descriptor prediction error; a model training unit configured to perform model-agnostic meta-learning training on the visual feature extraction module according to the loss function and the image data set; a carrier positioning unit configured to integrate the trained visual feature extraction module into a visual positioning method, deploy the method on a carrier, input infrared visual images collected by the carrier in real time into the trained visual feature extraction module, and complete real-time prediction of carrier positioning information.

Images