Planetary probe vehicle positioning method based on station image matching and related equipment

Abstract

The invention discloses a station image matching-based planetary probe vehicle positioning method and related equipment, and relates to the technical field of image processing, and the method comprises the steps: obtaining a target image of a planetary probe vehicle at a station; acquiring a key point of a local position of the planetary probe vehicle in the target image and a descriptor representing local features of the key point; based on the descriptor of the planetary probe vehicle, key points corresponding to two adjacent stations are matched as positioning points; and positioning the planetary probe vehicle based on the positioning point. By implementing the technical scheme provided by the invention, relatively high positioning precision can be still kept under the condition that the probe vehicle moves in a large span.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a planetary exploration vehicle positioning method based on site image matching and related equipment. Background Art

[0002] In deep space exploration missions, the precise positioning of planetary rovers is crucial to the smooth progress of the mission. Accurate positioning information is the basis for the rover to perceive the environment, plan paths, avoid obstacles, and conduct scientific research. However, due to the complex surface environment of planets, the lack of artificial landmarks, and the inability to use the global positioning system like on Earth, the positioning of planetary rovers faces huge challenges.

[0003] At present, the positioning of planetary rovers mainly relies on dead reckoning, which uses a combination of inertial measurement units and wheel odometers to obtain continuous positioning results. In addition, some exploration missions also use image-based positioning methods as an aid. This method usually analyzes images taken by the rover at different stations, extracts image features, and estimates the position change of the rover by matching these features.

[0004] However, existing positioning methods have insufficient accuracy when the planetary rover moves over long distances. Whether it is a method based on dead reckoning or an image matching method, it is difficult to maintain high-precision positioning results when the rover moves over a long span. Summary of the invention

[0005] The present invention provides a planetary rover positioning method based on site image matching and related equipment, which can maintain high positioning accuracy when the rover moves over a large span.

[0006] In a first aspect of the present invention, a planetary exploration vehicle positioning method based on site image matching is provided, comprising: Acquire target images of the planetary rover at the site; Acquire key points of the local position of the planetary rover in the target image, and descriptors representing local features of the key points; Based on the descriptor of the planetary rover, key points corresponding to two adjacent stations are matched as positioning points; The planetary exploration vehicle is positioned based on the positioning point.

[0007] In a second aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a planetary rover positioning method based on site image matching as described above is implemented.

[0008] In a third aspect of the present invention, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the planetary exploration vehicle positioning method based on site image matching as described in any one of the above is implemented.

[0009] In a fourth aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method for positioning a planetary rover based on site image matching as described in any one of the above is implemented.

[0010] In summary, one or more technical solutions provided in the present invention have at least the following technical effects or advantages: By acquiring the target images of the planetary rover at different stations, extracting key points and their descriptors, and matching the key points of adjacent stations based on the descriptors, the rover is finally positioned using the matching positioning points. Compared with the traditional dead reckoning method, the above-mentioned positioning method based on image matching can effectively reduce the accumulated positioning error during long-distance movement. By extracting and matching the key point features in the image, a high positioning accuracy can be maintained when the rover moves over a large span. Compared with the dead reckoning method that relies solely on the inertial measurement unit and the wheel odometer, this method uses visual information for positioning correction and can better cope with the challenges brought by the complex environment of the planetary surface. In addition, by using descriptors to characterize the local features of key points, this method improves the accuracy and robustness of feature matching, helps to achieve reliable positioning in planetary surface environments with poor texture information, and overcomes the problem of insufficient positioning accuracy of the existing technology when moving over long distances. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0012] Figure 1 It is a flow chart of a planetary rover positioning method based on site image matching provided in an embodiment of the present invention.

[0013] Figure 2 It is a flow chart of a structural diagram of a feature detection network provided by an embodiment of the present invention.

[0014] Figure 3 It is a structural diagram of a feature encoder provided by an embodiment of the present invention.

[0015] Figure 4It is a structural diagram of a descriptor decoder provided by an embodiment of the present invention.

[0016] Figure 5 It is a schematic diagram of a training process of a feature detection network provided by an embodiment of the present invention.

[0017] Figure 6 It is a schematic diagram of a loss function provided by an embodiment of the present invention.

[0018] Figure 7 It is a schematic diagram of the principle of an adaptive 2D-3D matching algorithm provided by an embodiment of the present invention.

[0019] Figure 8 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] In deep space exploration missions, high-precision positioning of the rover is crucial for environmental perception, path planning, and obstacle avoidance, and is the basis for planetary surface exploration and scientific research. However, compared with autonomous vehicles on Earth, planetary rovers are subject to strict load and power consumption constraints and are only equipped with limited sensor equipment, such as stereo cameras, inertial measurement units (IMUs), and wheel odometers. At the same time, the planetary surface environment is complex, the terrain is undulating, and the texture features are poor, all of which pose great challenges to the precise positioning of the rover.

[0022] At present, planetary rovers mainly rely on dead reckoning positioning to obtain continuous position information. This method uses an IMU and an odometer for positioning, but due to the IMU gyro drift and the vehicle slipping caused by terrain undulations, the positioning error accumulates greatly when driving over long distances. In order to correct these errors, cross-site visual positioning technology came into being. This method requires the rover to be stationary at each site and take stereo images, and then determine the position and attitude of the rover by matching images from adjacent sites. This technology has been used in the "Chang'e" and "Tianwen" series of domestic missions.

[0023] In the existing cross-site visual positioning, image feature matching is a key step. Although commonly used algorithms such as Affine-SIFT have certain scale, rotation and viewpoint invariance, they still face severe challenges when processing images of large-span sites. Due to the long distance between adjacent sites, there is a large geometric deformation in the overlapping area of ​​the image. In addition, the texture information in the deep space environment is weak and the geometric structure features are not significant, which makes feature extraction and matching extremely difficult. These factors make the existing technology likely to have concentrated feature points, a small number of matching points, or even complete failure in some complex scenarios, which seriously affects the positioning accuracy and reliability.

[0024] In order to overcome these difficulties, researchers are constantly exploring new technical methods. In recent years, deep learning has made significant progress in the field of image processing, and a series of feature extraction and matching algorithms based on deep neural networks have been proposed. These methods achieve automatic extraction from images to feature points and descriptors through end-to-end learning, and show excellent performance on standard data sets. However, there are still many challenges in directly applying these technologies to planetary exploration missions, such as computational efficiency, feature point repeatability, and model generalization ability. These problems need to be solved urgently.

[0025] In summary, the existing cross-site positioning technology for planetary exploration vehicles has problems such as low automation and unstable accuracy in large-span scenarios, which seriously restricts the efficiency and safety of exploration tasks. Based on this, the embodiment of the present application proposes a planetary exploration vehicle positioning method based on site image matching to solve the above problems.

[0026] Please refer to Figure 1 , Figure 1 This is a flow chart of a method for positioning a planetary rover based on site image matching provided by an embodiment of the present invention. The method can be implemented by a computer program, a single-chip microcomputer, or a planetary rover positioning system based on site image matching based on a von Neumann system. The computer program can be integrated into an application or run as an independent tool application. Specifically, the method may include the following steps: S101, obtaining a target image of the planetary exploration vehicle at the site.

[0027] The target image of the station refers to the visual data of the surrounding environment collected by the planetary rover at a specific docking location. In the embodiment of the present invention, the target image of the station can be understood as one or more sets of stereo image pairs taken by the rover navigation camera system at a fixed station. These images usually cover a 360-degree panoramic view around the rover and are pre-processed such as geometric correction and radiation correction to ensure image quality and accuracy.

[0028] Furthermore, the target image is used to provide basic data for visual positioning, contains rich environmental feature information, and can also be used to build a three-dimensional environmental model. By comparing the target images of different sites, the association between sites can be established, thereby achieving large-scale positioning of the rover. The target image of each site is associated with the specific position and posture of the rover, providing an important reference for subsequent posture estimation.

[0029] S102, obtaining key points of the local position of the planetary rover in the target image and descriptors representing local features of the key points.

[0030] Among them, the key points of the local position refer to the central points of the local area with significant features or unique structures related to the location of the rover in the target image. These key points pay special attention to the feature points in the environment around the rover, including but not limited to the rock edges, terrain undulations, surface texture changes and other areas near the rover. Specifically, the key points of the local position of the planetary rover can be understood as the image feature points that the MRSS-Net feature detection network recognizes and locates in the area around the rover through the key point decoder.

[0031] Furthermore, the key points of the local position of the planetary rover are mainly used to accurately characterize the characteristics of the environment in which the rover is located, providing reliable reference information for cross-site positioning. Since these key points are concentrated in the area around the rover, they can more accurately reflect the changes in the rover's position and help improve positioning accuracy. By comparing the distribution and characteristics of the key points of the local position of the rover in the images of different sites, a more accurate correspondence between sites can be established, thereby achieving high-precision and large-span visual positioning of the rover. In addition, these local key points can also be used to construct a local three-dimensional model of the environment around the rover, providing an important basis for the rover's path planning and obstacle avoidance.

[0032] Correspondingly, the descriptor that characterizes the local features of the key points refers to a high-dimensional vector that is used to encode and represent the unique visual features of the area around the key points of the local position of the planetary rover. Specifically, these descriptors can be understood as feature vectors output by the descriptor decoder in the feature detection network. Each descriptor is usually a numerical array of a fixed length. The above descriptors generate a compact and distinctive feature representation by capturing information such as texture, gradient, color distribution, etc. in the local area around the key points. In an embodiment of the present invention, the descriptor generation process integrates multi-scale feature information and performs global context enhancement through the Transformer structure, so that each descriptor can comprehensively and accurately characterize the local features of its corresponding key points.

[0033] Furthermore, descriptors that characterize local features of key points are mainly used to achieve accurate matching of images across sites. Since these descriptors are highly distinguishable and stable, they can maintain good consistency even under different viewing angles, lighting conditions or partial occlusion. When matching images between sites, corresponding points across images can be found by comparing the similarities between descriptors of different key points.

[0034] Based on the above embodiment, in S102, the step of obtaining key points of the local position of the planetary exploration vehicle in the target image and descriptors representing the local features of the key points may further include the following steps: S201, inputting the target image into a feature detection network, where the feature detection network includes a feature encoder, a key point decoder, and a descriptor decoder.

[0035] Please refer to Figure 2 , Figure 2 A schematic diagram of the structure of a feature detection network provided by an embodiment of the present invention is shown. The feature detection network refers to an end-to-end deep learning model specially designed to extract key points and their descriptors from target images collected by planetary exploration rovers. Specifically, the feature detection network can be understood as the MRSS-Net (Multi-scale Robust Self-supervised Network) structure proposed in an embodiment of the present invention, which is composed of three main components: a feature encoder, a key point decoder, and a descriptor decoder.

[0036] The feature detection network extracts multi-scale and multi-level visual features from the input target image through the feature encoder, effectively characterizing the complex texture and terrain structure of the planetary surface. Subsequently, the key point decoder uses these features to accurately locate local areas with significant features in the image, namely the key points of the local position of the planetary rover. At the same time, the descriptor decoder generates a high-dimensional feature vector, namely the descriptor, for each located key point, which is used to characterize the local features around the key point. By incorporating the DCNv3 module and the Transformer structure, the network can adaptively process image features at different scales and perspective changes, improving robustness in complex planetary environments.

[0037] The integrated design of the feature detection network realizes the unified processing of feature extraction, key point detection and descriptor generation, which improves the efficiency and consistency of the whole process. In addition, the end-to-end optimization capability of the network enables it to better adapt to the specific needs of planetary exploration missions. By using this feature detection network, the embodiments of the present invention can extract high-quality and highly distinguishable visual features from complex planetary surface images.

[0038] For further information, please refer to Figure 3 , Figure 3 A schematic diagram of the structure of a feature encoder provided in an embodiment of the present invention. The feature encoder refers to the core component in the feature detection network that is responsible for extracting multi-scale visual features from the input image. The feature encoder can be understood as a deep convolutional neural network based on an improved Inception network structure, which combines the DCNv3 deformable convolution module and can adaptively process image features in complex environments on the surface of a planet.

[0039] Specifically, the feature encoder in the embodiment of the present invention is mainly used to convert the input two-dimensional image into a series of multi-scale feature maps. It first performs preliminary processing on the input image through a standard convolution layer, and then uses multiple serially connected Inception_DCNv3 modules to extract deep features. These modules process information of different scales in parallel, and at the same time enhance the adaptability to geometric deformation through deformable convolution. The output of the feature encoder is a set of feature maps with rich semantic information, which retains the low-level detail information of the image and contains high-level semantic features.

[0040] Furthermore, the key point decoder refers to the component in the feature detection network that is specifically responsible for locating and extracting key points from the feature map. The key point decoder can be understood as a decoding structure based on a convolutional neural network, which receives the multi-scale feature maps output by the feature encoder and converts these feature maps into probabilistic heat maps representing the locations of key points.

[0041] Specifically, the key point decoder in the embodiment of the present invention is mainly used to accurately locate the key points of the local position of the planetary rover. It first gradually restores the low-resolution feature map output by the feature encoder to a resolution close to the original input image through a series of convolutional layers and upsampling operations. In this process, the key point decoder uses multi-scale feature information to capture significant features at different scales. Finally, a probability heat map with the same size as the original image is generated through a 1x1 convolutional layer and a softmax activation function.

[0042] Each pixel value in the above probability heat map represents the possibility of the position becoming a key point. By setting an appropriate threshold or selecting the local maximum, the specific key point coordinates can be extracted from this heat map. The design of the key point decoder enables it to adaptively identify and locate points with significant features in complex planetary surface environments, such as rock edges, terrain texture changes, etc.

[0043] For further information, please refer to Figure 4 , Figure 4A schematic diagram of the structure of a descriptor decoder provided in an embodiment of the present invention. The descriptor decoder refers to a component in a feature detection network that is specifically responsible for generating a feature descriptor for each detected key point. The descriptor decoder can be understood as a complex network module that combines a convolutional neural network and a Transformer structure. It receives the multi-scale feature map output by the feature encoder and the key point information located by the key point decoder, and generates a high-dimensional feature vector as the descriptor of each key point.

[0044] Specifically, the descriptor decoder in the embodiment of the present invention is mainly used to generate descriptors that characterize local features of key points. It first fuses and processes the multi-scale features output by the feature encoder through a series of convolution operations to retain semantic information at different levels. Then, the Transformer's attention mechanism, especially the interleaved sparse self-attention (ISSA) structure, is used to perform global context enhancement on the features. This design allows the descriptor decoder to capture a wider range of contextual information around the key points while maintaining computational efficiency.

[0045] When generating the final descriptor, the descriptor decoder extracts the features of the corresponding positions from the enhanced feature map according to the position information provided by the key point decoder. These features are further processed and reduced in dimension to form a fixed-dimensional feature vector, which is the descriptor of the key point.

[0046] S202: extracting features of different scales of the target image through a feature encoder.

[0047] Specifically, in the embodiment of the present invention, the design of the feature encoder fully considers the special needs of image feature extraction in planetary exploration missions. The feature encoder consists of a standard convolutional layer and a modular Inception_DCNv3 structure.

[0048] The standard convolution layer uses two stacked 3×3 convolutions, combined with a BN normalization layer and a ReLU activation function. This design helps to initially extract the basic features of the image. At the same time, the BN layer can accelerate network convergence and improve the generalization ability of the model, while the ReLU activation function introduces nonlinearity and enhances the network's expressiveness.

[0049] Inception_DCNv3 combines the multi-scale feature extraction capability of the Inception structure and the geometric adaptability of the DCNv3 deformable convolution. The structure contains conventional convolutions of 1×1 and 3×3 sizes, a Maxpooling pooling layer, and a DCNv3 deformable convolution module. After the previous layer of feature maps is input into the Inception_DCNv3 structure, the network uses convolution kernels of size 1 and 3 for parallel processing to achieve multi-scale feature construction. After each convolution operation, a BN normalization layer and a ReLU activation function are added to improve the feature expression ability and network training effect.

[0050] like Figure 3 As shown in the figure, there are two versions of the Inception_DCNv3 structure: InceptionA_DCNv3 and InceptionB_DCNv3. The output feature map size of the InceptionA_DCNv3 module is half of the input feature map size, which is used to gradually reduce the feature map size; while the InceptionB_DCNv3 module keeps the input and output feature map sizes consistent, which is used to extract features at the same scale. This design enables the network to effectively capture image features at different scales.

[0051] The input of the feature encoder is the original image with 3 channels. After a series of processing, the output channel number is 128 and the size is 1 / 8 of the original image. This dimensionality reduction design greatly reduces the computational complexity of subsequent processing while retaining key information.

[0052] In the DCNv3 module, the embodiment of the present invention introduces an innovative sampling point offset geometric constraint. By using the homography transformation geometric model, the original 2k^2 degrees of freedom sampling offset (k is the convolution kernel size) is reduced to 6. This design significantly reduces the complexity of the model and prevents excessive sampling point deformation. The homography geometric constraint of the sampling point offset is given by the formula:

[0053] In the formula, is the sampling offset, is the regular grid corresponding to the convolution kernel, is the enumeration sampling grid Location , The homography transformation geometric constraints are added according to the image change characteristics of the patroller across sites.

[0054] S203, extracting key points of the local position of the planetary rover in the target image through a key point decoder.

[0055] Specifically, the key point decoder adopts a decoding structure based on convolutional neural network, which can effectively utilize the multi-scale feature information extracted by the feature encoder. Specifically, the key point decoder first expands the input feature map with 128 channels to 256 channels through a series of 3×3 convolution kernels combined with BN normalization layers. This step aims to further enhance the expressiveness of the features and provide a richer information basis for subsequent key point positioning. Subsequently, a 1×1 convolution kernel is used in combination with a BN normalization layer to construct a fully connected layer to reduce the number of channels of the feature map to 65. Of these 65 channels, 64 channels correspond to 64 non-overlapping 8×8 pixel grid areas in the image, and the remaining 1 channel is used to represent non-interest point areas. This design allows the network to perform fine-grained key point detection on the entire image while maintaining efficient computation.

[0056] After the feature map is reduced in dimension, the key point decoder performs channel-dimensional Softmax activation on each element in the feature map of H / 8×W / 8×65. By converting the 65 channel values ​​of each position into a probability distribution, the possibility of each position becoming a key point is effectively represented. Finally, through sub-pixel convolution technology, the key point decoder reconstructs the feature map of size H / 8×W / 8 to a feature map of the same size as the original image H×W. This upsampling method not only restores the original resolution of the feature map, but also retains more detail information, which helps to improve the accuracy of key point positioning.

[0057] S204: The features of different scales corresponding to the key points are fused through a descriptor decoder to obtain a descriptor.

[0058] Specifically, the descriptor decoder first extracts feature maps from different levels of the feature encoder output. In order to effectively fuse these multi-scale features, the network aligns the feature maps of different scales to the same spatial size through 2, 4, and 8 times downsampling operations. Subsequently, these aligned feature maps are merged with the high-level semantic feature maps of size H / 8×W / 8×128 output by the feature encoder using 3×3 convolutions. This multi-scale feature fusion method ensures that both low-level detail information and high-level semantic information are retained during the descriptor generation process, which is crucial for stable feature matching in sparsely textured planetary surface environments.

[0059] However, simply fusing low-level and high-level semantic features is not enough to cope with the complex environmental changes on the planet's surface. In order to better construct global context information, the descriptor decoder introduces a Transformers-based structure. Specifically, the network first uses a multi-layer perceptron to project the fused semi-dense feature map D∈R^(H / 8×W / 8×256) into a new feature space to obtain the transformed feature map D_new∈R^(H / 8×W / 8×256). This enhances the expressiveness of the features and provides more suitable input for the subsequent attention mechanism.

[0060] Next, the transformed feature map is input into the Transformers' Multi-Head Attention Mechanism (MHA). The introduction of MHA enables the network to capture long-range dependencies between different positions in the feature map, which is particularly important for building global context information and better distinguishing local features in repeated patterns. However, the traditional Transformer attention mechanism requires a lot of memory and computing resources when processing large-size feature maps, which is a challenge for resource-constrained planetary probes.

[0061] In order to solve this problem, the embodiment of the present invention proposes an innovative interleaved sparse self-attention (ISSA) structure, such as Figure 4 ISSA adopts a long-short distance interleaving method to avoid applying attention directly to the full feature map. This design not only retains the ability to capture global context information, but also significantly reduces the computational complexity, allowing the method to run efficiently on a probe car with limited computing resources.

[0062] Finally, the descriptor decoder performs bilinear interpolation on the semi-dense feature map output after feature enhancement to obtain the final descriptor corresponding to each key point. This step ensures that the generated descriptor corresponds exactly to the position of the key point in the original image while maintaining the spatial continuity of the feature.

[0063] S205: Obtain key points and descriptors output by the feature detection network.

[0064] The above embodiment describes the process of extracting key points and descriptors from a target image by a feature detection network. Based on the above embodiment, the training process of the feature detection network will be described below. Specifically, the process may further include the following steps: S301, obtaining a sample image, inputting the sample image into a feature detection network, and obtaining a sample detection result output by the feature detection network.

[0065] For details, please refer to Figure 5 , Figure 5A schematic diagram of the training process of a feature detection network provided by an embodiment of the present invention is shown. First, in the image preprocessing stage, real data from actual planetary exploration missions can be obtained as sample images, such as the PCAM image data taken by the Yutu-2 lunar rover in the Chang'e-4 mission and the NaTeCam stereo image data taken by the Zhurong Mars rover in the Tianwen-1 mission. These original images are geometrically corrected and radiometrically corrected to generate high-quality corrected image data. This step ensures the accuracy and consistency of the training data, laying a solid foundation for subsequent network training.

[0066] In the dataset preparation stage, the Affine-SIFT detection algorithm combined with manual selection was used to extract the pixel coordinates of key points from the preprocessed images. With these key points as the center, image blocks of 256×256 pixels were cropped, which not only met the input requirements of the deep learning model, but also ensured that the network could focus on the most critical image areas. In order to further enhance the generalization ability of the model, standard data enhancement techniques including geometric transformation and photometric transformation, such as flipping, scaling, and histogram enhancement, can also be used, which greatly increases the diversity of training data.

[0067] In the network pre-training phase, synthetic images of 128×128 pixels composed of lines, polygons, cubes, and other shapes were first constructed. The corner points of these images are known, providing a clear learning target for the network. By pre-training the network's feature encoder and key point decoder on these synthetic shape datasets, the model has the initial ability to extract stable key points. This pre-training strategy effectively solves the problem that actual planetary surface image data may be insufficient, while providing a good initialization state for the network.

[0068] Finally, in the network self-supervised training phase, the cross-site localization method based on Affine-SIFT can be combined with a small number of manually selected corresponding points to fit the cross-site images using a homography geometry model. The fitting results are used to construct a corresponding set, and the cross-site transformation matrix is ​​randomly sampled under the Gaussian distribution function. This method simulates various perspectives and scale changes that may be encountered in actual detection tasks, greatly enhancing the generalization ability of the MRSS-Net model for cross-site images.

[0069] In the specific training process, the pre-trained encoder and feature point decoder can be used to process the 256×256 pixel rover image, and the transformed image is generated by 100 random homography transformations. Through the inverse homography transformation and data enhancement technology, a rich key point dataset is created, which enables the MRSS-Net model to learn to extract stable and highly distinguishable key points and their descriptors from the input image to obtain sample detection results.

[0070] S302. Based on the loss function, obtain the loss value of the sample detection result, where the loss value includes a key point detection loss function, a key point repeatability loss function, a descriptor loss function, and an enhanced descriptor loss function.

[0071] For details, please refer to Figure 6 , Figure 6 A schematic diagram of a loss function provided for an embodiment of the present invention. In the training process of the feature detection network, obtaining the loss value of the sample detection result based on the loss function is a key step, which directly affects the learning direction and final performance of the network. The embodiment of the present invention adopts a multi-objective loss function strategy, including a key point detection loss function, a key point repeatability loss function, a descriptor loss function, and an enhanced descriptor loss function. The purpose of this composite loss function design is to comprehensively evaluate and optimize the performance of the network in the two main tasks of key point detection and descriptor generation.

[0072] Among them, the key point detection loss function is mainly used to supervise the key point detection output of the network. The cross entropy loss function makes the network more accurate in detecting key points in each local area. For example, the key point detection loss function can be expressed as follows: ; In the formula, ; In the formula, Represents the loss function value of key point detection, represents the output of the network on the feature map, that is, the key point detection score of each local area in the feature map, and Represents the height and width of the feature map, Represents the true value label of the key point location, marking whether each region contains a key point; Represents the detection score vector at position (h,w) in the feature map; Represents the true value label of the position (h,w) in the feature map; represents the loss of each local region keypoint detection, which is the cross entropy loss, used to measure the difference between the predicted detection score and the true label.

[0073] This loss function supervises the accuracy of key point detection by calculating the cross entropy loss pixel by pixel. For each local area, the predicted value is converted into a probability distribution using the Softmax activation function, and then the error between it and the true value label is calculated by cross entropy. The average value of the loss function is calculated by the detection scores of all positions to better balance the key point detection at different positions.

[0074] Among them, the key point repeatability loss function is used to improve the detection rate of key points with the same name in cross-site images, ensuring that the network can detect the same key points under different viewing angles. Exemplarily, the key point repeatability loss function can be expressed as follows: ; In the formula, represents the key point repeatability loss function value, Represents the original image and the image after homography transformation, Represents that from the image To the transformed image The pixel correspondence relationship is Respectively represent images and The key point detection score map in , Represents a collection of local area blocks, using an 8×8 sliding window to detect the score map and Cutting is performed to obtain a local area.

[0075] This loss function ensures that the keypoint detection scores across images are as similar as possible by calculating the cosine similarity of the detection scores for the same region in the original image and the transformed image. If the detection score similarity for the same region is high, it means that the network is able to extract the same keypoints from different perspectives, thereby increasing the repetition rate of keypoints.

[0076] Among them, the descriptor loss function is used to supervise the generation of descriptors to ensure that the descriptors can accurately represent the local features of key points. Exemplarily, the descriptor loss function can be expressed as follows: ; In the formula, ; In the formula, ; In the formula, represents the descriptor loss function value, represents the descriptor feature maps in the original image and the transformed image, Represents the original image Is the pixel (h,w) in the transformed image Pixels in correspond, represents the loss function of the descriptor, which is used to measure the similarity between two descriptors. represents the inner product between descriptors, which is used to measure their similarity. and Represent the edge thresholds of positive samples and negative samples respectively, The weight used to balance the positive and negative samples.

[0077] This loss function supervises the distance of descriptors so that descriptors at the same position tend to be similar, while descriptors at different positions tend to be different. The descriptor similarity of positive samples should be greater than the threshold, while the descriptor similarity of negative samples should be less than the threshold, thereby distinguishing positive and negative samples.

[0078] Among them, the enhanced descriptor loss function is used to supervise the descriptor enhanced by MLP and Transformer to ensure that the enhanced descriptor is more discriminative than the original descriptor. Exemplarily, the enhanced descriptor loss function can be expressed as follows:

[0079] In the formula, represents the strong descriptor loss function value, represents the enhanced descriptor feature maps in the original image and the transformed image, The loss function form of the enhanced descriptor is similar to the loss function of the semi-dense descriptor. Represents the loss function form of the original descriptor.

[0080] This loss function compares the loss of the enhanced descriptor with the original descriptor to ensure that the enhanced descriptor is better than the original descriptor in terms of feature discrimination. The goal of the loss function is to make the enhanced descriptor have better feature expression than the unenhanced descriptor, helping to improve the accuracy of image matching.

[0081] Based on the above loss function, the total loss function can be defined as: ; In the formula, The hyperparameter used to balance the weight of the keypoint repetition rate loss, Hyperparameter used to balance the descriptor loss weights.

[0082] The total loss function combines key point detection loss, key point repetitive loss, descriptor loss and enhanced descriptor loss to ensure the overall performance of the network. By adjusting the hyperparameters σ and λ, the impact of different loss terms on network training can be controlled, so that the network can find a good balance between key point detection and descriptor generation.

[0083] S303: Adjust parameters in the feature detection network based on the loss value.

[0084] Specifically, the back propagation algorithm based on the ADAM optimizer can be used to achieve parameter adjustment. The ADAM optimizer combines the advantages of momentum and adaptive learning rate, and can automatically adjust the learning rate of each parameter. It is particularly effective for processing highly complex and changing data such as planetary surface images. The initial learning rate is set to 0.0001, and a small batch gradient descent method with a batch size of 4 is used. This setting can effectively utilize GPU resources to accelerate the training process while ensuring learning stability.

[0085] During parameter adjustment, it is important to balance the contributions of different loss terms. Different weight coefficients are set for each component in the total loss function. Using σ=0.5 to balance the keypoint detection loss and keypoint repeatability loss helps to improve the accuracy of keypoint detection while enhancing the network's robustness to cross-site image changes. For descriptor loss and enhanced descriptor loss, a weight coefficient of λ=0.0001 is used. This relatively small value ensures that descriptor learning does not overly affect the keypoint detection task while still effectively improving the quality of the descriptor.

[0086] In each iteration, the network first performs forward propagation to calculate the prediction results under the current parameters. Then, the total loss value is calculated based on the prediction results and the true label. Next, the gradient of the loss function for each network parameter is calculated by the back propagation algorithm. Finally, the ADAM optimizer updates the network parameters based on the calculated gradient. This process is repeated until the network converges or reaches a preset number of iterations, which is 200,000 times in the embodiment of the present invention.

[0087] Based on the above embodiment, the performance of the model can also be evaluated by the key point repetition rate and the Euclidean distance of the descriptor to ensure that the model has high accuracy and robustness in the cross-site image matching task.

[0088] Among them, the key point repetition rate reflects the model's ability to identify the same key points under different perspectives. A higher repetition rate means that the model has a stronger cross-perspective matching ability. The key point repetition rate in cross-site images can be used to measure the model's ability to detect the same key points in different site images. The formula is as follows: ; In the formula, Respectively represent the number of key points in the overlapping area of ​​cross-site images, Indicates the total number of key points that were successfully matched.

[0089] Among them, the descriptor Euclidean distance indicates that the descriptor has a good matching effect on the same key points. By measuring the similarity of descriptors, the robustness of the model in feature representation can be judged. The L2 distance between descriptor vectors can be used to evaluate the quality of the descriptor. The formula is as follows: ; In the formula, They represent the descriptor vectors of the same key points in two consecutive images, and n is the dimension of the descriptor vector.

[0090] During the model evaluation process, the repetition rate of key points and the Euclidean distance of descriptors are two important indicators of the performance of the detection model. High repetition rate and low descriptor distance mean that the model has good stability and discrimination ability between different images.

[0091] S103: Based on the descriptor of the planetary rover, key points corresponding to two adjacent stations are matched as positioning points.

[0092] Among them, the positioning point refers to the key point pair that is successfully matched in the images of two adjacent sites. In an embodiment of the present invention, the positioning point can be understood as a feature point extracted by the MRSS-Net feature detection network and successfully paired by a 2D-3D adaptive matching strategy. These points have a high degree of correspondence in the images of the two sites and can accurately reflect the position changes of the planetary rover between the two sites. Each positioning point consists of two parts: the key point and its three-dimensional coordinates in the image of the first site, and the corresponding key point and its two-dimensional pixel coordinates in the image of the second site. This structure not only contains information in the image space, but also contains three-dimensional information in the physical space, providing a rich data basis for subsequent pose estimation.

[0093] Based on the above embodiment, as an optional embodiment, please refer to Figure 7 , Figure 7 A schematic diagram of the principle of an adaptive 2D-3D matching algorithm provided by an embodiment of the present invention, in S103: based on the descriptor of the planetary exploration vehicle, the step of matching the key points corresponding to two adjacent sites as positioning points may further include the following steps: S401, based on the initial position and posture estimation of the planetary exploration vehicle, obtain a first projection point obtained by projecting a three-dimensional key point in a first station onto a two-dimensional plane of a second station, the first station and the second station being two adjacent stations.

[0094] Among them, the initial pose estimate refers to the rough description of the motion state of the planetary rover between two adjacent stations. It contains the approximate position change and attitude change information of the rover from the first station to the second station, which can be understood as a six-degree-of-freedom transformation, including three translation components and three rotation components, which are used to describe the movement and turning of the rover in three-dimensional space. This estimate is usually provided by the rover's dead reckoning system, which combines the attitude data of the inertial measurement unit and the displacement data of the wheel odometer to obtain the motion trajectory of the rover through integral calculation.

[0095] Furthermore, the initial pose estimation is mainly used to provide the basis for key point projection and constrain the matching space. It serves as the basis for transforming the three-dimensional key points in the first station to the two-dimensional plane of the second station, and defines the initial search range for the subsequent matching process. By limiting the possible matching area, this estimation significantly reduces the possibility of false matching and improves the efficiency and accuracy of matching. Especially when the rover moves significantly, the initial pose estimation can provide the necessary spatial correlation information for feature matching, which helps to deal with large perspective changes and large displacement scenes that are difficult to cope with by traditional pure image matching methods.

[0096] Specifically, first obtain the initial pose estimate of the planetary rover, which usually comes from the rover's dead reckoning system, such as the combined navigation results of the inertial measurement unit and the wheel odometer. Although this estimate may have certain errors, it provides the necessary spatial transformation information for key point projection. Then, using this initial pose estimate, combined with the known camera intrinsic parameters, construct a coordinate transformation matrix from the first station to the second station. This transformation matrix contains rotation and translation information, which can describe the approximate movement of the rover between the two stations. Then, for each key point with known three-dimensional coordinates in the first station, the above coordinate transformation matrix is ​​applied for projection transformation. This process can be achieved by homogeneous coordinate transformation, converting the three-dimensional space point into a point on the two-dimensional image plane to obtain the first projection point.

[0097] It should be noted that due to the possible errors in the initial pose estimation, the first projection points do not necessarily coincide exactly with the actual corresponding points in the second site image. Therefore, these projection points are mainly used to define the search area in the subsequent matching process, rather than directly as the final matching result. This strategy not only utilizes prior information to improve efficiency, but also retains sufficient flexibility to adapt to changes in actual conditions.

[0098] S402: Determine a coarse matching area of ​​the first projection point in the two-dimensional plane.

[0099] The coarse matching area refers to a specific range area delineated on the two-dimensional image plane of the second station with the first projection point as the center. The coarse matching area can be understood as a circular or rectangular area centered on the first projection point, and its size is dynamically adjusted according to the uncertainty of the initial pose estimation and the expected matching difficulty. This area is used to limit the search range in the subsequent precise matching process, which not only takes into account the possible errors in the initial pose estimation, but also provides a reasonable search space for finding accurate matching points.

[0100] Specifically, a basic radius or side length is first determined based on the accuracy evaluation of the initial pose estimation. This basic size can be obtained by analyzing historical data or simulation experiments, reflecting the average error range of the initial pose estimation. Then, considering the difference in projection changes of feature points at different depths in the image, this basic size is adaptively adjusted. For example, for feature points in the distance, since their position changes in the image are small, a smaller coarse matching area can be used; while for feature points in the vicinity, a relatively large area is required to accommodate possible position changes.

[0101] In addition, the embodiments of the present invention also take into account the particularity of the planetary surface environment. For example, in areas with sparse textures, it may be necessary to expand the coarse matching area to increase the probability of finding a correct match; while in areas with rich features, the area may be appropriately reduced to improve the matching efficiency. This adaptive strategy can be achieved by analyzing the local feature density of the second site image.

[0102] It should be noted that the process of determining the rough matching area also needs to consider the limitations of the image boundary. For the first projection point close to the edge of the image, its rough matching area may need to be appropriately cropped or adjusted to ensure that it does not exceed the image range. This boundary processing can avoid potential errors in the subsequent matching process.

[0103] S403 , obtaining a matching point pair in the rough matching area, where the matching point pair is two key points whose similarity between descriptors is greater than a threshold.

[0104] Among them, a matching point pair refers to two key points that are identified as corresponding to the same physical point in two adjacent site images. Specifically, a matching point pair can be understood as a key point in the first site image and its corresponding point found in the coarse matching area of ​​the second site image. This pair of key points has a high degree of similarity in the feature space, indicating that they are likely to represent feature points in the same physical world. Matching point pairs are mainly used to establish a spatial correspondence between two site images, providing basic data for subsequent pose estimation and 3D reconstruction.

[0105] Specifically, it is first necessary to calculate the similarity between the descriptor of the first station key point and the descriptors of all candidate key points in the rough matching area. The similarity calculation here usually adopts a measurement method such as Euclidean distance or cosine similarity.

[0106] After calculating the similarity, the embodiment of the present invention uses a threshold method to screen matching point pairs. It is crucial to set a suitable similarity threshold, which directly affects the accuracy and robustness of the match. The selection of the threshold needs to strike a balance between ensuring the accuracy of the match and obtaining a sufficient number of matching points. A threshold that is too high may lead to an insufficient number of matching point pairs, affecting the accuracy of subsequent positioning; while a threshold that is too low may introduce false matches and reduce the reliability of positioning. In an embodiment of the present invention, the setting of the threshold can be dynamically adjusted according to the actual application scenario. For example, a higher threshold can be used in areas with rich textures, while the threshold can be appropriately lowered in areas with sparse textures.

[0107] S404, obtaining a homography matrix obtained by fitting the matching pair, and projecting key points in the first station that are not matching point pairs onto a two-dimensional plane based on the homography matrix to obtain second projection points.

[0108] Among them, the homography matrix refers to a 3×3 matrix that describes the projection transformation relationship between two planes. Specifically, the homography matrix can be understood as a mathematical tool used to represent the geometric transformation from the image plane of the first site to the image plane of the second site. It can accurately describe the rotation, translation, scaling, and perspective transformations between the two sites, and is particularly suitable for processing image matching of planar scenes or distant objects. The homography matrix is ​​mainly used to correct the initial pose estimation and provide a more accurate projection transformation, thereby improving the accuracy of subsequent matching.

[0109] Correspondingly, the second projection point refers to the point obtained by projecting the unmatched key points in the first site onto the image plane of the second site using the homography matrix. In the embodiment of the present invention, the second projection point can be understood as the predicted position of the key point obtained by a more accurate geometric transformation model. These points are used to define a more accurate search area in the second site image to improve the efficiency and accuracy of matching.

[0110] Specifically, the homography matrix is ​​estimated by matching point pairs. This is usually achieved by the least squares method or the more robust RANSAC algorithm. The RANSAC algorithm is particularly helpful in obtaining reliable estimation results in the presence of noise and outliers.

[0111] When estimating the homography matrix, the embodiments of the present invention take into account the particularity of the planetary surface environment and introduce additional constraints. For example, the motion model of the rover is used to limit the degrees of freedom of the homography matrix, or known terrain information is used to enhance the stability of the estimate. These constraints help to obtain a more reliable homography matrix when the number of matching point pairs is limited or unevenly distributed.

[0112] After obtaining the homography matrix, it is applied to the key points in the first site that are not matched point pairs. This process is actually a coordinate transformation, which transforms the points in the first site image coordinate system to the second site image coordinate system. The result of the transformation is the second projection point, which represents the estimated position of these unmatched key points in the second site image.

[0113] It should be noted that although the homography matrix can provide a relatively accurate transformation, it may still have limitations in some cases. Therefore, when using the second projection point, the embodiment of the present invention still needs to consider a certain error range to ensure that potential correct matches are not missed.

[0114] Based on the above embodiment, as an optional embodiment, outliers in the matching point pairs may be excluded based on the homography matrix.

[0115] Specifically, after obtaining the homography matrix, each matching point pair can be evaluated. The key points in the first site are transformed to the second site image plane through the homography matrix, and then the Euclidean distance between the transformed points and the actual matching points is calculated. This distance is called the reprojection error. An adaptive threshold is set, which is dynamically adjusted according to the image resolution and the expected matching accuracy. If the reprojection error of a matching point pair is greater than, it is marked as a potential outlier.

[0116] S405: Determine a fine matching area of ​​the second projection point in the two-dimensional plane.

[0117] Among them, the fine matching area refers to a smaller search area delineated on the two-dimensional image plane of the second station with the second projection point as the center. It can be understood as a smaller and more accurate local area than the coarse matching area, and its size and shape are dynamically adjusted based on the accuracy of the homography matrix and the expected degree of local deformation. This area is used to further limit the search range in the exact matching process, aiming to improve the accuracy and efficiency of matching. The fine matching area is mainly used to perform a refined search in a smaller range to find the candidate point that best matches the key point of the first station.

[0118] Specifically, we first need to determine the base size of the fine matching area based on the uncertainty assessment of the homography matrix. This assessment can be achieved by analyzing the residual or covariance in the homography matrix estimation process. For example, the statistical distribution of the homography matrix projection error can be used to determine a reasonable search radius.

[0119] After determining the base size, the embodiment of the present invention also takes into account the particularity of the planetary surface environment to perform adaptive adjustments. For example, in areas with rich textures, smaller fine matching areas can be used because these areas generally provide more unique features, which are conducive to accurate matching. On the contrary, in areas with sparse textures or strong repetitiveness, relatively large fine matching areas may be required to increase the probability of finding a correct match. This adaptive strategy can be implemented by analyzing the local image features around the second projection point in the second site image.

[0120] In addition, the embodiment of the present invention also considers the influence of the depth information of the key point in the three-dimensional space on the size of the fine matching area. For feature points in the distance, since their position changes in the image are relatively small, a smaller fine matching area can be used; while for feature points in the vicinity, a relatively large area may be required to accommodate possible position changes.

[0121] S406: Iteratively match the coarse matching area and the fine matching area to obtain a positioning point.

[0122] Among them, the coarse matching area and the fine matching area constitute a two-stage adaptive matching strategy. From the perspective of spatial relationship, the coarse matching area usually contains the fine matching area. The coarse matching area defines a larger range around the first projection point obtained based on the initial pose estimation, while the fine matching area is based on this, and the smaller area around the second projection point is further accurately located through the homography matrix. This inclusion relationship reflects the progressive nature of the matching process, which gradually narrows down from a large-scale search to a local area where it is more likely to find an accurate match.

[0123] During the iterative matching process, the relationship between the coarse matching area and the fine matching area changes dynamically. Initially, the coarse matching area may be mainly relied on, and as the iteration proceeds, the role of the fine matching area gradually increases. Each iteration may update the homography matrix, causing the fine matching area to be adjusted. This dynamic relationship ensures that the matching process can be continuously optimized and converged.

[0124] Based on the above embodiment, as an optional embodiment, in S406: iteratively matching the coarse matching area and the fine matching area to obtain the positioning point may further include the following steps: S501. The intersection of the coarse matching area and the fine matching area is taken as the target area, and the matching point pairs that meet the dynamic threshold and have the smallest distance are determined in the target area as the matching point pairs, and the dynamic threshold is positively correlated with the number of iterations and the number of matching key points.

[0125] Specifically, we first need to determine the intersection of the coarse matching area and the fine matching area. This intersection area combines the information provided by the initial pose estimation and the homography matrix, retaining the wide search range of the coarse matching and incorporating the precise positioning of the fine matching, thus forming a more reliable search space. Matching within this target area can effectively reduce the amount of calculation while ensuring that potential correct matching points are not missed.

[0126] When matching in the target area, the embodiment of the present invention adopts a dynamic threshold strategy. The design of the dynamic threshold takes into account two factors: the number of iterations and the number of matched key points, which can be expressed as: ; In the formula, is the number of iterations, is the number of key points that are successfully matched, and the constant 0.3 is the confidence threshold obtained based on the characteristics of the network and a large number of manually selected corresponding points.

[0127] The advantage of the above dynamic threshold strategy is that it can adaptively adjust the matching criteria as the matching process progresses. In the early iterations, the lower threshold allows the algorithm to obtain more candidate matching points, which is conducive to establishing a preliminary matching relationship; as the number of iterations increases and the number of matching points increases, the threshold gradually increases, which can not only screen out more reliable matches, but also effectively reduce the probability of false matches.

[0128] In order to further improve the robustness of the matching, the embodiment of the present invention can also introduce a spatial consistency check in the matching process. For each newly found matching point pair, check whether the spatial relationship between it and the existing matching point pairs conforms to the expected geometric transformation. This can be achieved by comparing the relative positions of the matching point pairs in the two site images. If a matching point pair is consistent with the spatial relationship of most existing matching point pairs, its credibility is increased; otherwise, it may be a mismatch.

[0129] It should be noted that in practical applications, the parameters of the dynamic threshold and the strictness of the spatial consistency check may need to be adjusted according to specific task requirements and hardware limitations. For example, when computing resources are very limited, the criteria for the spatial consistency check can be appropriately relaxed in exchange for faster matching speed.

[0130] S502: Use the key points in each matching point pair as positioning points.

[0131] Based on the above embodiment, as an optional embodiment, after matching the key points corresponding to two adjacent stations as positioning points based on the descriptor of the planetary exploration vehicle, the following steps may also be included: S601, obtaining the geometric distance between the positioning points, where the geometric distance is obtained by solving the three-dimensional coordinates of the positioning.

[0132] The geometric distance refers to the Euclidean distance between two positioning points in three-dimensional space. Specifically, the geometric distance can be understood as the relative position difference of the positioning points restored by triangulation or other three-dimensional reconstruction methods in the actual physical space. This distance is used to evaluate the spatial consistency between positioning points, help screen out stable and reliable positioning points, and thus improve the overall positioning accuracy.

[0133] Specifically, when calculating the geometric distance, the embodiment of the present invention takes into account that feature points at different depths may have different uncertainties and adopts a depth-based weighted distance calculation method. For example, for feature points at a distance, a smaller weight can be given due to the large uncertainty of their three-dimensional positions; while for feature points at a nearby location, a larger weight can be given. This weighting strategy can better reflect the reliability of feature points at different depths.

[0134] S602, obtaining a reprojection error of the positioning point, where the reprojection error is obtained by performing posture calculation on the two-dimensional pixel coordinates of the positioning point, and filtering out unstable positioning points based on geometric distance and reprojection error.

[0135] Specifically, the 3D coordinates of the key points that have been matched successfully can be used as input to build a Euclidean distance matrix between all key points in the object space. Then, the distance difference between the corresponding point pairs in the previous station and the next station is compared. If the distance relationship between a point and other points changes significantly between the two stations, the point is considered as a potential unstable point. This method is particularly effective in utilizing the geometric constraints of the 3D space and can identify abnormal points that may be overlooked in 2D image matching.

[0136] Next, the quasi-Newton nonlinear optimization method (Broyden-Fletcher-Goldfarb-Shanno, BFGS) can be used to solve the pose. The 3D coordinates of the matching points in the previous station and the 2D pixel coordinates of the corresponding points in the current station are used as input. As an efficient second-order optimization algorithm, the BFGS method can quickly converge to the local optimal solution and is particularly suitable for dealing with nonlinear optimization problems involving multiple parameters. By minimizing the reprojection error, the BFGS algorithm can effectively estimate the relative pose transformation between the two stations.

[0137] Then, the average minimum reprojection error of the matching points is calculated based on the solved pose results. This error reflects the accuracy of the estimated pose and is an important indicator for evaluating the overall matching quality. The calculation of the reprojection error takes into account the distance between each 3D point and the actual observation point after it is projected onto the image plane through the estimated pose transformation and the camera intrinsic parameters.

[0138] Finally, an iterative optimization process can be introduced to further improve positioning accuracy. The core of this process is to judge whether the current solution meets the accuracy requirements based on the average minimum reprojection error. The empirical threshold can be set to 5. If the average minimum reprojection error exceeds this threshold, it can be considered that the current matching point set and pose estimation are not ideal. In this case, the key point pair with the largest reprojection error can be deleted, and then the bundle adjustment optimization can be performed again to solve the pose. This process will be repeated until the average minimum reprojection error drops below the threshold, or the preset maximum number of iterations is reached.

[0139] Through the above iterative optimization, possible abnormal matching points can be gradually eliminated, and the quality of the remaining point set can be continuously improved. Secondly, through repeated optimization, a more accurate pose estimation can be obtained. Furthermore, this method has good adaptability and can automatically adjust the number and distribution of matching points according to actual conditions to achieve the expected accuracy standard.

[0140] S104: Positioning the planetary exploration vehicle based on the positioning point.

[0141] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the planetary rover positioning method based on site image matching.

[0142] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0143] On the other hand, an embodiment of the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the planetary rover positioning method based on site image matching provided by the above-mentioned methods.

[0144] On the other hand, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the planetary rover positioning method based on site image matching provided by the above-mentioned methods.

[0145] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than to limit them. Although the embodiments of the present invention have been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the embodiments of the present invention.

Claims

1. A planetary rover positioning method based on site image matching, characterized in that: include: Acquire target images of the planetary rover at the site; Acquire key points of the local position of the planetary rover in the target image, and descriptors representing local features of the key points; Based on the descriptor of the planetary rover, key points corresponding to two adjacent stations are matched as positioning points; The planetary exploration vehicle is positioned based on the positioning point.

2. The planetary exploration vehicle positioning method based on site image matching according to claim 1, characterized in that: The step of acquiring the key points of the local position of the planetary rover in the target image and the descriptors characterizing the local features of the key points includes: Inputting the target image into a feature detection network, wherein the feature detection network includes a feature encoder, a key point decoder, and a descriptor decoder; Extracting features of different scales of the target image by the feature encoder; Extracting key points of the local position of the planetary rover in the target image by the key point decoder; The descriptor decoder is used to fuse the features of different scales corresponding to the key points to obtain a descriptor; The key points and the descriptors output by the feature detection network are obtained.

3. The planetary exploration vehicle positioning method based on site image matching according to claim 2, characterized in that: The training process of the feature detection network includes: Obtaining a sample image, inputting the sample image into a feature detection network, and obtaining a sample detection result output by the feature detection network; Based on the loss function, a loss value of the sample detection result is obtained, wherein the loss value includes a key point detection loss function, a key point repeatability loss function, a descriptor loss function, and an enhanced descriptor loss function; Parameters in the feature detection network are adjusted based on the loss value.

4. The planetary exploration vehicle positioning method based on site image matching according to claim 1, characterized in that: The step of matching key points corresponding to two adjacent stations as positioning points based on the descriptor of the planetary exploration vehicle includes: Based on the initial position and posture estimation of the planetary exploration vehicle, a first projection point is obtained by projecting a three-dimensional key point in a first station onto a two-dimensional plane of a second station, wherein the first station and the second station are two adjacent stations; Determine a coarse matching area of ​​the first projection point in the two-dimensional plane; Acquire a matching point pair in the rough matching area, where the matching point pair is two key points whose similarity between descriptors is greater than a threshold; Obtaining a homography matrix obtained by fitting the matching pair, and projecting key points in the first station that are not matching point pairs onto the two-dimensional plane based on the homography matrix to obtain second projection points; determining a fine matching area of ​​the second projection point in the two-dimensional plane; Iterative matching is performed on the coarse matching area and the fine matching area to obtain a positioning point.

5. The planetary exploration vehicle positioning method based on site image matching according to claim 4, characterized in that: The iterative matching based on the coarse matching area and the fine matching area to obtain a positioning point includes: Taking the intersection of the coarse matching area and the fine matching area as the target area; In the target area, a matching point pair that satisfies a dynamic threshold and has a minimum distance is determined as a matching point pair, and the dynamic threshold is positively correlated with the number of iterations and the number of matching key points. The key point in each of the matching point pairs is used as a positioning point.

6. The planetary exploration vehicle positioning method based on site image matching according to claim 4, characterized in that: After obtaining the homography matrix obtained by fitting the matching pair, the method further includes: Outliers in the matching point pairs are excluded based on the homography matrix.

7. The planetary exploration vehicle positioning method based on site image matching according to claim 1, characterized in that: After matching the key points corresponding to two adjacent sites as positioning points based on the descriptor of the planetary exploration vehicle, the method further includes: Acquire a geometric distance between the positioning points, where the geometric distance is obtained by solving the three-dimensional coordinates of the positioning; Obtaining a reprojection error of the positioning point, where the reprojection error is obtained by performing posture calculation on the two-dimensional pixel coordinates of the positioning point; Unstable positioning points are screened out based on the geometric distance and the reprojection error.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the planetary rover positioning method based on site image matching as described in any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the planetary exploration rover positioning method based on site image matching as described in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the planetary exploration rover positioning method based on site image matching as described in any one of claims 1 to 7 is implemented.

Images