Planetary rover positioning method based on site image matching and related equipment

Abstract

The application discloses a planet exploration vehicle positioning method based on site image matching and related equipment, and relates to the technical field of image processing. The method comprises the following steps: acquiring a target image of a planet exploration vehicle at a site; acquiring key points of a local position of the planet exploration vehicle in the target image and descriptors representing local features of the key points; matching the key points corresponding to two adjacent sites as positioning points based on the descriptors of the planet exploration vehicle; and positioning the planet exploration vehicle based on the positioning points. The technical scheme provided by the application can still maintain high positioning accuracy in the case of large-span movement of the planet exploration vehicle.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a planetary rover positioning method based on site image matching and related equipment. Background Art

[0002] Precise positioning of planetary rovers is crucial for the smooth progress of deep space exploration missions. Accurate positioning information is fundamental to the rover's environmental perception, path planning, obstacle avoidance, and scientific research. However, due to the complex surface environment of planets, the lack of artificial landmarks, and the inability to use the Global Positioning System (GPS) as on Earth, planetary rover positioning presents significant challenges.

[0003] Currently, planetary rover positioning primarily relies on dead reckoning, which uses a combination of inertial measurement units and wheel odometry to obtain continuous positioning results. In addition, some missions also employ image-based positioning methods as a supplement. These methods typically analyze images captured by the rover at different locations, extract image features, and then match these features to estimate the rover's position.

[0004] However, existing positioning methods suffer from insufficient accuracy when planetary rovers move over long distances. Both dead reckoning and image matching methods struggle to maintain high-precision positioning over large distances. Summary of the Invention

[0005] The present invention provides a planetary rover positioning method based on site image matching and related equipment, which can maintain high positioning accuracy when the rover moves over a large span.

[0006] In a first aspect of the present invention, a planetary rover positioning method based on site image matching is provided, comprising:

[0007] Acquire target images of the planetary rover at the site;

[0008] Acquire key points of the local position of the planetary rover in the target image, and descriptors representing local features of the key points;

[0009] Based on the descriptor of the planetary rover, key points corresponding to two adjacent stations are matched as positioning points;

[0010] The planetary exploration rover is positioned based on the positioning point.

[0011] In a second aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a planetary rover positioning method based on site image matching as described above is implemented.

[0012] In a third aspect of the present invention, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it implements any of the planetary rover positioning methods based on site image matching as described above.

[0013] In a fourth aspect of the present invention, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements any of the above-described planetary rover positioning methods based on site image matching.

[0014] In summary, one or more technical solutions provided in the present invention have at least the following technical effects or advantages:

[0015] By acquiring target images of the planetary rover at different sites, extracting key points and their descriptors, and matching key points of adjacent sites based on the descriptors, the rover is finally positioned using the matching positioning points. Compared with the traditional dead reckoning method, the above-mentioned positioning method based on image matching can effectively reduce the accumulated positioning error during long-distance movement. By extracting and matching key point features in the image, it is possible to maintain a high positioning accuracy when the rover moves over a large span. Compared with the dead reckoning method that relies solely on inertial measurement units and wheel odometers, this method uses visual information for positioning correction and can better cope with the challenges posed by the complex environment of the planetary surface. In addition, by using descriptors to characterize the local features of key points, this method improves the accuracy and robustness of feature matching, helps to achieve reliable positioning in planetary surface environments with poor texture information, and overcomes the problem of insufficient positioning accuracy of existing technologies when moving over long distances. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 This is a flow chart of a planetary rover positioning method based on site image matching provided by an embodiment of the present invention.

[0018] Figure 2It is a flow chart of a structural diagram of a feature detection network provided by an embodiment of the present invention.

[0019] Figure 3 It is a structural diagram of a feature encoder provided by an embodiment of the present invention.

[0020] Figure 4 It is a structural diagram of a descriptor decoder provided by an embodiment of the present invention.

[0021] Figure 5 This is a schematic diagram of a training process of a feature detection network provided by an embodiment of the present invention.

[0022] Figure 6 Schematic diagram of a loss function provided by an embodiment of the present invention.

[0023] Figure 7 3D-2D matching algorithm according to an embodiment of the present invention.

[0024] Figure 8 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0026] In deep space exploration missions, high-precision positioning of rovers is crucial for environmental perception, path planning, and obstacle avoidance, and is fundamental to planetary surface exploration and scientific research. However, compared to autonomous vehicles on Earth, planetary rovers are subject to strict payload and power consumption constraints and are equipped with limited sensor equipment, such as stereo cameras, inertial measurement units (IMUs), and wheel odometry. Furthermore, the complex environment, rugged terrain, and lack of texture features on planetary surfaces pose significant challenges to precise positioning of rovers.

[0027] Currently, planetary rovers primarily rely on dead reckoning to obtain continuous position information. This method utilizes an IMU (Infrared Measurement Unit) and an odometer for positioning. However, due to IMU gyro drift and vehicle slippage caused by terrain undulations, positioning errors accumulate significantly over long distances. To correct these errors, cross-site visual positioning technology has emerged. This method requires the rover to remain stationary at each site and capture stereo images. The rover's position and attitude are then determined by matching images from adjacent sites. This technology has been employed in both the Chang'e and Tianwen missions in China.

[0028] Image feature matching is a key step in existing cross-site visual positioning. While commonly used algorithms like Affine-SIFT offer certain invariance to scale, rotation, and viewpoint, they still face significant challenges when processing images from sites spanning a large area. The long distances between adjacent sites create significant geometric distortion in overlapping image regions. Furthermore, in deep space, texture information is weak and geometric features are less prominent, making feature extraction and matching extremely difficult. These factors can lead to existing technologies experiencing concentrated feature point distribution, a low number of matching points, or even complete failure in certain complex scenarios, severely impacting positioning accuracy and reliability.

[0029] To overcome these difficulties, researchers are constantly exploring new technical approaches. In recent years, deep learning has made significant progress in image processing, and a series of feature extraction and matching algorithms based on deep neural networks have been proposed. These methods achieve automatic extraction of feature points and descriptors from images through end-to-end learning, demonstrating excellent performance on standard datasets. However, direct application of these techniques to planetary exploration missions still faces numerous challenges, such as computational efficiency, feature point reproducibility, and model generalization.

[0030] In summary, existing cross-site positioning technologies for planetary rovers suffer from low automation and unstable accuracy in large-span scenarios, severely restricting the efficiency and safety of exploration missions. Based on this, this application proposes a planetary rover positioning method based on site image matching to address these issues.

[0031] Please refer to Figure 1 , Figure 1 This is a flow chart of a method for planetary rover positioning based on site image matching, provided by an embodiment of the present invention. This method can be implemented using a computer program, a single-chip microcomputer, or run on a von Neumann-based planetary rover positioning system based on site image matching. The computer program can be integrated into an application or run as a standalone tool application. Specifically, the method may include the following steps:

[0032] S101. Acquire a target image of the planetary exploration rover at the site.

[0033] The target image at a station refers to visual data of the surrounding environment collected by a planetary rover at a specific docking location. In embodiments of the present invention, the target image at a station can be understood as one or more stereo image pairs captured by the rover's navigation camera system at a fixed station. These images typically cover a 360-degree panoramic view around the rover and undergo pre-processing such as geometric and radiometric correction to ensure image quality and accuracy.

[0034] Furthermore, target images provide the foundational data for visual positioning, containing rich information about environmental features and used to construct 3D environmental models. By comparing target images from different sites, correlations can be established between them, enabling large-scale positioning of the rover. Each site's target image is associated with the rover's specific position and posture, providing a crucial reference for subsequent pose estimation.

[0035] S102: Acquire key points of the local position of the planetary rover in the target image and descriptors representing local features of the key points.

[0036] Local keypoints are the center points of local regions in the target image that have significant features or unique structures associated with the rover's location. These keypoints specifically focus on characteristic features in the rover's surrounding environment, including but not limited to rock edges, terrain undulations, and surface texture variations near the rover. Specifically, local keypoints for a planetary rover can be understood as image feature points identified and located in the area around the rover by the MRSS-Net feature detection network using the keypoint decoder.

[0037] Furthermore, the key points of a planetary rover's local position are primarily used to accurately characterize the rover's environment, providing reliable reference information for cross-site positioning. Because these key points are concentrated in the area surrounding the rover, they can more accurately reflect changes in the rover's position, helping to improve positioning accuracy. By comparing the distribution and characteristics of the rover's local position key points in images from different sites, a more precise correspondence between sites can be established, enabling high-precision, large-span visual positioning of the rover. Furthermore, these local key points can be used to construct a local three-dimensional model of the rover's surroundings, providing an important basis for path planning and obstacle avoidance.

[0038] Correspondingly, the descriptor that characterizes the local features of the key points refers to a high-dimensional vector that is used to encode and represent the unique visual features of the area around the key points of the local position of the planetary rover. Specifically, these descriptors can be understood as the feature vectors output by the descriptor decoder in the feature detection network. Each descriptor is usually a fixed-length numerical array. The above descriptors generate a compact and distinctive feature representation by capturing information such as texture, gradient, and color distribution in the local area around the key points. In an embodiment of the present invention, the descriptor generation process integrates multi-scale feature information and performs global context enhancement through the Transformer structure, so that each descriptor can comprehensively and accurately characterize the local features of its corresponding key points.

[0039] Furthermore, descriptors representing the local features of keypoints are primarily used to achieve accurate cross-site image matching. Because these descriptors are highly discriminative and stable, they maintain good consistency even under varying viewpoints, lighting conditions, or partial occlusion. When performing cross-site image matching, corresponding points across images can be found by comparing the similarities between descriptors of different keypoints.

[0040] Based on the above embodiment, the step of obtaining key points of the local position of the planetary rover in the target image and descriptors representing local features of the key points in S102 may further include the following steps:

[0041] S201. Input the target image into a feature detection network, which includes a feature encoder, a key point decoder, and a descriptor decoder.

[0042] Please refer to Figure 2 , Figure 2 The figure shows a schematic diagram of the structure of a feature detection network provided by an embodiment of the present invention. The feature detection network refers to an end-to-end deep learning model specifically designed to extract keypoints and their descriptors from target images collected by planetary rovers. Specifically, the feature detection network can be understood as the MRSS-Net (Multi-scale Robust Self-supervised Network) structure proposed in an embodiment of the present invention. It consists of three main components: a feature encoder, a keypoint decoder, and a descriptor decoder.

[0043] The feature detection network extracts multi-scale, multi-level visual features from the input target image through a feature encoder, effectively characterizing the complex textures and terrain structures of the planetary surface. The keypoint decoder then uses these features to precisely locate local regions with significant features within the image, namely the keypoints of the planetary rover's local position. Simultaneously, the descriptor decoder generates a high-dimensional feature vector, or descriptor, for each located keypoint, which characterizes the local features surrounding the keypoint. By incorporating the DCNv3 module and the Transformer architecture, the network can adaptively process image features at varying scales and viewpoints, improving its robustness in complex planetary environments.

[0044] The integrated design of the feature detection network enables unified processing of feature extraction, keypoint detection, and descriptor generation, improving the efficiency and consistency of the entire process. Furthermore, the network's end-to-end optimization capabilities enable it to better adapt to the specific needs of planetary exploration missions. By utilizing this feature detection network, embodiments of the present invention are able to extract high-quality, highly discriminative visual features from complex planetary surface images.

[0045] For further information, please refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of a feature encoder provided in an embodiment of the present invention. A feature encoder is the core component of a feature detection network responsible for extracting multi-scale visual features from input images. The feature encoder can be understood as a deep convolutional neural network based on a modified Inception network structure, combined with the DCNv3 deformable convolution module, capable of adaptively processing image features in complex environments on planetary surfaces.

[0046] Specifically, in this embodiment of the present invention, the feature encoder is primarily used to convert an input two-dimensional image into a series of multi-scale feature maps. It first performs preliminary processing on the input image using standard convolutional layers, and then utilizes multiple cascaded Inception_DCNv3 modules to extract deep features. These modules process information at different scales in parallel, while enhancing adaptability to geometric deformations through deformable convolutions. The output of the feature encoder is a set of feature maps rich in semantic information, which preserves low-level image details while also incorporating high-level semantic features.

[0047] Furthermore, the keypoint decoder refers to the component in the feature detection network specifically responsible for locating and extracting keypoints from feature maps. The keypoint decoder can be understood as a decoding structure based on a convolutional neural network. It receives the multi-scale feature maps output by the feature encoder and converts them into probabilistic heatmaps representing the locations of keypoints.

[0048] Specifically, in this embodiment of the present invention, the keypoint decoder is primarily used to accurately locate key points in the local position of a planetary rover. It first uses a series of convolutional layers and upsampling operations to gradually restore the low-resolution feature map output by the feature encoder to a resolution similar to that of the original input image. In this process, the keypoint decoder utilizes multi-scale feature information to capture salient features at different scales. Finally, a 1x1 convolutional layer and a softmax activation function are used to generate a probability heatmap of the same size as the original image.

[0049] Each pixel value in the probability heat map represents the probability of that location being a keypoint. By setting an appropriate threshold or selecting a local maximum, the specific keypoint coordinates can be extracted from this heat map. This design of the keypoint decoder enables it to adaptively identify and locate points with significant features, such as rock edges and areas with varying terrain textures, in complex planetary surface environments.

[0050] For further information, please refer to Figure 4 , Figure 4 A schematic diagram of the structure of a descriptor decoder provided in an embodiment of the present invention. The descriptor decoder refers to the component in the feature detection network specifically responsible for generating feature descriptors for each detected keypoint. The descriptor decoder can be understood as a complex network module that combines a convolutional neural network and a Transformer architecture. It receives the multi-scale feature map output by the feature encoder and the keypoint information located by the keypoint decoder, generating a high-dimensional feature vector as the descriptor for each keypoint.

[0051] Specifically, in this embodiment of the present invention, the descriptor decoder is primarily used to generate descriptors that represent the local features of keypoints. It first fuses and processes the multi-scale features output by the feature encoder through a series of convolution operations to preserve semantic information at different levels. It then leverages the Transformer's attention mechanism, specifically the Interleaved Sparse Self-Attention (ISSA) architecture, to enhance the features with global context. This design allows the descriptor decoder to capture a wider range of contextual information around keypoints while maintaining computational efficiency.

[0052] When generating the final descriptor, the descriptor decoder extracts the features of the corresponding positions from the enhanced feature map based on the position information provided by the key point decoder. These features are further processed and reduced in dimension to form a fixed-dimensional feature vector, which is the descriptor of the key point.

[0053] S202: Extract features of different scales of the target image through a feature encoder.

[0054] Specifically, in this embodiment of the present invention, the design of the feature encoder fully considers the special requirements of image feature extraction in planetary exploration missions. The feature encoder consists of standard convolutional layers and a modular Inception_DCNv3 structure.

[0055] The standard convolutional layer uses two stacked 3×3 convolutions, coupled with a batch normalization layer and a ReLU activation function. This design helps to initially extract basic image features. The batch normalization layer accelerates network convergence and improves the model's generalization ability, while the ReLU activation function introduces nonlinearity, enhancing the network's expressiveness.

[0056] Inception_DCNv3 combines the multi-scale feature extraction capabilities of the Inception architecture with the geometric adaptability of DCNv3's deformable convolutions. This architecture includes conventional 1×1 and 3×3 convolutions, a Maxpooling layer, and a DCNv3 deformable convolution module. After the previous layer's feature map is input to the Inception_DCNv3 architecture, the network processes it in parallel using convolution kernels of sizes 1 and 3 to achieve multi-scale feature construction. Each convolution operation is followed by a batch normalization layer and a ReLU activation function to improve feature representation and network training performance.

[0057] like Figure 3 As shown in Figure 1, the Inception_DCNv3 architecture has two versions: InceptionA_DCNv3 and InceptionB_DCNv3. The output feature map size of the InceptionA_DCNv3 module is half the size of the input feature map, gradually reducing the feature map size. The InceptionB_DCNv3 module, on the other hand, maintains the same size of the input and output feature maps, extracting features at the same scale. This design enables the network to effectively capture image features at different scales.

[0058] The input of the feature encoder is a 3-channel original image. After a series of processing, the output is a feature map with 128 channels and a size of 1 / 8 the original image. This dimensionality reduction design retains key information while greatly reducing the computational complexity of subsequent processing.

[0059] In the DCNv3 module, this embodiment of the present invention introduces innovative geometric constraints for sampling point offsets. By utilizing a homography transformation geometric model, the original 2k^2 degrees of freedom of sampling offsets (k is the convolution kernel size) are reduced to 6. This design significantly reduces model complexity while preventing excessive sampling point deformation. The homography geometric constraints for sampling point offsets are given by the formula:

[0060]

[0061] Where, is the sampling offset, is the regular grid corresponding to the convolution kernel, is the enumeration sampling grid Location , The geometric constraints of the homography transformation are added based on the characteristics of image changes across patrol sites.

[0062] S203: extract key points of the local position of the planetary rover in the target image through a key point decoder.

[0063] Specifically, the keypoint decoder employs a decoding structure based on a convolutional neural network. This design effectively utilizes the multi-scale feature information extracted by the feature encoder. Specifically, the keypoint decoder first expands the 128-channel input feature map to 256 channels using a series of 3×3 convolution kernels combined with batch normalization layers. This step aims to further enhance the expressive power of the features and provide a richer information foundation for subsequent keypoint localization. Subsequently, a fully connected layer is constructed using a 1×1 convolution kernel combined with a batch normalization layer, reducing the number of channels in the feature map to 65. Of these 65 channels, 64 correspond to 64 non-overlapping 8×8 pixel grid regions in the image, and the remaining channel is used to represent non-points of interest. This design allows the network to perform fine-grained keypoint detection across the entire image while maintaining efficient computation.

[0064] After feature map dimensionality reduction, the keypoint decoder performs a channel-wise softmax activation on each element in the H / 8 × W / 8 × 65 feature map. By converting the 65 channel values ​​at each location into a probability distribution, it effectively represents the likelihood of each location being a keypoint. Finally, using sub-pixel convolution, the keypoint decoder reconstructs the H / 8 × W / 8 feature map to a feature map of the same size as the original image (H × W). This upsampling method not only restores the original resolution of the feature map but also preserves more detailed information, helping to improve the accuracy of keypoint localization.

[0065] S204: The features of different scales corresponding to the key points are fused through the descriptor decoder to obtain a descriptor.

[0066] Specifically, the descriptor decoder first extracts feature maps from the output of the feature encoder at different levels. To effectively fuse these multi-scale features, the network aligns the feature maps of different scales to the same spatial size through downsampling operations by 2, 4, and 8 times. Subsequently, these aligned feature maps are merged with the final high-level semantic feature map of size H / 8×W / 8×128 output from the feature encoder using 3×3 convolutions. This multi-scale feature fusion method ensures that the descriptor generation process retains both low-level details and high-level semantic information, which is critical for stable feature matching in sparsely textured planetary surface environments.

[0067] However, simply fusing low-level and high-level semantic features is insufficient to address the complex environmental changes on a planet's surface. To better construct global context, the descriptor decoder introduces a Transformer-based architecture. Specifically, the network first uses a multi-layer perceptron to project the fused semi-dense feature map D∈R^(H / 8×W / 8×256) into a new feature space, resulting in a transformed feature map D_new∈R^(H / 8×W / 8×256). This enhances the expressive power of the features and provides more suitable input for the subsequent attention mechanism.

[0068] Next, the transformed feature map is fed into the Transformer's Multi-Head Attention (MHA) mechanism. The introduction of MHA enables the network to capture long-range dependencies between different locations in the feature map, which is particularly important for building global context and better distinguishing local features from repetitive patterns. However, the traditional Transformer attention mechanism requires a large amount of memory and computational resources when processing large feature maps, which is a challenge for resource-constrained planetary probes.

[0069] In order to solve this problem, the embodiment of the present invention proposes an innovative interleaved sparse self-attention (ISSA) structure, such as Figure 4 ISSA employs an interleaving of long and short distances to avoid applying attention directly to the entire feature map. This design retains the ability to capture global contextual information while significantly reducing computational complexity, enabling the method to run efficiently on rover vehicles with limited computing resources.

[0070] Finally, the descriptor decoder performs bilinear interpolation on the semi-dense feature map output after feature enhancement to obtain the final descriptor corresponding to each keypoint. This step ensures that the generated descriptor accurately corresponds to the location of the keypoint in the original image while maintaining the spatial continuity of the features.

[0071] S205: Obtain key points and descriptors output by the feature detection network.

[0072] The above embodiment describes the process of extracting key points and descriptors from a target image using a feature detection network. Based on the above embodiment, the following describes the training process of the feature detection network. Specifically, the process may further include the following steps:

[0073] S301: Obtain a sample image, input the sample image into a feature detection network, and obtain a sample detection result output by the feature detection network.

[0074] For details, please refer to Figure 5 , Figure 5 A schematic diagram of the training process for a feature detection network provided by an embodiment of the present invention is shown. First, during the image preprocessing stage, real data from actual planetary exploration missions is obtained as sample images, such as the PCAM image data captured by the Yutu-2 lunar rover on the Chang'e-4 mission and the NaTeCam stereo image data captured by the Zhurong Mars rover on the Tianwen-1 mission. These raw images undergo geometric and radiometric correction to generate high-quality corrected image data. This step ensures the accuracy and consistency of the training data, laying a solid foundation for subsequent network training.

[0075] During the dataset creation phase, the Affine-SIFT detection algorithm, combined with manual selection, was used to extract keypoint pixel coordinates from the preprocessed images. 256×256 pixel image patches were cropped centered around these keypoints. This not only met the input requirements of the deep learning model but also ensured that the network focused on the most critical image regions. To further enhance the model's generalization capabilities, standard data augmentation techniques, including geometric and photometric transformations, such as flipping, scaling, and histogram augmentation, were employed, significantly increasing the diversity of the training data.

[0076] During the network pretraining phase, 128×128 pixel synthetic images composed of lines, polygons, cubes, and other shapes were first constructed. The corner points of these images were known, providing a clear learning target for the network. By pretraining the network's feature encoder and keypoint decoder on this dataset of synthetic shapes, the model initially developed the ability to extract stable keypoints. This pretraining strategy effectively addresses the potential shortage of actual planetary surface image data while providing a good initial state for the network.

[0077] Finally, during the network's self-supervised training phase, a homography-based geometric model is fitted to the cross-site images, based on the Affine-SIFT cross-site localization method and a small number of manually selected corresponding points. The fitted results are used to construct a correspondence set, and the cross-site transformation matrix is ​​randomly sampled under a Gaussian distribution function. This approach simulates the various viewpoints and scale variations that may be encountered in real-world detection tasks, significantly enhancing the MRSS-Net model's generalization capabilities for cross-site images.

[0078] During training, a pre-trained encoder and feature point decoder are used to process 256×256 pixel rover images, while 100 random homography transformations are applied to generate the transformed images. Through inverse homography and data augmentation techniques, a rich dataset of keypoints is created, enabling the MRSS-Net model to learn to extract stable and highly discriminative keypoints and their descriptors from the input images and generate sample detection results.

[0079] S302. Based on the loss function, obtain the loss value of the sample detection result, where the loss value includes the key point detection loss function, the key point repeatability loss function, the descriptor loss function, and the enhanced descriptor loss function.

[0080] For details, please refer to Figure 6 , Figure 6 A schematic diagram of a loss function provided for an embodiment of the present invention. During the training process of a feature detection network, obtaining the loss value of sample detection results based on the loss function is a key step, which directly affects the learning direction and final performance of the network. This embodiment of the present invention adopts a multi-objective loss function strategy, including a key point detection loss function, a key point repeatability loss function, a descriptor loss function, and an enhanced descriptor loss function. The purpose of this composite loss function design is to comprehensively evaluate and optimize the network's performance in the two main tasks of key point detection and descriptor generation.

[0081] Among them, the key point detection loss function is mainly used to supervise the key point detection output of the network, and the cross entropy loss function makes the network more accurate in detecting key points in each local area. For example, the key point detection loss function can be expressed as follows:

[0082] ;

[0083] Where, ;

[0084] Where, Represents the loss function value of key point detection, Represents the output of the network on the feature map, that is, the key point detection score of each local area in the feature map, and Represents the height and width of the feature map, Represents the true value label of the key point location, marking whether each area contains a key point; Represents the detection score vector at position (h,w) in the feature map; Represents the true value label of the position (h,w) in the feature map; represents the loss of keypoint detection in each local region, which is the cross entropy loss and is used to measure the difference between the predicted detection score and the true label.

[0085] This loss function monitors the accuracy of keypoint detection by calculating cross-entropy loss on a pixel-by-pixel basis. For each local region, the predicted value is converted into a probability distribution using the Softmax activation function, and the error between it and the true value label is calculated using cross-entropy. The loss function averages the detection scores across all locations to better balance keypoint detection at different locations.

[0086] Among them, the key point repeatability loss function is used to improve the detection rate of key points with the same name in images across sites, ensuring that the network can detect the same key points under different viewing angles. For example, the key point repeatability loss function can be expressed as follows:

[0087] ;

[0088] Where, represents the key point repeatability loss function value, Represents the original image and the image after homography transformation, Represents that from the image To the transformed image The pixel correspondence relationship, Represents images respectively and The key point detection score map in , Represents a collection of local area blocks, using an 8×8 sliding window to detect the score map and Cut to obtain a local area.

[0089] This loss function ensures that the keypoint detection scores across images are as similar as possible by calculating the cosine similarity of the detection scores for the same region in the original and transformed images. A high similarity in the detection scores for the same region means that the network is able to extract the same keypoints from different viewpoints, thereby increasing the repetition rate of keypoints.

[0090] Among them, the descriptor loss function is used to supervise the generation of descriptors to ensure that the descriptors can accurately represent the local features of key points. For example, the descriptor loss function can be expressed as follows:

[0091] ;

[0092] Where, ;

[0093] Where, ;

[0094] Where, represents the descriptor loss function value, represents the descriptor feature maps in the original image and the transformed image, Represents the original image Is the pixel (h,w) in the transformed image consistent with the Pixels in correspond, represents the loss function of the descriptor, which is used to measure the similarity between two descriptors. represents the inner product between descriptors, which is used to measure their similarity. and Represent the edge thresholds of positive samples and negative samples respectively, The weight used to balance the positive and negative samples.

[0095] This loss function supervises the distance between descriptors, making descriptors at the same location tend to be similar, while descriptors at different locations tend to be different. The descriptor similarity of positive samples must be greater than a threshold, while the descriptor similarity of negative samples must be less than a threshold, thereby distinguishing between positive and negative samples.

[0096] Among them, the enhanced descriptor loss function is used to supervise the descriptor enhanced by MLP and Transformer to ensure that the enhanced descriptor is more discriminative than the original descriptor. For example, the enhanced descriptor loss function can be expressed as follows:

[0097]

[0098] Where, represents the strong descriptor loss function value, represents the enhanced descriptor feature maps in the original image and the transformed image, Represents the loss function form of the enhanced descriptor, which is similar to the loss function of the semi-dense descriptor. Represents the loss function form of the original descriptor.

[0099] This loss function compares the loss of the enhanced descriptor with the original descriptor to ensure that the enhanced descriptor is superior to the original descriptor in terms of feature discrimination. The goal of the loss function is to make the enhanced descriptor have better feature representation than the unenhanced descriptor, helping to improve the accuracy of image matching.

[0100] Based on the above loss function, the total loss function can be defined as:

[0101] ;

[0102] Where, The hyperparameter used to balance the weight of the keypoint repetition rate loss, Hyperparameter used to balance the descriptor loss weights.

[0103] The total loss function combines keypoint detection loss, keypoint repetitiveness loss, descriptor loss, and enhanced descriptor loss to ensure the overall performance of the network. By adjusting the hyperparameters σ and λ, the impact of different loss terms on network training can be controlled, allowing the network to find a good balance between keypoint detection and descriptor generation.

[0104] S303: Adjust the parameters in the feature detection network based on the loss value.

[0105] Specifically, we used a backpropagation algorithm based on the ADAM optimizer to implement parameter adjustment. The ADAM optimizer combines the advantages of momentum and adaptive learning rates to automatically adjust the learning rate for each parameter. This is particularly effective for processing highly complex and variable data such as planetary surface images. We set an initial learning rate of 0.0001 and used mini-batch gradient descent with a batch size of 4. This setup ensures learning stability while effectively utilizing GPU resources to accelerate the training process.

[0106] During parameter tuning, it's important to balance the contributions of different loss terms. Different weight coefficients are set for each component of the total loss function. Using σ = 0.5 to balance the keypoint detection loss and keypoint repeatability loss helps improve keypoint detection accuracy while enhancing the network's robustness to cross-site image variations. For the descriptor loss and enhanced descriptor loss, a weight coefficient of λ = 0.0001 is used. This relatively small value ensures that descriptor learning does not overly impact the keypoint detection task while still effectively improving descriptor quality.

[0107] In each iteration, the network first performs forward propagation to calculate the prediction results under the current parameters. Then, the total loss value is calculated based on the prediction results and the true labels. Next, the gradient of the loss function with respect to each network parameter is calculated through the backpropagation algorithm. Finally, the ADAM optimizer updates the network parameters based on the calculated gradients. This process is repeated until the network converges or reaches a preset number of iterations, which is 200,000 in this embodiment of the present invention.

[0108] Based on the above embodiment, the performance of the model can also be evaluated by the key point repetition rate and the Euclidean distance of the descriptors to ensure that the model has high accuracy and robustness in the cross-site image matching task.

[0109] The keypoint repetition rate reflects the model's ability to identify the same keypoints across different viewpoints. A higher repetition rate means the model has stronger cross-viewpoint matching capabilities. The keypoint repetition rate in cross-site images can be used to measure the model's ability to detect the same keypoints in images from different sites. The formula is as follows:

[0110] ;

[0111] Where, Represents the number of key points in the overlapping area of ​​cross-site images, Indicates the total number of keypoints that are successfully matched.

[0112] Among them, the descriptor Euclidean distance indicates that the descriptor matches the same key points well. By measuring the similarity of descriptors, the robustness of the model in feature representation can be judged. The L2 distance between descriptor vectors can be used to evaluate the quality of the descriptor. The formula is as follows:

[0113] ;

[0114] Where, They represent the descriptor vectors of the same key points in two consecutive images, and n is the dimension of the descriptor vector.

[0115] During model evaluation, the repetition rate of key points and the Euclidean distance of descriptors are two important indicators of model performance. A high repetition rate and a low descriptor distance mean that the model has good stability and discrimination ability between different images.

[0116] S103: Based on the descriptor of the planetary rover, key points corresponding to two adjacent sites are matched as positioning points.

[0117] Among them, the positioning point refers to the key point pair that is successfully matched in the images of two adjacent stations. In an embodiment of the present invention, the positioning point can be understood as the feature point extracted by the MRSS-Net feature detection network and successfully paired through the 2D-3D adaptive matching strategy. These points have a high degree of correspondence in the images of the two stations and can accurately reflect the position changes of the planetary rover between the two stations. Each positioning point consists of two parts: the key point and its three-dimensional coordinates in the image of the first station, and the corresponding key point and its two-dimensional pixel coordinates in the image of the second station. This structure not only contains information in the image space, but also contains three-dimensional information in the physical space, providing a rich data basis for subsequent pose estimation.

[0118] Based on the above embodiment, as an optional embodiment, please refer to Figure 7 , Figure 7 A schematic diagram of the principle of an adaptive 2D-3D matching algorithm provided in an embodiment of the present invention. In S103, the step of matching key points corresponding to two adjacent sites as positioning points based on the descriptors of the planetary rover may further include the following steps:

[0119] S401, based on the initial pose estimation of the planetary rover, obtain a first projection point obtained by projecting a three-dimensional key point in a first station onto a two-dimensional plane of a second station, the first station and the second station being two adjacent stations.

[0120] The initial pose estimate is a rough description of the rover's motion between two adjacent stations. It contains information about the approximate change in position and attitude from the first station to the second. It can be understood as a six-degree-of-freedom transformation, consisting of three translational components and three rotational components, describing the rover's movement and steering in three-dimensional space. This estimate is typically provided by the rover's dead reckoning system, which combines attitude data from the inertial measurement unit with displacement data from the wheel odometers to calculate the rover's trajectory through integral calculations.

[0121] Furthermore, the initial pose estimation is mainly used to provide the basis for key point projection and constrain the matching space. It serves as the basis for the transformation of the three-dimensional key points in the first station to the two-dimensional plane of the second station, defining the initial search range for the subsequent matching process. By limiting the possible matching area, this estimation significantly reduces the possibility of false matches and improves the efficiency and accuracy of matching. Especially when the rover moves significantly, the initial pose estimation can provide the necessary spatial correlation information for feature matching, helping to deal with large perspective changes and large displacement scenarios that traditional pure image matching methods have difficulty in dealing with.

[0122] Specifically, the initial pose estimate of the planetary rover is first obtained, which usually comes from the rover's dead reckoning system, such as the combined navigation results of the inertial measurement unit and the wheel odometer. Although this estimate may have certain errors, it provides the necessary spatial transformation information for key point projection. Then, using this initial pose estimate, combined with the known camera intrinsic parameters, a coordinate transformation matrix from the first station to the second station is constructed. This transformation matrix contains rotation and translation information and can describe the approximate movement of the rover between the two stations. Then, for each key point with known three-dimensional coordinates in the first station, the above coordinate transformation matrix is ​​applied to perform a projection transformation. This process can be achieved through homogeneous coordinate transformation, converting the three-dimensional space point into a point on the two-dimensional image plane to obtain the first projection point.

[0123] It's important to note that due to potential errors in the initial pose estimation, the first projected points may not exactly coincide with the actual corresponding points in the second site image. Therefore, these projected points are primarily used to define the search area in the subsequent matching process, rather than being directly used as the final matching results. This strategy leverages prior information to improve efficiency while retaining sufficient flexibility to adapt to changing circumstances.

[0124] S402: Determine a coarse matching area of ​​the first projection point in the two-dimensional plane.

[0125] The coarse matching area refers to a specific range defined on the 2D image plane of the second station, centered on the first projection point. This area can be understood as a circular or rectangular region centered on the first projection point, whose size is dynamically adjusted based on the uncertainty of the initial pose estimate and the expected matching difficulty. This area is used to limit the search range during the subsequent precise matching process, accounting for potential errors in the initial pose estimate while providing a reasonable search space for finding accurate matching points.

[0126] Specifically, a base radius or side length is first determined based on the accuracy of the initial pose estimate. This base size can be obtained by analyzing historical data or simulation experiments and reflects the average error range of the initial pose estimate. This base size is then adaptively adjusted to account for the differences in projection changes of feature points at different depths in the image. For example, for distant feature points, a smaller coarse matching area can be used because their position changes in the image are small; however, for nearby feature points, a relatively large area is required to accommodate possible position changes.

[0127] Furthermore, this embodiment of the present invention takes into account the particularities of planetary surface environments. For example, in areas with sparse textures, the coarse matching area may need to be expanded to increase the probability of finding a correct match; whereas in areas with rich features, the area can be appropriately reduced to improve matching efficiency. This adaptive strategy can be implemented by analyzing the local feature density of the second-site image.

[0128] It's important to note that the process of determining the coarse matching area also needs to consider image boundary constraints. For first projection points near the edge of the image, the coarse matching area may need to be cropped or adjusted appropriately to ensure it does not exceed the image boundary. This boundary processing can avoid potential errors in the subsequent matching process.

[0129] S403 : Acquire a matching point pair in the coarse matching area, where the matching point pair is two key points whose similarity between descriptors is greater than a threshold.

[0130] A matching point pair refers to two key points identified as corresponding to the same physical point in two adjacent site images. Specifically, a matching point pair can be understood as a key point in the first site image and its corresponding point found within the coarse matching area of ​​the second site image. This pair of key points exhibits a high degree of similarity in feature space, indicating that they likely represent feature points in the same physical world. Matching point pairs are primarily used to establish a spatial correspondence between the two site images, providing foundational data for subsequent pose estimation and 3D reconstruction.

[0131] Specifically, the similarity between the descriptor of the first station key point and the descriptors of all candidate key points in the coarse matching area needs to be calculated. The similarity calculation here usually adopts a measurement method such as Euclidean distance or cosine similarity.

[0132] After calculating the similarity, the embodiment of the present invention uses a threshold method to screen matching point pairs. Setting an appropriate similarity threshold is crucial, as it directly affects the accuracy and robustness of the matching. The selection of the threshold needs to strike a balance between ensuring matching accuracy and obtaining a sufficient number of matching points. A threshold that is too high may result in an insufficient number of matching point pairs, affecting the accuracy of subsequent positioning; while a threshold that is too low may introduce false matches and reduce the reliability of positioning. In the embodiment of the present invention, the setting of the threshold can be dynamically adjusted according to the actual application scenario. For example, a higher threshold can be used in areas with rich textures, while the threshold can be appropriately lowered in areas with sparse textures.

[0133] S404: Obtain a homography matrix obtained by fitting the matching pair, and project key points in the first station that do not form a matching point pair onto a two-dimensional plane based on the homography matrix to obtain second projection points.

[0134] A homography is a 3×3 matrix that describes the projective transformation between two planes. Specifically, a homography can be understood as a mathematical tool used to represent the geometric transformation from the image plane of the first station to the image plane of the second station. It accurately describes the rotation, translation, scaling, and perspective transformations between the two stations, making it particularly suitable for image matching of planar scenes or distant objects. The homography is primarily used to correct the initial pose estimate, providing a more precise projective transformation, thereby improving the accuracy of subsequent matching.

[0135] Correspondingly, the second projected points refer to the points obtained by projecting the unmatched keypoints in the first site onto the second site image plane using the homography matrix. In this embodiment of the present invention, the second projected points can be understood as the predicted positions of the keypoints obtained using a more accurate geometric transformation model. These points are used to define a more precise search area in the second site image, improving matching efficiency and accuracy.

[0136] Specifically, the homography matrix is ​​estimated by matching point pairs. This is usually achieved using the least squares method or the more robust RANSAC algorithm. The RANSAC algorithm is particularly helpful in obtaining reliable estimation results in the presence of noise and outliers.

[0137] When estimating the homography matrix, embodiments of the present invention take into account the particularities of the planetary surface environment and introduce additional constraints. For example, the rover's motion model can be used to limit the degrees of freedom of the homography matrix, or known terrain information can be used to enhance the stability of the estimate. These constraints help obtain a more reliable homography matrix when the number of matching point pairs is limited or unevenly distributed.

[0138] After obtaining the homography matrix, it is applied to the keypoints in the first site that do not form a matching point pair. This process is actually a coordinate transformation, converting the points in the coordinate system of the first site image to the coordinate system of the second site image. The result of the transformation is the second projection point, which represents the estimated position of these unmatched keypoints in the second site image.

[0139] It should be noted that although the homography matrix can provide a relatively accurate transformation, it may still have limitations in some cases. Therefore, when using the second projection point, the embodiment of the present invention still needs to consider a certain error range to ensure that potential correct matches are not missed.

[0140] Based on the above embodiment, as an optional embodiment, outliers in the matching point pairs may be excluded based on the homography matrix.

[0141] Specifically, after obtaining the homography matrix, each matching point pair can be evaluated. This is done by transforming the key points in the first station onto the image plane of the second station using the homography matrix, and then calculating the Euclidean distance between the transformed points and the actual matching points. This distance is called the reprojection error. An adaptive threshold is set that is dynamically adjusted based on the image resolution and expected matching accuracy. If the reprojection error of a matching point pair is greater than , it is marked as a potential outlier.

[0142] S405: Determine a fine matching area of ​​the second projection point in the two-dimensional plane.

[0143] The fine matching area refers to a smaller search region defined on the 2D image plane of the second station, centered on the second projection point. It can be understood as a smaller and more precise local region than the coarse matching area, whose size and shape are dynamically adjusted based on the accuracy of the homography matrix and the expected degree of local deformation. This area is used to further limit the search range during the precise matching process, aiming to improve matching accuracy and efficiency. The fine matching area is primarily used to perform a refined search within a smaller range to find candidate points that best match the key points of the first station.

[0144] Specifically, we first need to determine the base size of the fine matching region based on the uncertainty assessment of the homography matrix. This assessment can be achieved by analyzing the residuals or covariances during the homography matrix estimation process. For example, the statistical distribution of the homography matrix projection error can be used to determine a reasonable search radius.

[0145] After determining the base size, embodiments of the present invention also take into account the specificities of the planetary surface environment to perform adaptive adjustments. For example, in areas with rich textures, a smaller fine matching area can be used, as these areas generally provide more unique features, facilitating accurate matching. Conversely, in areas with sparse or highly repetitive textures, a relatively larger fine matching area may be required to increase the probability of finding a correct match. This adaptive strategy can be implemented by analyzing local image features around the second projection point in the second station image.

[0146] Furthermore, the present invention also considers the impact of the keypoint's depth information in three-dimensional space on the size of the fine matching region. For distant feature points, since their positional variations in the image are relatively small, a smaller fine matching region can be used. However, for nearby feature points, a relatively larger region may be required to accommodate possible positional variations.

[0147] S406: Perform iterative matching on the coarse matching area and the fine matching area to obtain a positioning point.

[0148] The coarse and fine matching regions form a two-stage adaptive matching strategy. Spatially, the coarse matching region often encompasses the fine matching region. The coarse matching region defines a larger area around the first projection point obtained from the initial pose estimate, while the fine matching region is a smaller area around the second projection point, further precisely located using the homography matrix. This inclusion reflects the progressive nature of the matching process, which gradually narrows the search from a large area to a localized area where an accurate match is more likely to be found.

[0149] During the iterative matching process, the relationship between the coarse and fine matching regions changes dynamically. Initially, the coarse matching region may be the primary focus, but as the iterations proceed, the role of the fine matching region gradually increases. Each iteration may update the homography matrix, causing adjustments to the fine matching region. This dynamic relationship ensures continuous optimization and convergence of the matching process.

[0150] Based on the above embodiment, as an optional embodiment, in S406, the step of iteratively matching the coarse matching area and the fine matching area to obtain the positioning point may further include the following steps:

[0151] S501. The intersection of the coarse matching area and the fine matching area is used as the target area. In the target area, a matching point pair that satisfies a dynamic threshold and has the smallest distance is determined as the matching point pair. The dynamic threshold is positively correlated with the number of iterations and the number of matched key points.

[0152] Specifically, the intersection of the coarse and fine matching regions must first be determined. This intersection combines information from the initial pose estimate and the homography matrix, preserving the broad search range of coarse matching while incorporating the precise positioning of fine matching, thereby forming a more reliable search space. Matching within this target region effectively reduces computational effort while ensuring that no potential correct matching points are missed.

[0153] When matching within the target area, the embodiment of the present invention adopts a dynamic threshold strategy. The design of the dynamic threshold takes into account two factors: the number of iterations and the number of matched key points, which can be expressed as:

[0154] ;

[0155] Where, is the number of iterations, is the number of key points that are successfully matched, and the constant 0.3 is the confidence threshold obtained based on the characteristics of the network and a large number of manually selected corresponding points.

[0156] The advantage of this dynamic threshold strategy is that it adaptively adjusts the matching criteria as the matching process progresses. In early iterations, a lower threshold allows the algorithm to obtain more candidate matching points, which is beneficial for establishing preliminary matching relationships. As the number of iterations increases and the number of matching points grows, the threshold is gradually increased, which not only selects more reliable matches but also effectively reduces the probability of false matches.

[0157] To further enhance matching robustness, embodiments of the present invention also incorporate spatial consistency checks into the matching process. For each newly found matching point pair, its spatial relationship with existing matching point pairs is checked to see if it conforms to the expected geometric transformation. This is achieved by comparing the relative positions of the matching point pairs in the two site images. If a matching point pair's spatial relationship is consistent with that of most existing matching point pairs, its credibility is increased; otherwise, it may be a mismatch.

[0158] It should be noted that in practical applications, the parameters of the dynamic threshold and the strictness of the spatial consistency check may need to be adjusted according to specific task requirements and hardware limitations. For example, when computing resources are very limited, the spatial consistency check criteria can be appropriately relaxed in exchange for faster matching speed.

[0159] S502: Use the key points in each matching point pair as positioning points.

[0160] Based on the above embodiment, as an optional embodiment, after matching key points corresponding to two adjacent sites as positioning points based on the descriptor of the planetary rover, the following steps may be further included:

[0161] S601: Acquire the geometric distance between the positioning points. The geometric distance is obtained based on the three-dimensional coordinates of the positioning.

[0162] Geometric distance refers to the Euclidean distance between two positioning points in 3D space. Specifically, geometric distance can be understood as the relative position difference between the positioning points in real physical space, as recovered through triangulation or other 3D reconstruction methods. This distance is used to assess the spatial consistency between positioning points, helping to select stable and reliable positioning points, thereby improving overall positioning accuracy.

[0163] Specifically, when calculating geometric distance, embodiments of the present invention take into account the varying uncertainties associated with feature points at different depths and employ a depth-based weighted distance calculation method. For example, distant feature points, due to the greater uncertainty in their three-dimensional positions, may be assigned a smaller weight, while nearby feature points may be assigned a larger weight. This weighting strategy better reflects the reliability of feature points at different depths.

[0164] S602: Obtain the reprojection error of the positioning point. The reprojection error is obtained by performing pose calculation on the two-dimensional pixel coordinates of the positioning point. Unstable positioning points are screened out based on the geometric distance and the reprojection error.

[0165] Specifically, the 3D coordinates of successfully matched keypoints are used as input to construct a Euclidean distance matrix between all keypoints in object space. The distance differences between corresponding point pairs in the previous and next stations are then compared. If the distance relationship between a point and other points changes significantly between the two stations, the point is considered a potentially unstable point. This method is particularly effective in leveraging the geometric constraints of 3D space and can identify outliers that might be overlooked in 2D image matching.

[0166] Next, the pose can be solved using the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton nonlinear optimization method. This method uses the 3D coordinates of the matching points at the previous station and the 2D pixel coordinates of the corresponding points at the current station as input. As an efficient second-order optimization algorithm, the BFGS method quickly converges to a local optimum and is particularly well-suited for nonlinear optimization problems involving multiple parameters. By minimizing the reprojection error, the BFGS algorithm effectively estimates the relative pose transformation between the two stations.

[0167] The average minimum reprojection error of the matching points is then calculated based on the calculated pose. This error reflects the accuracy of the estimated pose and is an important metric for evaluating overall matching quality. The reprojection error takes into account the distance between each 3D point and the actual observation point after it is projected onto the image plane using the estimated pose transformation and camera intrinsic parameters.

[0168] Finally, an iterative optimization process can be introduced to further improve positioning accuracy. The core of this process is to determine whether the current solution meets the accuracy requirements based on the average minimum reprojection error. An empirical threshold of 5 can be set. If the average minimum reprojection error exceeds this threshold, it can be considered that the current set of matching points and pose estimation is not ideal. In this case, the keypoint pair with the largest reprojection error can be deleted, and the bundle adjustment optimization can be re-performed to solve the pose. This process is repeated until the average minimum reprojection error drops below the threshold or the preset maximum number of iterations is reached.

[0169] This iterative optimization process gradually eliminates possible anomalous matching points, continuously improving the quality of the remaining point set. Furthermore, repeated optimization yields more accurate pose estimates. Furthermore, this method is highly adaptive, automatically adjusting the number and distribution of matching points to achieve the desired accuracy based on actual conditions.

[0170] S104: Positioning the planetary rover based on the positioning point.

[0171] Figure 8 An example of a physical structure diagram of an electronic device is shown below. Figure 8As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call logic instructions in the memory 830 to execute a planetary rover positioning method based on site image matching.

[0172] Furthermore, the logic instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0173] On the other hand, an embodiment of the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the planetary rover positioning method based on site image matching provided by the above methods.

[0174] On the other hand, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the planetary rover positioning method based on site image matching provided by the above methods.

[0175] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0176] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than to limit them. Although the embodiments of the present invention have been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A planetary rover positioning method based on site image matching, characterized in that: include: Acquire target images of the planetary rover at the site; Acquire key points of the local position of the planetary rover in the target image, and descriptors representing local features of the key points; Based on the descriptor of the planetary rover, key points corresponding to two adjacent stations are matched as positioning points; Positioning the planetary rover based on the positioning point; The step of obtaining the key points of the local position of the planetary rover in the target image and the descriptors representing the local features of the key points includes: Inputting the target image into a feature detection network, wherein the feature detection network includes a feature encoder, a key point decoder, and a descriptor decoder; Extracting features of different scales of the target image by the feature encoder; Extracting key points of the local position of the planetary rover in the target image by the key point decoder; The descriptor decoder is used to fuse the features of different scales corresponding to the key points to obtain a descriptor; Obtaining the key points and the descriptors output by the feature detection network; The step of matching key points corresponding to two adjacent sites as positioning points based on the descriptor of the planetary rover includes: Based on the initial pose estimation of the planetary rover, obtaining a first projection point by projecting a three-dimensional key point at a first station onto a two-dimensional plane of a second station, the first station and the second station being two adjacent stations; determining a coarse matching area of ​​the first projection point in the two-dimensional plane; Obtain a matching point pair in the rough matching area, where the matching point pair is two key points whose similarity between descriptors is greater than a threshold; Obtaining a homography matrix obtained by fitting the matching point pairs, and projecting key points in the first station that do not form a matching point pair onto the two-dimensional plane based on the homography matrix to obtain second projection points; determining a fine matching area of ​​the second projection point in the two-dimensional plane; Iterative matching is performed on the coarse matching area and the fine matching area to obtain a positioning point.

2. The planetary rover positioning method based on site image matching according to claim 1, characterized in that: The training process of the feature detection network includes: Obtaining a sample image, inputting the sample image into a feature detection network, and obtaining a sample detection result output by the feature detection network; Based on the loss function, a loss value of the sample detection result is obtained, where the loss value includes a key point detection loss function, a key point repeatability loss function, a descriptor loss function, and an enhanced descriptor loss function; Parameters in the feature detection network are adjusted based on the loss value.

3. The planetary rover positioning method based on site image matching according to claim 1, characterized in that: The iterative matching based on the coarse matching area and the fine matching area to obtain a positioning point includes: taking the intersection of the coarse matching area and the fine matching area as the target area; Determine in the target area a matching point pair that satisfies a dynamic threshold and has the smallest distance as a matching point pair, wherein the dynamic threshold is positively correlated with the number of iterations and the number of matched key points; The key points in each of the matching point pairs are used as positioning points.

4. The planetary rover positioning method based on site image matching according to claim 1, characterized in that: After obtaining the homography matrix obtained by fitting the matching point pairs, the method further includes: Outliers in the matching point pairs are excluded based on the homography matrix.

5. The planetary rover positioning method based on site image matching according to claim 1, characterized in that: After matching key points corresponding to two adjacent sites as positioning points based on the descriptor of the planetary rover, the method further includes: Obtaining a geometric distance between the positioning points, where the geometric distance is obtained based on the three-dimensional coordinates of the positioning points; Obtaining a reprojection error of the positioning point, where the reprojection error is obtained by performing pose solving on the two-dimensional pixel coordinates of the positioning point; Unstable positioning points are screened out based on the geometric distance and the reprojection error.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the planetary rover positioning method based on site image matching as described in any one of claims 1 to 5 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the planetary rover positioning method based on site image matching as described in any one of claims 1 to 5 is implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the planetary rover positioning method based on site image matching as described in any one of claims 1 to 5 is implemented.

Images