Visual positioning method and system
By using pre-trained machine learning models and reinforcement learning optimization, combined with feature fusion and refined processing, the accuracy and robustness issues of visual localization in complex scenarios are solved, and a high-precision and robust visual localization method is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- E SURFING VISION TECHNOLOGY CO LTD
- Filing Date
- 2026-01-29
- Publication Date
- 2026-05-08
AI Technical Summary
Existing visual positioning methods fail to match under conditions of drastic changes in lighting, seasonal changes, occlusion, or significant differences in viewing angle, resulting in inaccurate positioning, especially in scenarios such as repetitive building facades in cities where fuzzy or incorrect positioning occurs.
By employing a pre-trained machine learning model, the system acquires the image to be tested and fuses one-dimensional orientation-aware feature vectors and two-dimensional feature maps to generate a position probability distribution map and orientation vector field. Combined with non-maximum suppression and sub-pixel-level refinement processing, loss calculation and reinforcement learning are optimized to achieve high-precision and robust visual localization.
It achieves high-precision and robust visual positioning in complex scenarios, effectively reduces position and angle estimation errors, provides uncertainty expression for multimodal positioning results, and improves positioning accuracy and robustness.
Smart Images
Figure CN121999047A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer technology, and in particular relates to a visual positioning method and system. Background Technology
[0002] Among related technologies, mainstream techniques for solving localization problems include GNSS and vision-based localization methods. Cross-view pose estimation is a research hotspot in the field of visual localization. This technique typically employs a Siamese-like network structure, containing two deep learning branches that extract features from the ground query image and the aerial reference image, respectively. Then, a metric learning loss function (such as contrastive loss) is used to train the model, enabling it to narrow the feature distance between matching views and widen the feature distance between non-matching views. However, the goal of metric learning is merely to make "matching" images closer in feature distance and "non-matching" images farther away. When encountering drastic changes in lighting, seasonal changes, severe occlusion, or significant differences in viewpoint, this implicit similarity can be very fragile, leading to matching failures and consequently, inaccurate model training and visual localization. Summary of the Invention
[0003] In view of the shortcomings of the prior art, the purpose of the invention is to provide a visual positioning method and system.
[0004] In a first aspect, the present invention provides a visual positioning method, comprising: S1, acquiring a test image, the test image including a query image and a reference image; S2, inputting the test image into a pre-trained machine learning model to obtain a position probability distribution map and a direction vector field aligned with the position probability distribution map, wherein each grid coordinate in the position probability distribution map corresponds to a position probability value, and each grid coordinate corresponds to a direction vector; S3, determining candidate peaks from the position probability distribution map based on the position probability values of each grid coordinate, and performing non-maximum suppression (NMS) and sub-pixel level refinement on multiple candidate peaks to obtain target grid coordinates, so that the grid coordinates are transformed based on a mapping relationship to obtain positioning position coordinates; S4, determining candidate directions from the direction vector field based on the direction vectors corresponding to each grid coordinate, and performing weighted averaging and normalization on multiple candidate directions to obtain a direction angle, and transforming the direction angle to obtain a positioning angle; S5, determining a visual pose based on the positioning position coordinates and the positioning angle.
[0005] Further, the pre-trained machine learning model is trained in the following manner: the query image is input into a query image decoder to obtain a one-dimensional orientation-aware feature vector; the reference image is input into a reference image decoder to obtain a two-dimensional feature map; the one-dimensional orientation-aware feature vector and the two-dimensional feature map are fused to obtain an initial position probability distribution map and an initial orientation vector field; the initial position probability distribution map and the initial orientation vector field are optimized based on loss calculation to obtain the position probability distribution map and the orientation vector field aligned with the position probability distribution map; wherein, inputting the query image into the query image decoder to obtain the one-dimensional orientation-aware feature vector includes: preprocessing the query image, ... The preprocessed query image is input into a convolutional neural network or a visual Transformer backbone network with translational equivariance to extract feature maps from multiple intermediate channels. Using the center of the preprocessed query image or a preset horizon as a reference, the feature maps are divided into multiple discrete directional intervals from a Cartesian coordinate system. Region pooling is performed on the features within each directional interval to obtain directional feature vectors. These directional feature vectors are then input into a set of fully connected layers and / or subnetworks with attention mechanisms to perform weighted modeling of responses in different directions, resulting in encoded directional-aware feature vectors. These encoded directional-aware feature vectors are then normalized to obtain the one-dimensional directional-aware feature vector. The reference image is input into a reference image decoder to obtain a two-dimensional feature map, including: preprocessing the reference image; inputting the preprocessed reference image into a convolutional neural network or a network backbone with a multi-scale feature extraction structure to extract local texture and global scene semantic information layer by layer to obtain a multi-scale feature map; upsampling or downsampling the multi-scale feature map and fusing it using feature pyramid fusion, pixel-wise weighted summation / / or channel concatenation to obtain a fused two-dimensional feature map; based on the fused two-dimensional feature map, compressing or reconstructing the channel dimension through a 1×1 convolution or linear projection layer to obtain a new feature map, and normalizing the new feature map to obtain the two-dimensional feature map; wherein, the reference image is input into a reference image decoder to obtain a two-dimensional feature map. The process of fusing a one-dimensional orientation-aware feature vector and a two-dimensional feature map to obtain an initial position probability distribution map and an initial orientation vector field includes: determining that the dimensions of the one-dimensional orientation-aware feature vector and the two-dimensional feature map are matched; fusing the one-dimensional orientation-aware feature vector and the two-dimensional feature map, and inputting the fused feature map into a convolutional layer to output an unnormalized position score map; normalizing the position score map through a softmax operation to obtain the initial position probability distribution map; using the fused feature map or the two-dimensional feature map as input, outputting an orientation vector at each position through an orientation prediction head, and encoding the camera orientation in the orientation vector; and normalizing the orientation vector to obtain the initial orientation vector field.The optimization of the initial position probability distribution map and the initial direction vector field based on loss calculation to obtain the position probability distribution map and the direction vector field aligned with the position probability distribution map includes: marking the true position of the sample with a ground truth label, based on; Calculate the cross-entropy loss, where, Represents cross-entropy loss, Indicates the truth label, Indicates the actual location. The initial position probability distribution map is represented, and the difference between the ground truth label and the initial position probability distribution map is determined based on the cross-entropy loss; the predicted direction vector of the ground truth position is determined based on... Calculate L2 loss. Indicates L2 loss, This indicates the camera orientation, and the difference between the predicted direction vector of the true position and the initial direction vector field is determined based on the L2 loss; based on Calculate the total loss, where, The balance coefficient representing the cross-entropy loss. The balance coefficient of the L2 loss is represented, and backpropagation is performed on the total loss to obtain the position probability distribution map and the direction vector field aligned with the position probability distribution map.
[0006] Furthermore, it also includes: driving the machine learning model to perform G randomized forward inferences based on noise injection and increased model temperature to obtain G pose prediction results; scoring each pose prediction result based on a reward function, and calculating an average score based on the scores of multiple pose prediction results; determining a target score for each pose prediction result relative to the average score; increasing the prediction probability when the target score is positive, and decreasing the prediction probability when the target score is negative.
[0007] Further, determining candidate peaks from the location probability distribution map based on the location probability value of each grid coordinate includes: determining the location probability values of other grid coordinates within a preset range centered on each grid coordinate; comparing the location probability values of the grid coordinates with the location probability values of other grid coordinates; and, if the location probability value of the grid coordinate is the largest and the location probability value of the grid coordinate is greater than a preset threshold, using the location probability value of the grid coordinate as the candidate peak, thereby obtaining multiple candidate peaks and multiple candidate points.
[0008] Further, performing Non-Maximum Suppression (NMS) and sub-pixel-level refinement on multiple candidate peaks to obtain target grid coordinates includes: determining whether the distance between any two candidate points is less than a preset pixel interval, and obtaining candidate points that are less than the pixel interval; taking the highest value of the candidate peaks among the candidate points less than the pixel interval as a coarse position, and determining the grid coordinates within a preset range of the coarse position; performing a weighted summation of the position probability values of the grid coordinates within the preset range to obtain refined continuous coordinates, and determining the target grid coordinates based on the refined continuous coordinates.
[0009] Further, determining the candidate direction from the direction vector field based on the direction vector corresponding to each grid coordinate includes: determining the neighboring grid coordinates of the target grid coordinates, and using the direction vectors of the neighboring grid coordinates as the candidate direction.
[0010] Further, a weighted average and normalization process is performed on the multiple candidate directions to obtain the direction angle, including: performing a weighted average on the multiple candidate directions to obtain a weighted average direction vector; and normalizing the weighted average direction vector to obtain a processed unit vector; based on The direction angle is obtained, where, Represents the arctangent function in the four quadrants. This represents the direction vector components of the target grid coordinates along the X-axis. This represents the direction vector component of the target grid coordinates on the Y-axis.
[0011] A second aspect of the present invention provides a visual positioning system, comprising: an acquisition module for acquiring a test image, the test image including a query image and a reference image; a obtaining module for inputting the test image into a pre-trained machine learning model to obtain a position probability distribution map and a direction vector field aligned with the position probability distribution map, wherein each grid coordinate in the position probability distribution map corresponds to a position probability value, and each grid coordinate corresponds to a direction vector; a first processing module for determining candidate peaks from the position probability distribution map based on the position probability values of each grid coordinate, and performing non-maximum suppression (NMS) and sub-pixel-level refinement on multiple candidate peaks to obtain target grid coordinates, so that the grid coordinates are transformed based on a mapping relationship to obtain positioning position coordinates; a second processing module for determining candidate directions from the direction vector field based on the direction vectors corresponding to each grid coordinate, and performing weighted averaging and normalization on multiple candidate directions to obtain direction angles, and transforming the direction angles to obtain positioning angles; and a determination module for determining a visual pose based on the positioning position coordinates and the positioning angles.
[0012] A third aspect of the present invention provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method described in any one aspect of the present invention.
[0013] A fourth aspect of the present invention provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described in any one of the first aspects of the present invention.
[0014] The beneficial effects of this invention are as follows: The visual positioning method and system of this invention acquires a test image, which includes a query image and a reference image; inputs the test image into a pre-trained machine learning model to obtain a position probability distribution map and a direction vector field aligned with the position probability distribution map, wherein each grid coordinate in the position probability distribution map corresponds to a position probability value and each grid coordinate corresponds to a direction vector; candidate peaks are determined from the position probability distribution map based on the position probability value of each grid coordinate, and multiple candidate peaks are subjected to non-maximum suppression (NMS) and sub-pixel level refinement to obtain target grid coordinates, so that the grid coordinates are transformed based on a mapping relationship to obtain the positioning position coordinates; candidate directions are determined from the direction vector field based on the direction vector corresponding to each grid coordinate, and multiple candidate directions are weighted averaged and normalized to obtain direction angles, and the direction angles are transformed to obtain positioning angles; the visual pose is determined based on the positioning position coordinates and positioning angles. This method achieves a high-precision, high-robustness, high-efficiency, and highly practical visual positioning method through a parallel prediction architecture of a probabilistic position map and an aligned direction field, combined with a refined post-processing strategy. Attached Figure Description
[0015] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. It is obvious that the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings.
[0016] Figure 1 This is a flowchart of a visual positioning method according to an embodiment of the present invention; Figure 2 This is a flowchart of a machine learning model training method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a visual positioning system according to an embodiment of the present invention; Figure 4 This is a structural block diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation
[0017] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0018] Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts disclosed in this invention.
[0019] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The terms "installed," "connected," and "linked" should be interpreted broadly; for example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.
[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of methods and systems consistent with some aspects of the invention as detailed in the appended claims.
[0021] Currently, mainstream technologies for solving localization problems include GNSS and vision-based localization methods. Among them, cross-view pose estimation is a research hotspot in the field of visual localization. This technique typically employs a Siamese-like network structure, containing two deep learning branches that extract features from the ground query image and the aerial reference image, respectively. Then, the model is trained using a metric learning loss function (such as contrastive loss) to enable it to narrow the feature distance between matching views and widen the feature distance between mismatched views. Some advanced methods, such as those described in "Convolutional Cross-View Pose Estimation," encode ground images by designing convolutional networks with translational isovariance, allowing their feature descriptors to perceive orientation changes, thereby achieving simultaneous estimation of position and orientation.
[0022] However, the following problems exist: In scenarios where visual features are highly similar or exhibit periodic or symmetrical distributions (e.g., repetitive building facades or intersections in cities), models relying solely on feature similarity for matching are prone to confusion, leading to significant ambiguity in the localization results or convergence to incorrect local optima. Most existing methods employ supervised fine-tuning (SFT) for training, resulting in a relatively fixed decision-making logic after training. When faced with ambiguous scenarios exceeding the training data distribution and exhibiting high "perceptual complexity," the model lacks the ability to further explore and optimize its judgment strategy, leading to performance bottlenecks. How to effectively apply reinforcement learning to high-precision pose estimation requiring sub-meter accuracy, and design a reward mechanism that reflects geometric accuracy while effectively guiding model training, is a pressing technical challenge.
[0023] To this end, the present invention proposes a visual positioning method, system and related equipment. Specifically, the visual positioning method, system and related equipment of the present invention are described below with reference to the accompanying drawings.
[0024] Figure 1 This is a flowchart of a visual positioning method according to an embodiment of the present invention. It should be noted that the visual positioning method of this embodiment can be applied to the visual positioning system of this embodiment. This visual positioning system can be configured on an electronic device or in a server. This application does not limit the scope of the application.
[0025] like Figure 1 As shown, the visual positioning method is characterized by comprising: S110, Obtain the image to be tested, which includes the query image and the reference image.
[0026] In an embodiment of the present invention, a test image can be obtained based on a terminal device, and the test image includes a query image and a reference image.
[0027] In this context, a query image can be understood as an unknown image whose location and orientation need to be determined, including but not limited to street view images and shopping mall images. A reference image can be understood as a baseline image that provides spatial coordinates and orientation, such as a satellite image.
[0028] S120, the image to be tested is input into the pre-trained machine learning model to obtain a position probability distribution map and a direction vector field aligned with the position probability distribution map. Each grid coordinate in the position probability distribution map corresponds to a position probability value, and each grid coordinate corresponds to a direction vector.
[0029] In embodiments of the present invention, such as Figure 2 As shown, the machine learning model is trained in the following way: S210, input the query image into the query image decoder to obtain a one-dimensional orientation-aware feature vector.
[0030] In an embodiment of the present invention, the query image is input into a query image decoder to obtain a one-dimensional direction-aware feature vector. This can be achieved as follows: the query image is preprocessed, and the preprocessed query image is input into a convolutional neural network or a visual Transformer backbone network with translational equivariance to extract the feature maps of the intermediate multi-channels; with the center of the preprocessed query image or a preset horizon as a reference, the feature maps are divided into multiple discrete direction intervals from the Cartesian coordinate system, and the features in each direction interval are subjected to region pooling to obtain a direction feature vector; the direction feature vector is input into a set of fully connected layers and / or a sub-network with an attention mechanism to perform weighted modeling of the responses in different directions to obtain an encoded direction-aware feature vector, and the encoded direction-aware feature vector is normalized to obtain a one-dimensional direction-aware feature vector.
[0031] The process involves preprocessing the query image, including scaling and normalization, to obtain an input image with a uniform resolution (i.e., the preprocessed query image).
[0032] The preprocessed query image is input into a convolutional neural network or a visual Transformer backbone network with translation equivariance to extract feature maps from the intermediate multi-channels. ,in, This represents a tensor of size H*W with C channels. Represents the space of real numbers. Indicates the number of rows in the tensor. Represents the space of real numbers. This represents the depth of the tensor, where H and W vary depending on the algorithm settings. H and W are the number of rows and columns of the tensor during neural network processing.
[0033] This process involves dividing the feature map from a Cartesian coordinate system into multiple discrete directional intervals, and performing region pooling on the features within each directional interval to obtain directional feature vectors. For example, the feature map... The system is divided into multiple discrete directional intervals from the Cartesian coordinate system. For example, the 0°~360° range is divided into K directional slots according to the azimuth angle. Region pooling (such as average pooling or max pooling) is performed on the features within each directional slot to obtain a K-dimensional directional feature vector. .
[0034] Through the above steps, the obtained one-dimensional orientation-aware feature vector not only contains the overall semantic information of the query image, but also explicitly encodes the response distribution in different orientation directions, thus possessing orientation-aware capability.
[0035] S220: Input the reference image into the reference image decoder to obtain a two-dimensional feature map.
[0036] In embodiments of the present invention, the reference image is input into a reference image decoder to obtain a two-dimensional feature map, which can be achieved in the following ways: the reference image is preprocessed, and the preprocessed reference image is input into a convolutional neural network or a network backbone with a multi-scale feature extraction structure to extract local texture and global scene semantic information layer by layer to obtain a multi-scale feature map; the multi-scale feature map is upsampled or downsampled, and fused using feature pyramid fusion, pixel-wise weighted summation / / or channel concatenation to obtain a fused two-dimensional feature map; based on the fused two-dimensional feature map, the channel dimension is compressed or reconstructed through a 1×1 convolution or linear projection layer to obtain a new feature map, and the new feature map is normalized to obtain a two-dimensional feature map.
[0037] The process involves preprocessing the reference image, including cropping, scaling, and normalizing, to ensure that the input resolution and pixel value range are within a preset range (i.e., the preprocessed reference image).
[0038] The new feature map is normalized to enhance numerical stability and cross-sample comparability, thereby obtaining a two-dimensional feature map.
[0039] The two-dimensional feature map maintains a one-to-one correspondence with the location of the reference image in space. The feature vector of each location represents the semantic and geometric information of the local region at that location, which is used to align with the features of the query image and infer the location probability distribution.
[0040] S230 fuses the one-dimensional orientation sensing feature vector and the two-dimensional feature map to obtain the initial position probability distribution map and the initial orientation vector field.
[0041] In embodiments of the present invention, the one-dimensional orientation-aware feature vector and the two-dimensional feature map are fused to obtain an initial position probability distribution map and an initial orientation vector field. This can be achieved by: ensuring that the dimensions of the one-dimensional orientation-aware feature vector and the two-dimensional feature map are matched; fusing the one-dimensional orientation-aware feature vector and the two-dimensional feature map, and inputting the fused feature map into a convolutional layer to output an unnormalized position score map; normalizing the position score map using a softmax operation to obtain an initial position probability distribution map; using the fused feature map or the two-dimensional feature map as input, outputting an orientation vector at each position through an orientation prediction head, and encoding the camera orientation in the orientation vector; and normalizing the orientation vector to obtain the initial orientation vector field.
[0042] The process of determining that the size of the one-dimensional orientation-aware feature vector and the two-dimensional feature map are matched includes: mapping the one-dimensional orientation-aware feature vector to an embedding vector with channel number suppression of the reference feature map (i.e., the mapped feature vector), and broadcasting and copying the mapped feature vector in the spatial dimension so that its size matches that of the two-dimensional feature map.
[0043] in, The initial position probability distribution map is obtained, where, This represents the summation of all points in a two-dimensional feature map. This represents a location rating map. Represents row i and column j. This represents the value of point (i, j) on the two-dimensional feature map S. The initial position probability distribution map reflects the model's confidence that different spatial locations are the actual shooting locations.
[0044] The orientation field is obtained by taking the fused feature map or two-dimensional feature map as input and outputting the orientation vector at each position through the orientation prediction head. The output at each position Indicates the camera's orientation, where, This represents the number of rows in the initial position probability distribution tensor. This represents the number of columns in the initial position probability distribution tensor, where... and This is the result of neural network processing.
[0045] Specifically, the direction vector is normalized so that it represents only directional information and is unaffected by scale. The direction field is spatially aligned with the initial position probability distribution map, which can simultaneously express the confidence level and predicted orientation for each position.
[0046] Based on the above, the machine learning model completed the forward propagation and provided its own set of "pose assumptions" for the input image.
[0047] S240, based on loss calculation, optimize the initial position probability distribution map and the initial direction vector field to obtain the position probability distribution map and the direction vector field aligned with the position probability distribution map.
[0048] In embodiments of the present invention, the true location of the sample is marked with a truth label, based on... Calculate the cross-entropy loss, where, Represents cross-entropy loss, Indicates the truth label, Indicates the actual location. This involves representing the initial position probability distribution map and determining the difference between the ground truth label and the initial position probability distribution map based on cross-entropy loss; determining the predicted direction vector of the true position based on... Calculate L2 loss. Indicates L2 loss, This represents the camera orientation, and the difference between the predicted direction vector and the initial direction vector field, determined based on L2 loss, is used to determine the true position. Calculate the total loss, where, The balance coefficient representing the cross-entropy loss. This represents the balance coefficient of the L2 loss, and backpropagation is performed on the total loss to obtain the location probability distribution map and the direction vector field aligned with the location probability distribution map.
[0049] In other words, once the machine learning model has completed forward propagation, the aforementioned predictions can be quantitatively compared with the known true values, i.e., the machine learning model can be evaluated by calculating the loss; thereby improving the machine learning model.
[0050] The logic behind the difference between the ground truth label and the initial position probability distribution is as follows: the true position (usually treated as a Gaussian distribution centered on the ground truth) is considered the standard answer, and the difference between the probability distribution predicted by the machine learning model and this "standard answer" is the goal. The aim is to make the initial probability distribution predicted by the machine learning model show a sharp peak at the true position.
[0051] The logic for determining the difference between the predicted direction vector at the true location and the initial direction vector field is as follows: In the direction field D, find the predicted direction vector corresponding to the true location and calculate its difference from the true direction vector. The goal is to ensure that the model predicts the completely correct direction at the true location point.
[0052] In this process, given the difference between the true label and the initial position probability distribution map, and the difference between the predicted direction vector of the true position and the initial direction vector field, the total error is calculated to adjust all parameters in the machine learning model in reverse, so that the machine learning model can do better next time.
[0053] S250, based on noise injection and increased model temperature, drives the machine learning model to perform G forward inferences with randomness, and obtains G pose prediction results.
[0054] S260 scores each pose prediction result based on the reward function and calculates the average score based on the scores of multiple pose prediction results.
[0055] In an embodiment of the present invention, based on a reward function Each pose prediction result is scored, where... This represents the weighting coefficient of the IoU term. This represents the location bounding box obtained from the i-th prediction. Let represent the pose prediction obtained from the i-th model inference. Represents the ground-truth location bounding box. This represents the weighting coefficient for the location distance error term. This represents the pixel position coordinates predicted in the i-th prediction. This represents the pixel position coordinates predicted in the i-th prediction. This represents the weighting coefficient of the orientation angle error term. This represents the deviation between the predicted angle and the true angle, and is used to score multiple pose prediction results. Calculate the average score The average score represents the model's average performance or baseline in the current state (for this training sample).
[0056] S270, determine the target score for each pose prediction result relative to the average score.
[0057] In embodiments of the present invention, based on Calculate the target score, or advantage score, of each pose prediction result relative to the average score.
[0058] S280, when the target score is positive, the machine learning model increases the prediction probability; when the target score is negative, the machine learning model decreases the prediction probability.
[0059] In an embodiment of the present invention, if the target score is greater than 0, it indicates a prediction. A performance better than the average is a good prediction, indicating that the model's learning has increased its probability of occurrence; a target score less than 0 indicates a poor prediction. The performance was worse than the average, which is a bad prediction, meaning the model learned to reduce its probability of occurrence. This, in turn, optimized the decision-making strategy of the machine learning model in scenarios with high perceptual complexity.
[0060] Higher accuracy: The introduction of reinforcement learning enables the model to transcend the limitations of supervised learning and discover better perception strategies through exploratory learning. Especially when dealing with scenes with high "perceptual complexity" (such as visual blur and scene symmetry), it can make more accurate judgments and significantly reduce the estimation error of position and angle.
[0061] Enhanced robustness: This method outputs a probability distribution of location rather than a single coordinate point, effectively representing the uncertainty of the localization result. This multimodal output capability enables it to capture multiple possible locations simultaneously in ambiguous scenes, providing richer and more reliable information for downstream applications.
[0062] Training is efficient and autonomous: The rule-based reinforcement learning and direct visual discrimination reward function avoid the reliance on expensive manual preference labeling, making the policy optimization process more direct and efficient, and enabling the model to learn autonomously to a greater extent.
[0063] Good versatility: The trained model can be directly applied to query images with different field of view without modification, and can effectively utilize externally provided coarse orientation prior information to further improve positioning accuracy.
[0064] S130: Determine candidate peaks based on the position probability values of each grid coordinate from the position probability distribution map, and perform non-maximum suppression (NMS) and sub-pixel level refinement on multiple candidate peaks to obtain target grid coordinates, so that the grid coordinates are transformed based on the mapping relationship to obtain the positioning position coordinates.
[0065] In an embodiment of the present invention, based on each grid coordinate, the position probability values of other grid coordinates within a preset range centered on the grid coordinate are determined; the position probability values of the grid coordinates are compared with the position probability values of other grid coordinates; when the position probability value of the grid coordinate is the largest and the position probability value of the grid coordinate is greater than a preset threshold, the position probability value of the grid coordinate is used as a candidate peak value to obtain multiple candidate peak values and multiple candidate points.
[0066] In an embodiment of the present invention, when multiple candidate peaks and multiple candidate points are obtained, it is determined whether the distance between any two candidate points is less than a preset pixel interval, and candidate points less than the pixel interval are obtained; the highest value of the candidate peaks among the candidate points less than the pixel interval is taken as the rough position, and the grid coordinates within a preset range of the rough position are determined; the position probability values of the grid coordinates within the preset range are weighted and summed to obtain refined continuous coordinates, and the target grid coordinates are determined based on the refined continuous coordinates.
[0067] Among them, the refined continuous coordinates can be understood as the calculation target, and the target grid coordinates (refined grid coordinates) can be understood as a practical approximation for subsequent operations.
[0068] The refined continuous coordinates are essentially the theoretically optimal solution obtained through a sub-pixel refinement algorithm, i.e., a probability-weighted average within the neighborhood of the coarse peak point: ,in, Indicates a preset range for approximate location. This represents the probability value of the location. Indicates the actual position, u represents the column number, and v represents the row number.
[0069] The target grid coordinates (refined grid coordinates) are essentially a practical discrete approximation of continuous coordinates.
[0070] Therefore, continuous coordinates are a direct result of subpixel refinement and are used for operations requiring high precision, while target grid coordinates are a discrete approximation of continuous coordinates and are used for operations requiring integer indices.
[0071] The grid coordinates are transformed based on mapping relationships to obtain the location coordinates. For example, mapping relationships include, but are not limited to, affine transformation, homography mapping, or geographic projection of a known scale.
[0072] S140: Determine candidate directions from the direction vector field based on the direction vector corresponding to each grid coordinate, and perform weighted averaging and normalization on multiple candidate directions to obtain the direction angle, and then transform the direction angle to obtain the positioning angle.
[0073] In an embodiment of the present invention, the neighboring grid coordinates of the target grid coordinates are determined, and the direction vectors of the neighboring grid coordinates are used as candidate directions.
[0074] Among them, the grid point closest to the target grid coordinates (refined grid coordinates) or several grid points within a preset range of the target grid coordinates can be selected as the neighboring grid coordinates of the target grid coordinates, and the direction vector of the neighboring grid coordinates can be used as the candidate direction.
[0075] In an embodiment of the present invention, a weighted average is performed on multiple candidate directions to obtain a weighted average direction vector; the weighted average direction vector is then normalized to obtain a processed unit vector; based on We obtain the direction angle, where, Represents the arctangent function in the four quadrants. This represents the direction vector components of the target grid coordinates along the X-axis. This represents the direction vector component of the target grid coordinates on the Y-axis.
[0076] To suppress noise and local instability, a weighted average is calculated from multiple candidate directions to obtain a weighted average direction vector. The weights can be selected as the probability value at the corresponding location or a decay function based on the distance to the peak, for example: , ,in, P{pos}(u,v) represents the Gaussian weights, where each position (u,v) corresponds to a direction vector. = M represents a set of local regions, specifically a 3×3 region centered at u and v and surrounding it.
[0077] In this process, the direction vector of the weighted average is normalized to a magnitude of 1, thus retaining only the direction information.
[0078] S150 determines the visual pose based on the positioning coordinates and positioning angle.
[0079] In embodiments of the present invention, visual pose includes positioning coordinates and positioning angle.
[0080] According to an embodiment of the present invention, a visual localization method is used to acquire a test image, which includes a query image and a reference image. The test image is input into a pre-trained machine learning model to obtain a position probability distribution map and a direction vector field aligned with the position probability distribution map. Each grid coordinate in the position probability distribution map corresponds to a position probability value, and each grid coordinate corresponds to a direction vector. Candidate peaks are determined from the position probability distribution map based on the position probability value of each grid coordinate. Non-maximum suppression (NMS) and sub-pixel-level refinement are performed on multiple candidate peaks to obtain target grid coordinates. The grid coordinates are then transformed based on a mapping relationship to obtain the localization position coordinates. Candidate directions are determined from the direction vector field based on the direction vector corresponding to each grid coordinate. Multiple candidate directions are weighted averaged and normalized to obtain direction angles. The direction angles are then transformed to obtain the localization angle. The visual pose is determined based on the localization position coordinates and the localization angle. This method achieves a high-precision, high-robustness, high-efficiency, and highly practical visual localization method through a parallel prediction architecture of a probabilistic position map and an aligned direction field, combined with a refined post-processing strategy.
[0081] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.
[0082] According to one aspect of the present invention, a visual positioning system is also proposed. Figure 3 This is a schematic diagram of a visual positioning system according to an embodiment of the present invention; as shown Figure 3 As shown, it includes: The acquisition module 310 is used to acquire the image to be tested, which includes a query image and a reference image; The module 320 is used to input the image to be tested into a pre-trained machine learning model to obtain a position probability distribution map and a direction vector field aligned with the position probability distribution map, wherein each grid coordinate in the position probability distribution map corresponds to a position probability value and each grid coordinate corresponds to a direction vector; The first processing module 330 is used to determine candidate peaks from the position probability distribution map based on the position probability value of each grid coordinate, and to perform non-maximum suppression (NMS) and sub-pixel level refinement on multiple candidate peaks to obtain target grid coordinates, so that the grid coordinates are transformed based on the mapping relationship to obtain the positioning position coordinates; The second processing module 340 is used to determine candidate directions from the direction vector field based on the direction vector corresponding to each grid coordinate, and to perform weighted averaging and normalization on multiple candidate directions to obtain direction angles, and to transform the direction angles to obtain positioning angles; The determination module 350 is used to determine the visual pose based on the positioning position coordinates and the positioning angle.
[0083] According to an embodiment of the present invention, a visual positioning system acquires a test image, which includes a query image and a reference image; the test image is input into a pre-trained machine learning model to obtain a position probability distribution map and a direction vector field aligned with the position probability distribution map, wherein each grid coordinate in the position probability distribution map corresponds to a position probability value and each grid coordinate corresponds to a direction vector; candidate peaks are determined from the position probability distribution map based on the position probability value of each grid coordinate, and multiple candidate peaks are subjected to non-maximum suppression (NMS) and sub-pixel level refinement to obtain target grid coordinates, so that the grid coordinates are transformed based on a mapping relationship to obtain the positioning position coordinates; candidate directions are determined from the direction vector field based on the direction vector corresponding to each grid coordinate, and multiple candidate directions are subjected to weighted averaging and normalization to obtain direction angles, and the direction angles are transformed to obtain positioning angles; the visual pose is determined based on the positioning position coordinates and positioning angles. This method achieves a high-precision, high-robustness, high-efficiency, and highly practical visual positioning method through a parallel prediction architecture of a probabilistic position map and an aligned direction field, combined with a refined post-processing strategy.
[0084] Optionally, the pre-trained machine learning model is trained as follows: The query image is input into a query image decoder to obtain a one-dimensional orientation-aware feature vector; the reference image is input into a reference image decoder to obtain a two-dimensional feature map; the one-dimensional orientation-aware feature vector and the two-dimensional feature map are fused to obtain an initial position probability distribution map and an initial orientation vector field; the initial position probability distribution map and the initial orientation vector field are optimized based on loss calculation to obtain the position probability distribution map and the orientation vector field aligned with the position probability distribution map; wherein, inputting the query image into the query image decoder to obtain the one-dimensional orientation-aware feature vector includes: preprocessing the query image, and... The processed query image is input into a convolutional neural network or a visual Transformer backbone network with translational equivariance to extract feature maps from multiple intermediate channels. Using the center of the preprocessed query image or a preset horizon as a reference, the feature maps are divided into multiple discrete directional intervals from a Cartesian coordinate system. Region pooling is performed on the features within each directional interval to obtain directional feature vectors. These directional feature vectors are then input into a set of fully connected layers and / or sub-networks with attention mechanisms to perform weighted modeling of responses in different directions, resulting in encoded direction-aware feature vectors. These encoded direction-aware feature vectors are then normalized to obtain the one-dimensional direction-aware feature vector. The reference image is input into a reference image decoder to obtain a two-dimensional feature map, including: preprocessing the reference image; inputting the preprocessed reference image into a convolutional neural network or a network backbone with a multi-scale feature extraction structure to extract local texture and global scene semantic information layer by layer to obtain a multi-scale feature map; upsampling or downsampling the multi-scale feature map and fusing it using feature pyramid fusion, pixel-wise weighted summation / / or channel concatenation to obtain a fused two-dimensional feature map; based on the fused two-dimensional feature map, compressing or reconstructing the channel dimension through a 1×1 convolution or linear projection layer to obtain a new feature map, and normalizing the new feature map to obtain the two-dimensional feature map; wherein, the reference image is input into a reference image decoder to obtain a two-dimensional feature map. The process of fusing a one-dimensional orientation-aware feature vector and a two-dimensional feature map to obtain an initial position probability distribution map and an initial orientation vector field includes: determining that the dimensions of the one-dimensional orientation-aware feature vector and the two-dimensional feature map are matched; fusing the one-dimensional orientation-aware feature vector and the two-dimensional feature map, and inputting the fused feature map into a convolutional layer to output an unnormalized position score map; normalizing the position score map through a softmax operation to obtain the initial position probability distribution map; using the fused feature map or the two-dimensional feature map as input, outputting an orientation vector at each position through an orientation prediction head, and encoding the camera orientation in the orientation vector; and normalizing the orientation vector to obtain the initial orientation vector field.The optimization of the initial position probability distribution map and the initial direction vector field based on loss calculation to obtain the position probability distribution map and the direction vector field aligned with the position probability distribution map includes: marking the true position of the sample with a ground truth label, based on; Calculate the cross-entropy loss, where, Represents cross-entropy loss, Indicates the truth label, Indicates the actual location. The initial position probability distribution map is represented, and the difference between the ground truth label and the initial position probability distribution map is determined based on the cross-entropy loss; the predicted direction vector of the ground truth position is determined based on... Calculate L2 loss. Indicates L2 loss, This indicates the camera orientation, and the difference between the predicted direction vector of the true position and the initial direction vector field is determined based on the L2 loss; based on Calculate the total loss, where, The balance coefficient representing the cross-entropy loss. The balance coefficient of the L2 loss is represented, and backpropagation is performed on the total loss to obtain the position probability distribution map and the direction vector field aligned with the position probability distribution map.
[0085] Optionally, the method further includes: driving the machine learning model to perform G randomized forward inferences based on noise injection and increased model temperature to obtain G pose prediction results; scoring each pose prediction result based on a reward function, and calculating an average score based on the scores of multiple pose prediction results; determining a target score for each pose prediction result relative to the average score; increasing the prediction probability when the target score is positive, and decreasing the prediction probability when the target score is negative.
[0086] Optionally, the first processing module 330 is specifically configured to determine the position probability value of other grid coordinates within a preset range centered on each grid coordinate; compare the position probability value of the grid coordinate with the position probability value of other grid coordinates; and, if the position probability value of the grid coordinate is the largest and the position probability value of the grid coordinate is greater than a preset threshold, use the position probability value of the grid coordinate as the candidate peak value to obtain multiple candidate peak values and multiple candidate points.
[0087] Optionally, the first processing module 330 is specifically used to determine whether the distance between any two candidate points among the plurality of candidate points is less than a preset pixel interval, and to obtain the candidate points that are less than the pixel interval; to take the highest value of the candidate peak among the candidate points that are less than the pixel interval as a rough position, and to determine the grid coordinates within a preset range of the rough position; to perform a weighted summation of the position probability values of the grid coordinates within the preset range to obtain refined continuous coordinates, and to determine the target grid coordinates based on the refined continuous coordinates.
[0088] Optionally, the second processing module 340 is specifically used to determine the neighboring grid coordinates of the target grid coordinates and use the direction vector of the neighboring grid coordinates as the candidate direction.
[0089] Optionally, the second processing module 340 is specifically used to perform a weighted average of the multiple candidate directions to obtain a weighted average direction vector; to perform normalization processing on the weighted average direction vector to obtain a processed unit vector; based on The direction angle is obtained, where, Represents the arctangent function in the four quadrants. This represents the direction vector components of the target grid coordinates along the X-axis. This represents the direction vector component of the target grid coordinates on the Y-axis.
[0090] According to one aspect of the present invention, an electronic device is provided.
[0091] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Figure 4 As shown, an electronic device may include one or more ( Figure 4 Only one is shown in the image. A processor 102 (which may include, but is not limited to, a microprocessor unit (MPU) or a programmable logic device (PLD)) and a memory 104 for storing data are also shown. In one exemplary embodiment, the electronic device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 4 The structure shown is for illustrative purposes only and does not limit the structure of the terminal device described above. For example, the terminal device may also include components that are more... Figure 4 The more or fewer components shown, or having the same Figure 4 Equivalent functions or ratios shown Figure 4 The functions shown have more different configurations.
[0092] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the visual positioning method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to terminal devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0093] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the switching device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.
[0094] The present invention proposes a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform a visual positioning method.
[0095] The applicant of this invention has provided a detailed description of the embodiments of the invention in conjunction with the accompanying drawings. However, those skilled in the art should understand that the above embodiments are merely preferred embodiments of the invention. The detailed description is only intended to help readers better understand the spirit of the invention and is not intended to limit the scope of protection of the invention. On the contrary, any improvements or modifications made based on the inventive spirit of the invention should fall within the scope of protection of the invention.
[0096] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0097] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.
[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.
Claims
1. A visual positioning method, characterized in that, include: S1, Obtain the image to be tested, the image to be tested includes a query image and a reference image; S2, the image to be tested is input into the pre-trained machine learning model to obtain a position probability distribution map and a direction vector field aligned with the position probability distribution map, wherein each grid coordinate in the position probability distribution map corresponds to a position probability value and each grid coordinate corresponds to a direction vector; S3, determine candidate peaks based on the position probability values of each grid coordinate from the position probability distribution map, and perform non-maximum suppression (NMS) and sub-pixel level refinement on multiple candidate peaks to obtain target grid coordinates, so that the grid coordinates are transformed based on the mapping relationship to obtain the positioning position coordinates; S4, determine candidate directions from the direction vector field based on the direction vector corresponding to each grid coordinate, and perform weighted averaging and normalization on multiple candidate directions to obtain direction angles, and transform the direction angles to obtain positioning angles; S5. Determine the visual pose based on the positioning coordinates and the positioning angle.
2. The visual positioning method according to claim 1, characterized in that, The pre-trained machine learning model is trained in the following way: The query image is input into the query image decoder to obtain a one-dimensional orientation-aware feature vector; The reference image is input into a reference image decoder to obtain a two-dimensional feature map; The one-dimensional orientation sensing feature vector and the two-dimensional feature map are fused to obtain an initial position probability distribution map and an initial orientation vector field; The initial position probability distribution map and the initial direction vector field are optimized based on loss calculation to obtain the position probability distribution map and the direction vector field aligned with the position probability distribution map; The query image is input into the query image decoder to obtain a one-dimensional orientation-aware feature vector, including: The query image is preprocessed, and the preprocessed query image is input into a convolutional neural network or a visual Transformer backbone network with translation equivariance to extract the feature maps of the intermediate multi-channels. Using the center of the preprocessed query image or a preset horizon as a reference, the feature map is divided from the Cartesian coordinate system into multiple discrete directional intervals, and the features in each directional interval are subjected to region pooling to obtain directional feature vectors. The directional feature vector is input into a set of fully connected layers and / or a sub-network with an attention mechanism. The responses in different directions are weighted and modeled to obtain the encoded directional awareness feature vector. The encoded directional awareness feature vector is then normalized to obtain the one-dimensional directional awareness feature vector. The reference image is input into a reference image decoder to obtain a two-dimensional feature map, including: The reference image is preprocessed, and the preprocessed reference image is input into a convolutional neural network or a network backbone with a multi-scale feature extraction structure. Local texture and global scene semantic information are extracted layer by layer to obtain a multi-scale feature map. The multi-scale feature map is upsampled or downsampled, and then fused using feature pyramid fusion, pixel-wise weighted summation / or channel stitching to obtain a fused two-dimensional feature map. Based on the fused two-dimensional feature map, the channel dimension is compressed or reconstructed through a 1×1 convolution or linear projection layer to obtain a new feature map, and the new feature map is normalized to obtain the two-dimensional feature map. The process of fusing the one-dimensional orientation sensing feature vector and the two-dimensional feature map to obtain an initial position probability distribution map and an initial orientation vector field includes: The dimensions of the one-dimensional orientation-sensing feature vector and the two-dimensional feature map are determined to be compatible. The one-dimensional orientation-aware feature vector and the two-dimensional feature map are fused, and the fused feature map is input into a convolutional layer to output an unnormalized position score map. The position score map is then normalized using a softmax operation to obtain the initial position probability distribution map. Using the fused feature map or the two-dimensional feature map as input, the orientation prediction head outputs an orientation vector at each position, and the orientation vector encodes the camera orientation. The direction vector is normalized to obtain the initial direction vector field; Specifically, the initial position probability distribution map and the initial direction vector field are optimized based on loss calculation to obtain the position probability distribution map and the direction vector field aligned with the position probability distribution map, including: Label the true location of the sample with a ground truth label, based on Calculate the cross-entropy loss, where, Represents cross-entropy loss, Indicates the truth label, Indicates the actual location. The initial position probability distribution map is represented, and the difference between the truth label and the initial position probability distribution map is determined based on the cross-entropy loss; Determine the predicted direction vector of the true position, based on Calculate L2 loss. Indicates L2 loss, The camera orientation is indicated, and the difference between the predicted direction vector of the true position and the initial direction vector field is determined based on the L2 loss. based on Calculate the total loss, where, The balance coefficient representing the cross-entropy loss. The balance coefficient of the L2 loss is represented, and backpropagation is performed on the total loss to obtain the position probability distribution map and the direction vector field aligned with the position probability distribution map.
3. The visual positioning method according to claim 2, characterized in that, Also includes: Based on noise injection and increased model temperature, the machine learning model is driven to perform G randomized forward inferences to obtain G pose prediction results. Based on the reward function, each pose prediction result is scored, and the average score is calculated based on the scores of multiple pose prediction results. Determine a target score for each of the pose prediction results relative to the average score; When the target score is positive, the machine learning model increases the prediction probability; when the target score is negative, the machine learning model decreases the prediction probability.
4. The visual positioning method according to claim 1, characterized in that, Determining candidate peaks from the location probability distribution map based on the location probability value of each grid coordinate includes: Based on each grid coordinate, determine the position probability value of other grid coordinates within a preset range centered on the grid coordinate; Compare the position probability values of the grid coordinates with the position probability values of other grid coordinates; When the position probability value of the grid coordinate is the maximum and the position probability value of the grid coordinate is greater than a preset threshold, the position probability value of the grid coordinate is used as the candidate peak value, resulting in multiple candidate peak values and multiple candidate points.
5. The visual positioning method according to claim 4, characterized in that, Non-maximum suppression (NMS) and sub-pixel refinement are performed on multiple candidate peaks to obtain the target grid coordinates, including: Determine whether the distance between any two candidate points is less than a preset pixel interval, and obtain the candidate points that are less than the pixel interval; The highest value of the candidate peak among the candidate points smaller than the pixel interval is taken as the rough position, and the grid coordinates within a preset range of the rough position are determined. The position probability values of the grid coordinates within the preset range are weighted and summed to obtain refined continuous coordinates, and the target grid coordinates are determined based on the refined continuous coordinates.
6. The visual positioning method according to claim 5, characterized in that, Determining candidate directions from the direction vector field based on the direction vector corresponding to each grid coordinate includes: Determine the neighboring grid coordinates of the target grid coordinates, and use the direction vectors of the neighboring grid coordinates as the candidate directions.
7. The visual positioning method according to claim 6, characterized in that, The candidate directions are weighted, averaged, and normalized to obtain the direction angle, including: A weighted average of the multiple candidate directions is obtained to obtain a weighted average direction vector. The weighted average direction vector is normalized to obtain the processed unit vector; based on The direction angle is obtained, where, Represents the arctangent function in the four quadrants. This represents the direction vector components of the target grid coordinates along the X-axis. This represents the direction vector component of the target grid coordinates on the Y-axis.
8. A visual positioning system, characterized in that, include: The acquisition module is used to acquire the image to be tested, which includes a query image and a reference image; The module is used to input the image to be tested into a pre-trained machine learning model to obtain a position probability distribution map and a direction vector field aligned with the position probability distribution map, wherein each grid coordinate in the position probability distribution map corresponds to a position probability value and each grid coordinate corresponds to a direction vector; The first processing module is used to determine candidate peaks from the position probability distribution map based on the position probability value of each grid coordinate, and to perform non-maximum suppression (NMS) and sub-pixel level refinement on multiple candidate peaks to obtain target grid coordinates, so that the grid coordinates are transformed based on the mapping relationship to obtain the positioning position coordinates; The second processing module is used to determine candidate directions from the direction vector field based on the direction vector corresponding to each grid coordinate, and to perform weighted averaging and normalization on multiple candidate directions to obtain direction angles, and to transform the direction angles to obtain positioning angles; The determination module is used to determine the visual pose based on the positioning coordinates and the positioning angle.
9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 7.