An absolute pose regression method and related devices

Abstract

This invention discloses an absolute pose regression method and related equipment. The method includes: extracting a global feature vector from a query image using a deep neural network; inputting the global feature vector into a surrogate regression subnetwork to obtain a geometrically perceptual surrogate; extracting the original estimate of pose parameters using an analytical solver based on the geometrically perceptual surrogate, and then obtaining a standardized initial estimate through standardization processing; and using a sign correction subnetwork to eliminate the sign ambiguity of the standardized initial estimate, and a scale recovery subnetwork to recover the true physical scale of the translation parameters, thereby obtaining the camera pose. This invention, by allowing the neural network to regress the surrogate instead of directly outputting the pose, and combining analytical solving with post-processing correction, enables the network to implicitly learn the geometric structure of the scene, thus significantly improving the positioning accuracy. This method can effectively improve the accuracy of absolute pose regression and can be widely applied in the field of image processing technology.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an absolute pose regression method and related equipment. Background Technology

[0002] Visual relocalization is a key technology in computer vision. Its task is to calculate the camera pose of a newly acquired query image in the scene coordinate system based on a map or model of a known target scene (including explicit 3D modeling or neural network implicit modeling). This technology has broad application prospects in fields such as autonomous driving, augmented reality, and robot autonomous navigation.

[0003] Currently, visual relocalization techniques are mainly divided into two categories: structure-based methods and absolute pose regression methods.

[0004] Structure-based methods (such as feature point matching and PnP solving) construct explicit matching relationships between 3D scene features and 2D image features, and then use projective geometric constraints to solve for camera pose. Although these methods have high localization accuracy, their computational process is complex, involving multiple stages such as feature detection, description, matching, and geometric optimization, which is time-consuming and difficult to meet the needs of applications with high real-time requirements.

[0005] Absolute pose regression (APR) is an end-to-end localization paradigm that has emerged in recent years. Its core idea is to train a deep neural network to directly regress camera pose parameters from the input image. Compared to structure-based methods, APR, with its end-to-end inference architecture, offers advantages such as fast inference speed and low computational overhead, making it an important research direction for real-time localization applications. However, this type of method also faces significant technical bottlenecks: because the network directly maps image features to pose values ​​through fully connected layers, the entire process is "black box" and lacks explicit perception of scene geometry. Specifically, existing methods ignore the objectively existing projection geometric constraints during camera imaging, preventing the network from truly understanding the correspondence between 3D scene structure and 2D image features. This lack of geometric perception leads to a theoretical upper limit on localization accuracy and weak generalization ability—when faced with new perspectives or complex scenes not present in the training set, the network is prone to outputting predictions that violate geometric common sense. Summary of the Invention

[0006] The main objective of this invention is to provide an absolute pose regression method, apparatus, electronic device, storage medium, and program product, aiming to solve at least one problem in the prior art.

[0007] To achieve the above objectives, one aspect of the present invention proposes an absolute pose regression method, the method comprising: Obtain the query image to be located; Based on the query image, a global feature vector is extracted using a deep neural network. The global feature vector is input into the proxy regression subnetwork, and regression is used to obtain the geometry-aware proxy. The geometry-aware proxy uses the Gram matrix, which is used to encode the projective geometric constraint information between the geometry of the 3D scene and the features of the 2D image. The geometry-aware proxy includes rotation geometry-aware proxy and translation geometry-aware proxy. Based on the geometric perception agent, the original estimates of pose parameters are extracted by the analytical solver, and then standardized initial estimates are obtained through standardization processing; the standardized initial estimates include standardized rotation initial estimates and standardized translation initial estimates. Based on the standardized initial estimate, the symbolic ambiguity of the standardized initial estimate is eliminated by using a symbolic correction subnetwork, and the true physical scale of the translation parameters is recovered by using a scale recovery subnetwork, thus obtaining the camera pose; wherein, the camera pose includes the rotation matrix and the translation vector.

[0008] In some implementations, feature vectors are extracted from the query image using a deep neural network, including the following steps: The query image is input into a deep neural network, and forward inference maps the query image into a high-dimensional feature map. Deep neural networks include residual networks or self-attention-based network architectures. The high-dimensional feature map is compressed into a one-dimensional global feature vector by using a global average pooling layer.

[0009] In some implementations, the surrogate regression subnetwork employs a multilayer perceptron composed of fully connected layers. The surrogate regression subnetwork includes a rotation surrogate regression subnetwork and a translation surrogate regression subnetwork. The global feature vector is input into the surrogate regression subnetwork, and regression yields a geometrically perceptual surrogate, including the following steps: The global feature vector is input into the rotated surrogate regression subnetwork so that the rotated surrogate regression subnetwork outputs a rotated lower triangular vector; wherein the rotated lower triangular vector includes a first number of lower triangular elements; By utilizing the symmetry property of the matrix, the rotated lower triangular vector is reshaped and filled to obtain a real symmetric Gram matrix in the first dimension as a rotational geometry perception proxy. The global feature vector is input into the translation surrogate regression subnetwork so that the translation surrogate regression subnetwork outputs a translation lower triangular vector; wherein the translation lower triangular vector includes a second number of lower triangular elements; By utilizing the symmetry properties of matrices, the translated lower triangular vectors are reshaped and filled to obtain a second-dimensional real symmetric Gram matrix as a translational geometry perception proxy.

[0010] In some implementations, based on a geometry-aware agent, the original estimates of pose parameters are extracted through an analytical solver, and then standardized initial estimates are obtained through standardization processing, including the following steps: Based on rotational geometry-aware proxy, the eigenvector corresponding to the minimum eigenvalue is obtained by using an eigenvalue decomposition algorithm based on inverse power iteration as the original estimate of rotation parameters; The original estimates of the rotation parameters are subjected to Schmitt orthogonalization to obtain the standardized initial estimates of the rotation. Based on the translation geometry-aware proxy, the eigenvector corresponding to the minimum eigenvalue is obtained by processing the eigenvalue decomposition algorithm based on inverse power iteration as the original estimate of the translation parameter; The original translation parameter estimates are subjected to L2 normalization to obtain the standardized initial translation estimate as the unit direction vector.

[0011] In some implementations, the sign correction subnetwork includes a rotation sign correction subnetwork and a translation sign correction subnetwork. Based on the standardized initial estimate, the sign correction subnetwork is used to eliminate the sign ambiguity of the standardized initial estimate, including the following steps: The standardized rotation initial estimate is concatenated with the global feature vector to obtain the first concatenated vector; The first concatenated vector is input into the rotation sign correction subnetwork so that the rotation sign correction subnetwork outputs the rotation sign correction factor; Multiplying the rotation sign correction factor by the standardized initial rotation estimate yields the rotation parameter estimate; The rotation parameter estimates are reshaped into a matrix of a preset dimension to obtain the rotation matrix; The standardized translation initial estimate is concatenated with the global feature vector to obtain the second concatenated vector; The second concatenation vector is input into the translation sign correction subnetwork so that the translation sign correction subnetwork outputs the translation sign correction factor; Multiplying the translation sign correction factor by the standardized initial translation estimate yields the translation parameter estimate.

[0012] In some implementations, the true physical scale of the translation parameters is recovered using a scale recovery subnetwork, including the following steps: The translation parameter estimate is concatenated with the global feature vector to obtain the third concatenated vector; The third concatenation vector is input into the scale recovery subnetwork so that the scale recovery subnetwork outputs a positive real scalar; where the positive real scalar represents the true physical magnitude of the translation vector. Applying the actual physical modulus to the translation parameter estimation, the translation vector is recovered.

[0013] In some implementations, the method further includes the following steps: Obtain the query image sample and its corresponding real camera pose label; wherein, the real camera pose label includes the vector form of the real rotation matrix and the real translation vector; Based on the query image samples, the global feature vector of the samples is extracted through the original deep neural network; Input the global feature vector of the sample into the original agent regression subnetwork to obtain the geometrically perceptual agent of the sample; Based on the sample geometry-aware agent, the original estimates of the sample pose parameters are extracted by the original analytical solver, and then the sample standardized initial estimates are obtained through standardization processing; wherein, the sample standardized initial estimates include sample standardized rotation initial estimates and sample standardized translation initial estimates; Based on the real camera pose labels and sample standardization initial estimates, the initial estimated loss is obtained by constructing the square of the cosine similarity. Binary labels are generated based on the dot product of the initial standardized estimate of the samples and the actual camera pose labels; Based on the initial sample standardization estimate, the symbolic ambiguity of the initial sample standardization estimate is eliminated by the original symbolic correction subnetwork to obtain the symbolic parameter estimate, and the true physical scale of the translation parameter is recovered by the original scale recovery subnetwork to obtain the sample camera pose; wherein, the symbolic parameter estimate includes the sample rotation parameter estimate and the sample translation parameter estimate, and the sample camera pose includes the sample rotation matrix and the sample translation vector; Based on binary classification labels and symbol parameter estimation, a symbol correction loss is constructed using binary cross-entropy. Based on real camera pose labels and sample camera poses, the final pose loss is constructed using mean squared error. The total loss function is obtained by weighted summing of the initial estimated loss, sign correction loss, and final pose loss. Based on the total loss function, the backpropagation algorithm is used to optimize and update the network parameters of the original deep neural network, the original surrogate regression subnetwork, the original analytical solver, the original sign correction subnetwork, and the original scale recovery subnetwork, so as to obtain the deep neural network, the surrogate regression subnetwork, the analytical solver, the sign correction network, and the scale recovery subnetwork.

[0014] To achieve the above objectives, another aspect of the present invention provides an absolute pose regression device, the device comprising: The first module is used to obtain the query image to be located; The second module is used to extract global feature vectors based on the query image using a deep neural network. The third module is used to input the global feature vector into the proxy regression subnetwork and obtain the geometry-aware proxy. The geometry-aware proxy uses the Gram matrix, which is used to encode the projective geometric constraint information between the geometry of the 3D scene and the features of the 2D image. The geometry-aware proxy includes rotation geometry-aware proxy and translation geometry-aware proxy. The fourth module is used to extract the original estimates of pose parameters through an analytical solver based on a geometrically perceptual agent, and then obtain standardized initial estimates through standardization processing; wherein, the standardized initial estimates include standardized rotation initial estimates and standardized translation initial estimates; The fifth module is used to eliminate the symbolic ambiguity of the standardized initial estimate using a symbolic correction subnetwork and to recover the true physical scale of the translation parameters using a scale recovery subnetwork, thereby obtaining the camera pose; wherein the camera pose includes the rotation matrix and the translation vector.

[0015] In some embodiments, the apparatus further includes a sixth module for performing the following operations: Obtain the query image sample and its corresponding real camera pose label; wherein, the real camera pose label includes the vector form of the real rotation matrix and the real translation vector; Based on the query image samples, the global feature vector of the samples is extracted through the original deep neural network; Input the global feature vector of the sample into the original agent regression subnetwork to obtain the geometrically perceptual agent of the sample; Based on the sample geometry-aware agent, the original estimates of the sample pose parameters are extracted by the original analytical solver, and then the sample standardized initial estimates are obtained through standardization processing; wherein, the sample standardized initial estimates include sample standardized rotation initial estimates and sample standardized translation initial estimates; Based on the real camera pose labels and sample standardization initial estimates, the initial estimated loss is obtained by constructing the square of the cosine similarity. Binary labels are generated based on the dot product of the initial standardized estimate of the samples and the actual camera pose labels; Based on the initial sample standardization estimate, the symbolic ambiguity of the initial sample standardization estimate is eliminated by the original symbolic correction subnetwork to obtain the symbolic parameter estimate, and the true physical scale of the translation parameter is recovered by the original scale recovery subnetwork to obtain the sample camera pose; wherein, the symbolic parameter estimate includes the sample rotation parameter estimate and the sample translation parameter estimate, and the sample camera pose includes the sample rotation matrix and the sample translation vector; Based on binary classification labels and symbol parameter estimation, a symbol correction loss is constructed using binary cross-entropy. Based on real camera pose labels and sample camera poses, the final pose loss is constructed using mean squared error. The total loss function is obtained by weighted summing of the initial estimated loss, sign correction loss, and final pose loss. Based on the total loss function, the backpropagation algorithm is used to optimize and update the network parameters of the original deep neural network, the original surrogate regression subnetwork, the original analytical solver, the original sign correction subnetwork, and the original scale recovery subnetwork, so as to obtain the deep neural network, the surrogate regression subnetwork, the analytical solver, the sign correction network, and the scale recovery subnetwork.

[0016] To achieve the above objectives, another aspect of the present invention provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned method.

[0017] To achieve the above objectives, another aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method.

[0018] To achieve the above objectives, another aspect of the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method.

[0019] The present invention provides at least the following beneficial effects: It offers an absolute pose regression method, apparatus, electronic device, storage medium, and program product. This method involves acquiring a query image to be located; extracting a global feature vector from the query image using a deep neural network; inputting the global feature vector into a proxy regression subnetwork to obtain a geometric perception proxy; wherein the geometric perception proxy uses a Gram matrix, which encodes the projective geometric constraint information between the geometry of the 3D scene and the features of the 2D image; the geometric perception proxy includes a rotational geometric perception proxy and a translational geometric perception proxy; based on the geometric perception proxy, an original estimate of the pose parameters is extracted using an analytical solver, and then standardized initial estimates are obtained through standardization processing; wherein the standardized initial estimates include standardized rotational initial estimates and standardized translational initial estimates; based on the standardized initial estimates, a sign correction subnetwork is used to eliminate the sign ambiguity of the standardized initial estimates, and a scale recovery subnetwork is used to recover the true physical scale of the translation parameters to obtain the camera pose; wherein the camera pose includes a rotation matrix and a translation vector. This invention abandons the traditional approach of directly regressing pose values ​​and innovatively introduces a geometric perception proxy—a Gram matrix of a specific dimension. This matrix encodes the projective geometric constraint information between the geometry of the 3D scene and the features of the 2D image. Specifically, by having the neural network regress this proxy instead of directly outputting the pose, and combining analytical solving with post-processing correction, the network can implicitly learn the geometric structure of the scene, thereby significantly improving the localization accuracy and effectively enhancing the accuracy of absolute pose regression methods. Furthermore, the overall scheme of this invention still retains the end-to-end inference process, which can meet the requirements of high real-time performance. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of an implementation environment for the absolute pose regression method provided in this embodiment of the invention; Figure 2 This is a flowchart illustrating the absolute pose regression method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the overall process of the absolute pose regression method provided in the embodiments of the present invention; Figure 4 This is a schematic diagram illustrating the flowchart and principle of the absolute pose regression method provided in this embodiment of the invention. Figure 5 This is a schematic diagram of the absolute pose regression device provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0022] It is understood that the terms "first," "second," etc., used in this invention may be used to describe various concepts, but unless specifically stated otherwise, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of embodiments of this invention, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words "if" or "when" as used herein may be interpreted as "when," "in response to determination," or "in the event of a determination."

[0023] The terms “at least one,” “multiple,” “each,” “any,” etc., used in this invention, “at least one” includes one, two, or more than two; “multiple” includes two or more than two; “each” refers to each of the corresponding multiple; and “any” refers to any one of the multiple.

[0024] Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this invention is for descriptive purposes only and is not intended to limit the invention.

[0025] To facilitate understanding of the technical content of the present invention, the meanings of the terms related to the technical features that may be involved in the present invention will be explained first: Visual relocalization refers to the process of calculating the camera pose in the target scene coordinate system for a newly acquired query image, given a known map or model of the target scene (including explicit modeling or implicit neural network modeling). This invention is primarily applied to this task.

[0026] Absolute pose regression: a paradigm of visual relocalization. It refers to training a neural network to establish a direct mapping from an input image to the camera pose in the scene coordinate system. This process does not require explicit feature matching. After training, for a new input query image, the network can directly infer its camera pose.

[0027] Deep learning originates from research on artificial neural networks. It discovers distributed feature representations of data by combining low-level features to form more abstract high-level representations of attribute categories or features.

[0028] Camera pose: A parameter describing the spatial state of the camera in a coordinate system, containing six degrees of freedom: namely, the translation vector describing the coordinates of the camera's optical center and the rotation matrix describing the orientation of the camera's optical axis.

[0029] Geometric Perception Agent: This invention proposes an intermediate supervised representation (specifically embodied in a Gram matrix). It acts as a bridge connecting high-level image features and camera pose, explicitly encoding the projective geometric constraints between 3D scene geometry and 2D image features in a neural network, replacing direct numerical regression in traditional methods.

[0030] Gram matrix: A positive semi-definite matrix in linear algebra. In this invention, it specifically refers to an algebraic structure built upon absolute pose estimation theory, which is constructed from 3D scene points / lines and 2D image features based on projection constraints, and its feature vectors contain solution information for rotation and translation. The network implicitly perceives the scene's geometric structure and geometric projection relationships by regressing this matrix.

[0031] In related technologies, how to improve the positioning accuracy and geometric interpretability of the absolute pose regression method while maintaining its high efficiency advantage has become a technical problem that urgently needs to be solved in this field.

[0032] In view of this, this invention provides an absolute pose regression method and related device. The method involves acquiring a query image to be located; extracting a global feature vector from the query image using a deep neural network; inputting the global feature vector into a proxy regression subnetwork to obtain a geometric perception proxy; wherein the geometric perception proxy uses a Gram matrix, which encodes the projective geometric constraint information between the geometry of the 3D scene and the features of the 2D image; the geometric perception proxy includes a rotational geometric perception proxy and a translational geometric perception proxy; based on the geometric perception proxy, an original estimate of the pose parameters is extracted using an analytical solver, and then standardized initial estimates are obtained through standardization; wherein the standardized initial estimates include standardized rotational initial estimates and standardized translational initial estimates; based on the standardized initial estimates, a sign correction subnetwork is used to eliminate the sign ambiguity of the standardized initial estimates, and a scale recovery subnetwork is used to recover the true physical scale of the translation parameters to obtain the camera pose; wherein the camera pose includes a rotation matrix and a translation vector. This invention abandons the traditional approach of directly regressing pose values ​​and innovatively introduces a geometric perception proxy—a Gram matrix of a specific dimension. This matrix encodes the projective geometric constraint information between the geometry of the 3D scene and the features of the 2D image. Specifically, by having the neural network regress this proxy instead of directly outputting the pose, and combining analytical solving with post-processing correction, the network can implicitly learn the geometric structure of the scene, thereby significantly improving the localization accuracy and effectively enhancing the accuracy of the absolute pose regression method. Furthermore, the overall scheme of this invention still retains the end-to-end inference process, which can meet the requirements of high real-time performance.

[0033] It is understood that the absolute pose regression method provided by this invention can be applied to any computer device with data processing and computing capabilities, and this computer device can be various terminals or servers. When the computer device in the embodiment is a server, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal can be a smartphone, tablet, laptop, or desktop computer, but it is not limited to these.

[0034] like Figure 1 The diagram shown is a schematic representation of an implementation environment provided by an embodiment of the present invention. (Refer to...) Figure 1The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected via a network, either wirelessly or via a wired connection, to complete data transmission and exchange.

[0035] Server 101 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0036] Additionally, server 101 can also be a node server in a blockchain network. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms.

[0037] Terminal 102 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. Terminal 102 and server 101 can be directly or indirectly connected via wired or wireless communication, and this embodiment of the invention does not impose any limitations.

[0038] For example, based on Figure 1 The implementation environment shown in this embodiment of the invention provides an absolute pose regression method. The following description uses the application of this absolute pose regression method in server 101 as an example. It can be understood that this absolute pose regression method can also be applied in terminal 102.

[0039] Reference Figure 2 , Figure 2 This is an optional flowchart of an absolute pose regression method provided in an embodiment of the present invention. The execution subject of this absolute pose regression method can be any of the aforementioned computer devices (including servers or terminals). Figure 2 The method may include, but is not limited to, steps S100 to S500.

[0040] Step S100: Obtain the query image to be located; Step S200: Based on the query image, a global feature vector is extracted using a deep neural network; It should be noted that in some embodiments, step S200 may include the following steps: inputting the query image into a deep neural network, mapping the query image into a high-dimensional feature map through forward inference; wherein, the deep neural network includes a residual network or a network architecture based on self-attention; and compressing the high-dimensional feature map into a one-dimensional global feature vector through a global average pooling layer.

[0041] For example, in some specific implementations, the detailed steps of feature extraction can be implemented as follows: Step 1: Feature Encoding: Encode the query image to be located (denoted as...) The input image is fed into a pre-trained deep neural network (i.e., the backbone network). This backbone network can be a residual network (such as the ResNet series) or a self-attention-based network architecture (such as the Transformer series). The backbone network maps the input image into a high-dimensional feature map (denoted as ) through forward inference. ).

[0042] Step 2: Global Pooling: Apply a global average pooling layer to the above high-dimensional feature map. The data is processed and compressed into a one-dimensional global feature vector (denoted as ). ,in (Indicates the number of feature channels). This vector aggregates global contextual information from the image, serving as the input basis for subsequent geometrically perceptual proxy regression steps.

[0043] Step S300: Input the global feature vector into the proxy regression subnetwork and obtain the geometric perception proxy through regression; Among them, the geometry-aware agent uses a Gram matrix, which is used to encode the projective geometric constraint information between the geometry of the 3D scene and the features of the 2D image. The geometry-aware agent includes a rotation geometry-aware agent and a translation geometry-aware agent. It should be noted that the surrogate regression subnetwork adopts a multilayer perceptron composed of fully connected layers. The surrogate regression subnetwork includes a rotation surrogate regression subnetwork and a translation surrogate regression subnetwork. In some embodiments, step S300 may include the following steps: inputting the global feature vector into the rotation surrogate regression subnetwork so that the rotation surrogate regression subnetwork outputs a rotational lower triangular vector; wherein, the rotational lower triangular vector includes a first number of lower triangular elements; using the symmetry property of the matrix, reshaping and filling the rotational lower triangular vector to obtain a first-dimensional real symmetric Gram matrix as a rotational geometry perception surrogate; inputting the global feature vector into the translation surrogate regression subnetwork so that the translation surrogate regression subnetwork outputs a translational lower triangular vector; wherein, the translational lower triangular vector includes a second number of lower triangular elements; using the symmetry property of the matrix, reshaping and filling the translational lower triangular vector to obtain a second-dimensional real symmetric Gram matrix as a translational geometry perception surrogate.

[0044] For example, in some specific implementations, the detailed steps of geometry-aware proxy regression can be implemented as follows: (1) Explanation of technical principles: The core of this step lies in introducing a specific Gram matrix as a geometric perception proxy, based on the classical absolute pose estimation theory. In traditional geometric solutions, a set of homogeneous linear equations (of the form of...) can be constructed based on the projection correspondence between 3D scene points / lines and 2D image features. ),in The coefficient matrix and vector This system of equations contains the pose parameters to be determined. According to algebraic theory, the solution to this system of equations (i.e., the pose parameters) corresponds to the Gram form of the coefficient matrix (i.e.,...). The eigenvector corresponding to the smallest eigenvalue of the matrix is ​​given by the eigenvector. Based on this principle, the neural network designed in this invention does not directly regress the pose, but rather regresses this Gram matrix. Because this matrix mathematically encodes the constraints of scene structure and projective geometry, the network can indirectly grasp the geometric structure information of the scene by learning to regress this matrix.

[0045] (2) The specific detailed steps are as follows: Step 1: Processing branch construction (corresponding to the branch structure in the diagram): Process the global feature vector output from Step 1. Simultaneously, the input is fed into two parallel multilayer perceptrons consisting of fully connected layers: namely, the rotated surrogate regression subnetwork (denoted as...). ) and translation agent return subnetwork (denoted as This allows for the construction of independent rotation parameter regression branches and translation parameter regression branches, which are used to extract features of different pose attributes.

[0046] Step 2: Regressing the Lower Triangular Vector of the Proxy: Since the geometric perception proxy (Gram matrix) possesses the property of a real symmetric matrix, this invention utilizes this property to reduce the redundancy of the network output. Specifically, the two regression branches mentioned above are used to regress the lower triangular elements of the geometric perception proxy (Gram matrix) corresponding to the rotation and translation parameters, respectively: Regression of the rotated surrogate lower triangular vector: setting the rotation parameters corresponding to The rotation matrix has 9 elements, therefore its corresponding geometric perception proxy (Gram matrix) is: This is a real symmetric matrix. Calculations show that the lower triangular element count of this matrix is ​​45, therefore a rotated surrogate regression subnetwork is configured. Output a one-dimensional vector with dimension 45 (denoted as ). ), which serves as the rotated lower triangular vector.

[0047] Lower triangular vector of the regression translation proxy: The translation parameter is set to correspond to a translation vector containing 3 elements; therefore, its corresponding geometric perception proxy (Gram matrix) is... The matrix is ​​a real symmetric matrix. Calculations show that the lower triangular element count is 6; therefore, a translational surrogate regression subnetwork is configured. Output a one-dimensional vector with dimension 6 (denoted as ). ), which serves as a translational lower triangular vector.

[0048] Step 3: Construct a geometry-aware proxy: Utilizing the symmetry property of matrices, reshape and fill the one-dimensional vector output above to construct a complete real symmetric matrix, which serves as the corresponding geometry-aware proxy (Gram matrix). Rotation proxy construction: The above 45-dimensional vector Mapped to The lower triangular portion of the matrix (containing diagonal elements) is used, and the values ​​are copied and filled into the upper triangular portion using symmetry to generate the complete matrix. Real symmetric matrix .

[0049] Translation proxy construction: The above 6-dimensional vector Mapped to The lower triangular portion of the matrix (containing diagonal elements) is used, and the values ​​are copied and filled into the upper triangular portion using symmetry to generate the complete matrix. Real symmetric matrix .

[0050] To facilitate understanding of the principles and logic of the embodiments of the present invention, the implementation principles of geometric perception proxy regression in the training and inference phases are explained in detail below: 1. Training Phase: In the training phase of this invention, the numerical values ​​of the Gram matrix are not supervised by explicitly constructing the target matrix using 3D structural information, nor is it necessary to use 3D structural information as the basis for constructing the supervision signal. Specifically, this invention adopts an end-to-end supervision strategy: since the analytical solver used in step three is fully differentiable, the gradient generated by the loss function (calculated based on the estimated pose and the true pose of the network output) can pass through the analytical solution step and be directly fed back to the Gram matrix regression sub-network. Under this mechanism, the network will spontaneously and implicitly learn how to construct the Gram matrix and continuously evolve its numerical representation, so that its analytically derived pose feature vector continuously approaches the true label. Therefore, the training phase does not require the construction of an explicit target matrix, nor does it require inputting any 3D structural information.

[0051] 2. Reasoning stage: During the inference phase, the network requires no 3D information whatsoever; it can regress the Gram matrix solely from image features. This is because, during training, the geometric projection rules and structural constraints of the target scene are implicitly modeled and encoded into the weight parameters of the neural network through the backpropagation algorithm. Therefore, when actually performing the relocalization (i.e., inference) task, the system only needs to input the query image to be localized. The Gram matrix regression subnetwork can directly leverage the high-dimensional image features extracted by the backbone network. This mapping yields a specific Gram matrix. This approach enables a direct conversion from "pure image input" to "geometric perception proxy," completely eliminating reliance on external 3D point clouds, 3D maps, or other auxiliary geometric information.

[0052] Step S400: Based on the geometric perception agent, the original estimate of the pose parameters is extracted by the analytical solver, and then the standardized initial estimate is obtained through standardization. The standardized initial estimates include standardized rotation initial estimates and standardized translation initial estimates; It should be noted that in some embodiments, step S400 may include the following steps: based on the rotation geometry-aware agent, the eigenvector corresponding to the minimum eigenvalue is processed by the eigenvalue decomposition algorithm based on inverse power iteration as the original estimate of the rotation parameters; the original estimate of the rotation parameters is subjected to Schmitt orthogonalization to obtain a standardized initial estimate of the rotation; based on the translation geometry-aware agent, the eigenvector corresponding to the minimum eigenvalue is processed by the eigenvalue decomposition algorithm based on inverse power iteration as the original estimate of the translation parameters; the original estimate of the translation parameters is subjected to L2 normalization to obtain a standardized initial estimate of the translation as the unit direction vector.

[0053] For example, in some specific implementations, the detailed steps of the analytical solution can be implemented as follows: Step 1: Parameter Analysis and Solving: Using algebraic methods, the initial pose parameters are extracted from the specific Gram matrix constructed in Step 2 using a differentiable analytical solver (specifically, an eigenvalue decomposition algorithm based on power iteration). Analytical solution of rotation parameters: for the constructed geometry-aware proxy corresponding to the rotation parameters Calculate its eigenvalues ​​and corresponding eigenvectors. Based on the absolute pose estimation theory, select the eigenvector corresponding to the smallest eigenvalue as the original estimate of the rotation parameters (denoted as ). ).

[0054] Analytical solution of translation parameters: for the constructed geometry-aware proxy corresponding to the translation parameters Calculate its eigenvalues ​​and corresponding eigenvectors. Based on the absolute pose estimation theory, select the eigenvector corresponding to the smallest eigenvalue as the original estimate of the translation parameters (denoted as ). ).

[0055] Step 2: Parameter Standardization: To adapt to the input requirements of subsequent pose correction steps and satisfy geometric constraints, the initial solution vectors calculated above are standardized. Rotation parameter standardization: the calculated 9-dimensional rotation primitive estimate Performing Schmidt orthogonalization to conform to the orthogonal geometric specification of the rotation matrix yields a standardized initial estimate of the rotation (denoted as ). ).

[0056] Translation parameter standardization: This involves standardizing the calculated 3D translation parameters. Perform L2 normalization to make its magnitude 1, thus obtaining the pure translation direction vector (denoted as ). ).

[0057] To facilitate understanding of the principles and logic of the embodiments of the present invention, the implementation principle of the analytical solver will be explained in detail below: 1. Regarding the mathematical mapping from the Gram matrix to the pose vector in the analytical solver: This mapping principle is based on the algebraic properties of the least squares solution in absolute pose estimation (APE).

[0058] Mathematical logic: In pose estimation tasks, a homogeneous system of linear equations is typically constructed. This describes the geometric constraints. According to algebraic theory, in the presence of noise, the optimal solution to this system of equations corresponds to the Gram matrix of the coefficient matrix. The eigenvector corresponding to the smallest eigenvalue.

[0059] Relationship with Direct Linear Transformation (DLT): The basic idea of ​​this invention is inspired by the DLT principle, but this step (step three) specifically performs the process of extracting core feature vectors from the regressed geometric attribute matrix (i.e., the Gram matrix). This can be seen as an explicit neural network implementation of the "eigenvalue calculation stage" in the DLT principle.

[0060] 2. Specific implementation methods and algorithm principles for differentiability: Implementation principle: This invention employs an approximate eigenvalue solution scheme based on inverse power iteration. This process consists entirely of basic linear algebra operators such as matrix multiplication, vector subtraction, and norm normalization. Since these basic operators all possess continuously differentiable functions within the deep learning framework, the entire iterative process is mathematically differentiable, ensuring that the gradient of the loss function is successfully backpropagated to the front-end network.

[0061] Algorithm pseudocode: Input: A specific Gram matrix obtained from neural network regression. Preset number of iterations .

[0062] Output: The eigenvector corresponding to the smallest eigenvalue.

[0063] Algorithm steps: (1) Matrix preprocessing: Calculate the auxiliary matrix ,in This is a preset, extremely small constant (e.g., 0.001) used to ensure the numerical stability of matrix inversion.

[0064] (2) Vector initialization: Randomly initialize a unit initial vector. (Modulus length is 1, dimension is...) match).

[0065] (3) Differentiable iterative computation: For Execute the following loop: a. Vector update: ; b. Vector L2 norm normalization; (4) Return result: Output the feature vector obtained from the final iteration. .

[0066] Comparative advantages: Ensuring continuous gradient propagation (avoiding gradient blocking): Traditional eigenvalue decomposition methods typically require sorting eigenvalues ​​and selecting specific eigenvectors (e.g., using the logical index of the minimum value, argmin). This selective operation often involves discontinuous discrete logic, which can easily lead to gradient blocking or numerical instability in the computational graph during neural network training. In contrast, the inverse power iteration method used in this invention spontaneously converges to the eigenvector corresponding to the minimum eigenvalue through a fixed number of algebraic loops. The entire process does not involve any discontinuous operators based on judgment or selection, ensuring that the gradient can be propagated back along the iteration path completely and smoothly, thus achieving true end-to-end training.

[0067] Numerical Stability: Standard eigenvalue solving algorithms often suffer from severe gradient explosion when dealing with "repeated eigenvalues" (i.e., two extremely close eigenvalues ​​in a matrix), as the denominator approaches zero. This invention effectively "softens" this mathematical singularity by using a fixed-step numerical approximation strategy through inverse power iteration. This approximation mechanism avoids the risk of infinite derivatives that may be encountered during exact solutions, thus significantly improving the numerical stability of the network during the training phase.

[0068] 3. Regarding numerical stability measures: Although it lacks complex regularization terms, the inverse power iteration algorithm includes a stability guarantee (see the algorithm pseudocode above): during each inverse power iteration, this invention performs L2 norm normalization on the candidate vectors. This operation is not only a requirement for the convergence of the inverse power iteration algorithm, but also plays a role in preventing numerical overflow and constraining the prediction range in the forward inference of the neural network, ensuring that the backpropagated gradient is within a reasonable order of magnitude.

[0069] Step S500: Based on the standardized initial estimate, the sign correction subnetwork is used to eliminate the sign ambiguity of the standardized initial estimate, and the scale recovery subnetwork is used to recover the true physical scale of the translation parameters to obtain the camera pose; wherein, the camera pose includes the rotation matrix and the translation vector. It should be noted that the sign correction subnetwork includes a rotation sign correction subnetwork and a translation sign correction subnetwork. In some embodiments, based on the standardized initial estimate, the sign correction subnetwork is used to eliminate the sign ambiguity of the standardized initial estimate, which may include the following steps: concatenating the standardized rotation initial estimate with the global feature vector to obtain a first concatenated vector; inputting the first concatenated vector into the rotation sign correction subnetwork so that the rotation sign correction subnetwork outputs a rotation sign correction factor; multiplying the rotation sign correction factor with the standardized rotation initial estimate to obtain a rotation parameter estimate; reshaping the rotation parameter estimate into a matrix of a preset dimension to obtain a rotation matrix; concatenating the standardized translation initial estimate with the global feature vector to obtain a second concatenated vector; inputting the second concatenated vector into the translation sign correction subnetwork so that the translation sign correction subnetwork outputs a translation sign correction factor; multiplying the translation sign correction factor with the standardized translation initial estimate to obtain a translation parameter estimate.

[0070] It should be noted that in some embodiments, recovering the true physical scale of the translation parameters using the scale recovery subnetwork may include the following steps: concatenating the translation parameter estimate with the global feature vector to obtain a third concatenated vector; inputting the third concatenated vector into the scale recovery subnetwork so that the scale recovery subnetwork outputs a positive real scalar; wherein, the positive real scalar represents the true physical magnitude of the translation vector; applying the true physical magnitude to the translation parameter estimate to recover the translation vector.

[0071] For example, in some specific embodiments, step S500 can be implemented by the following steps: Because the initial pose solution (i.e., the standardized initial estimate) obtained in step S400 has mathematical ambiguity (i.e., uncertainty in the direction of the eigenvectors), and the translation vector loses its true physical scale, this step uses a neural network to correct the initial pose parameters. The detailed steps are as follows: Step 1: Parameter Sign Correction: Construct sign correction subnetworks for rotation and translation parameters respectively (denoted as...). and These subnetworks were designed as binary classification models (the network is configured to first output a probability value representing whether the original orientation is retained). Then, it is mapped to an interval through a linear transformation. (Used as a sign correction factor), it aims to eliminate the directional ambiguity of eigenvectors in analytical solutions: Rotation parameter sign correction: This corrects the standardized initial rotation estimate. Compared with the global feature vector extracted in step one The data is then spliced ​​and input into the rotation sign correction subnetwork. The network predicts a sign correction factor (denoted as ) used to correct the vector direction. Then, the scalar is multiplied by the initial vector to obtain the rotation parameter estimate after direction correction (denoted as...). ).

[0072] Translation parameter sign correction: This corrects the standardized initial translation estimate. Compared with the global feature vector extracted in step one The data is then spliced ​​and input into the translation symbol correction subnetwork. The network predicts a sign correction factor (denoted as ) used to correct the vector direction. Then, the scalar is multiplied by the initial vector to obtain the translation parameter estimate after direction correction (denoted as ). ).

[0073] Step 2: Translation Parameter Scale Recovery: For the translation branch, due to the translation parameter estimation after direction correction... The modulus is still 1, lacking scale information, therefore scale recovery is required: Construct a scale recovery subnetwork (denoted as...) Its input includes global feature vectors. And the translation parameter estimation after sign correction .

[0074] This subnetwork outputs a positive real scalar (denoted as ) through regression operations. ), which is the true physical magnitude of the translation vector.

[0075] Apply the modulus length to This allows us to recover the translation vector with a true physical scale. ).

[0076] Step 3: Pose Output: Convert the corrected pose parameters into a standard pose representation and output them. Output rotation matrix: Estimates the corrected rotation parameters Remodeling into the standard Rotation matrix .

[0077] Output translation vector: the corrected translation vector. Output the translation vector.

[0078] To facilitate understanding of the principles and logic of the embodiments of the present invention, the implementation principle of the symbol correction subnetwork will be explained in detail below: 1. Logic regarding the utilization of input features: In this invention, the sign correction subnetwork does not simply calculate numerical differences, but rather incorporates global features. As a background for judgment, a consistency check is performed on the initial estimate: Global context support: due to the global features extracted by the backbone network It contains global contextual information of the entire target scene (i.e., the macroscopic geometric distribution and semantic features of the scene), which provides a reference frame for pose determination.

[0079] Symbolic disambiguation logic: Although the initial estimate provides the geometric axis, it suffers from algebraic ambiguity, being either "forward" or "backward." The role of the subnetwork is to address this ambiguity within a known global scene context. Under this premise, observe whether the current "initial estimate" conforms to the geometric projection logic of the scene. Through end-to-end training, the subnetwork can learn to recognize... The potential correlation between the initial solution and the "initial estimate" can be used to determine whether the initial solution needs to be flipped, thus achieving the purpose of sign disambiguation.

[0080] 2. Regarding lightweight network architecture: The symbol correction subnetwork used in this invention ( and This can be achieved using a very lightweight multilayer perceptron (MLP) structure. To ensure millisecond-level processing speed, each subnetwork consists of only 2-3 fully connected layers supplemented with nonlinear activation functions.

[0081] Implementation details: An "MLP + binary classification" architecture is adopted. The network outputs a probability value activated by a sigmoid function, representing the confidence level of the initial estimate that "preserves the original direction". Subsequently, a simple linear mapping is used to transform the probability into a correction factor of [-1, 1], which is then multiplied by the initial estimate. This structure ensures high-precision disambiguation while having extremely low computational load, without affecting the overall system's real-time performance.

[0082] Furthermore, to facilitate understanding of the principles and logic of the embodiments of the present invention, the implementation principles of the scale recovery of each branch are explained in detail below: 1. Regarding the restoration of the scale of the translated branch: The scale recovery subnetwork does indeed only apply to the translation branch, and its input does indeed only contain the translation estimate after direction correction. With global feature vectors The logic of this design is as follows: the translation parameter represents the absolute position of the camera in space, and its initial solution obtained by analytical solution... In the standardization step, the vectors are scaled to unit vectors, losing the magnitude information representing the true physical distance. Therefore, a scale recovery subnetwork must be used to utilize feature vectors containing scene context. Combined with direction vector To find the true physical scale of this pose.

[0083] 2. Explanation regarding the requirement for dimensional restoration of rotating branches: In contrast, rotation branches do not require scale recovery. This is because rotation matrices are mathematically subject to strict orthogonality constraints. Through parameter normalization (such as Schmidt orthogonalization) in the aforementioned steps, the geometric specification of the rotation matrix can be enforced, ensuring that the magnitude of each row / column vector is 1 and that they are orthogonal to each other. This means that the scale of the rotation part is geometrically inherent, and directional ambiguity can be eliminated simply by sign correction, without the need for additional learning of physical magnitudes through subnetworks as in translation branches.

[0084] It should be noted that in some embodiments, the method may further include the following steps: obtaining query image samples and their corresponding real camera pose labels; wherein, the real camera pose labels include vector forms of real rotation matrices and real translation vectors; based on the query image samples, extracting global feature vectors of the samples through the original deep neural network; inputting the global feature vectors of the samples into the original proxy regression subnetwork to obtain sample geometric perception proxies; based on the sample geometric perception proxies, extracting the original estimates of sample pose parameters through the original analytical solver, and then obtaining sample standardized initial estimates through standardization processing; wherein, the sample standardized initial estimates include sample standardized rotation initial estimates and sample standardized translation initial estimates; based on the real camera pose labels and sample standardized initial estimates, constructing an initial estimation loss using the square of cosine similarity; generating binary classification labels based on the dot product sign of the sample standardized initial estimates and the real camera pose labels; based on the sample standardized initial estimates, using the original... The symbolic correction subnetwork eliminates the symbolic ambiguity of the initial estimate of the standardized samples, obtaining the symbolic parameter estimate. The original scale recovery subnetwork then recovers the true physical scale of the translation parameters, yielding the sample camera pose. The symbolic parameter estimate includes sample rotation parameter estimation and sample translation parameter estimation, while the sample camera pose includes the sample rotation matrix and sample translation vector. Based on the binary classification labels and symbolic parameter estimates, a symbolic correction loss is constructed using binary cross-entropy. Based on the true camera pose labels and sample camera poses, a final pose loss is constructed using mean squared error. The initial estimated loss, symbolic correction loss, and final pose loss are weighted and summed to obtain the total loss function. Based on the total loss function, the backpropagation algorithm is used to optimize and update the network parameters of the original deep neural network, the original surrogate regression subnetwork, the original analytical solver, the original symbolic correction subnetwork, and the original scale recovery subnetwork, resulting in the deep neural network, surrogate regression subnetwork, analytical solver, symbolic correction subnetwork, and scale recovery subnetwork.

[0085] For example, in some specific implementations, before performing absolute pose regression on the query image to be located, the network model to be applied for absolute pose regression can first be obtained by training the network and building and adjusting the loss function. Specifically, this can be achieved through the following steps: To achieve stable network training, this invention designs a composite loss function to provide multi-stage supervision of the network's intermediate states (initial analytical solution, symbol prediction) and final result (final pose). The specific implementation steps are as follows: Step 1: Data Preparation: Obtain the real camera pose label corresponding to the query image, denoted as... Simultaneously, the actual rotation matrix and translation vector are converted into their corresponding vector forms (denoted as ). and ,in Numerically, that is This symbol is introduced for the convenience of subsequent descriptions and for the uniformity of formula symbols, serving as a benchmark for intermediate supervision.

[0086] Step 2: Construct the initial estimated loss ( This loss is used to supervise the standardized initial estimate vector calculated in step three (analytic solution). and Since the ambiguity of the symbol has not yet been eliminated (e.g., and Geometrically, they all point to the same straight line. A direction alignment loss is used as a constraint, ignoring the difference between positive and negative directions. Using cosine similarity The square of the square is used to measure the degree of collinearity between the predicted vector and the true vector. The formula is as follows:

[0087] Initial estimated loss This forces the calculated initial pose vector to align with the true pose vector on a straight line.

[0088] Step 3: Construct the symbolic correction loss ( ): This loss is used to supervise the symbolic correction subnetwork in step four (pose correction).

[0089] First, based on the initial estimated vector ( and ) and truth vector ( and The dot product symbol generates binary labels. A value of 1 indicates that the initial estimated vector is in the same direction as the true vector, meaning no sign correction is needed; otherwise, it is 0. Using binary cross-entropy loss to supervise the prediction probability values ​​of rotation and translation symbol subnetworks .

[0090] Step 4: Construct the final pose loss ( This loss is used to supervise the pose of the final output. .

[0091] The Euclidean distance between the predicted pose and the true pose label can be directly calculated using the mean square error:

[0092] Specifically, for translation branches, this loss implicitly supervises the scale-restoring subnetwork. The learning process forces the network to revert to the correct physical scale.

[0093] Step 5: Total Loss Optimization The total loss function is constructed by weighted summing of the three losses mentioned above. :

[0094] in , The hyperparameters for balancing the weights are defined. The total loss is minimized using the backpropagation algorithm. This allows for end-to-end updates of parameters for the backbone network and all subnetworks.

[0095] To explain in detail the principle of the technical solution of the present invention, the overall process of the present invention will be described below with reference to some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and should not be regarded as a limitation of the present invention.

[0096] In response to the shortcomings of existing technologies, such as Figure 3 As shown, this invention provides an absolute pose regression method based on a geometry-aware agent. The overall process of this method is as follows: First, the system acquires the input image to be localized. Through a feature extraction step, a deep neural network is used to extract global feature vectors containing scene semantics. Then, the core geometric perception proxy regression step maps the feature vectors into specific Gram matrices that encode the 3D scene structure and 2D projective geometric constraints. Next, an analytical solution step is performed, using algebraic methods to parse the initial pose parameters from the generated Gram matrix. Finally, to address the symbol ambiguity and scale loss issues in the analytical solution results, a pose correction step is performed, thereby outputting the final six-DOF camera pose.

[0097] In some specific implementations, such as Figure 4 As shown, the detailed process of this embodiment of the invention can be implemented as follows: Step 1: Feature Extraction. The detailed steps are as follows: Step 11: Feature Encoding: Encode the query image to be located (denoted as...) The input image is fed into a pre-trained deep neural network (i.e., the backbone network). This backbone network can be a residual network (such as the ResNet series) or a self-attention-based network architecture (such as the Transformer series). The backbone network maps the input image into a high-dimensional feature map (denoted as ) through forward inference. ).

[0098] Step 12: Global Pooling: Apply a global average pooling layer to the above high-dimensional feature maps. The data is processed and compressed into a one-dimensional global feature vector (denoted as ). ,in (Indicates the number of feature channels). This vector aggregates global contextual information from the image, serving as the input basis for subsequent geometrically perceptual proxy regression steps.

[0099] Step 2: Geometric perceptual surrogate regression.

[0100] (1) Explanation of technical principles The core of this step lies in introducing a specific Gram matrix as a geometric perception proxy, based on the classical absolute pose estimation theory. In traditional geometric solutions, a set of homogeneous linear equations (of the form of...) can be constructed based on the projection correspondence between 3D scene points / lines and 2D image features. ),in The coefficient matrix and vector This system of equations contains the pose parameters to be determined. According to algebraic theory, the solution to this system of equations (i.e., the pose parameters) corresponds to the Gram form of the coefficient matrix (i.e.,...). The eigenvector corresponding to the smallest eigenvalue of the matrix is ​​given by the eigenvector. Based on this principle, the neural network designed in this invention does not directly regress the pose, but rather regresses this Gram matrix. Because this matrix mathematically encodes the constraints of scene structure and projective geometry, the network can indirectly grasp the geometric structure information of the scene by learning to regress this matrix.

[0101] (2) The specific detailed steps are as follows: Step 21: Process branch construction (corresponding to the branch structure in the diagram): Process the global feature vector output from Step 1. Simultaneously, the input is fed into two parallel multilayer perceptrons consisting of fully connected layers: namely, the rotated surrogate regression subnetwork (denoted as...). ) and translation agent return subnetwork (denoted as This allows for the construction of independent rotation parameter regression branches and translation parameter regression branches, which are used to extract features of different pose attributes.

[0102] Step 22: Regressing the Lower Triangular Vector of the Proxy: Since the geometrically perceptual proxy (Gram matrix) possesses the property of a real symmetric matrix, this invention utilizes this property to reduce the redundancy of the network output. Specifically, the two regression branches mentioned above are used to regress the lower triangular elements of the geometrically perceptual proxy (Gram matrix) corresponding to the rotation and translation parameters, respectively: Regression of the rotated surrogate lower triangular vector: setting the rotation parameters corresponding to The rotation matrix has 9 elements, therefore its corresponding geometric perception proxy (Gram matrix) is: This is a real symmetric matrix. Calculations show that the lower triangular element count of this matrix is ​​45, therefore a rotated surrogate regression subnetwork is configured. Output a one-dimensional vector with dimension 45 (denoted as ). ), which serves as the rotated lower triangular vector.

[0103] Lower triangular vector of the regression translation proxy: The translation parameter is set to correspond to a translation vector containing 3 elements; therefore, its corresponding geometric perception proxy (Gram matrix) is... The matrix is ​​a real symmetric matrix. Calculations show that the lower triangular element count is 6; therefore, a translational surrogate regression subnetwork is configured. Output a one-dimensional vector with dimension 6 (denoted as ). ), which serves as a translational lower triangular vector.

[0104] Step 23: Construct a geometry-aware proxy: Utilizing the symmetry property of the matrix, reshape and fill the one-dimensional vector output above to construct a complete real symmetric matrix, which serves as the corresponding geometry-aware proxy (Gram matrix). Rotation proxy construction: The above 45-dimensional vector Mapped to The lower triangular portion of the matrix (containing diagonal elements) is used, and the values ​​are copied and filled into the upper triangular portion using symmetry to generate the complete matrix. Real symmetric matrix .

[0105] Translation proxy construction: The above 6-dimensional vector Mapped to The lower triangular portion of the matrix (containing diagonal elements) is used, and the values ​​are copied and filled into the upper triangular portion using symmetry to generate the complete matrix. Real symmetric matrix .

[0106] Step 3: Analytical Solution. The detailed steps are as follows: Step 31: Parameter Analysis and Solving: Using algebraic methods, the initial pose parameters are extracted from the specific Gram matrix constructed in Step 2 using a differentiable analytical solver (specifically, an eigenvalue decomposition algorithm based on power iteration). Analytical solution of rotation parameters: for the constructed geometry-aware proxy corresponding to the rotation parameters Calculate its eigenvalues ​​and corresponding eigenvectors. Based on the absolute pose estimation theory, select the eigenvector corresponding to the smallest eigenvalue as the original estimate of the rotation parameters (denoted as ). ).

[0107] Analytical solution of translation parameters: for the constructed geometry-aware proxy corresponding to the translation parameters Calculate its eigenvalues ​​and corresponding eigenvectors. Based on the absolute pose estimation theory, select the eigenvector corresponding to the smallest eigenvalue as the original estimate of the translation parameters (denoted as ). ).

[0108] Step 32: Parameter Standardization: To adapt to the input requirements of subsequent pose correction steps and satisfy geometric constraints, the initial solution vectors calculated above are standardized. Rotation parameter standardization: the calculated 9-dimensional rotation primitive estimate Performing Schmidt orthogonalization to conform to the orthogonal geometric specification of the rotation matrix yields a standardized initial estimate of the rotation (denoted as ). ).

[0109] Translation parameter standardization: This involves standardizing the calculated 3D translation parameters. Perform L2 normalization to make its magnitude 1, thus obtaining the pure translation direction vector (denoted as ). ).

[0110] Step 4: Pose Correction. Because the initial pose solution obtained in Step 3 has mathematical ambiguity (i.e., uncertainty in the direction of the eigenvectors), and the translation vector loses its true physical scale, this step uses a neural network to correct the initial pose parameters. The detailed steps are as follows: Step 41: Parameter Sign Correction: Construct sign correction subnetworks for rotation and translation parameters respectively (denoted as...). and These subnetworks were designed as binary classification models (the network is configured to first output a probability value representing whether the original orientation is retained). Then, it is mapped to an interval through a linear transformation. (Used as a sign correction factor), it aims to eliminate the directional ambiguity of eigenvectors in analytical solutions: Rotation parameter sign correction: This corrects the standardized initial rotation estimate. Compared with the global feature vector extracted in step one The data is then spliced ​​and input into the rotation sign correction subnetwork. The network predicts a sign correction factor (denoted as ) used to correct the vector direction. Then, the scalar is multiplied by the initial vector to obtain the rotation parameter estimate after direction correction (denoted as...). ).

[0111] Translation parameter sign correction: This corrects the standardized initial translation estimate. Compared with the global feature vector extracted in step one The data is then spliced ​​and input into the translation symbol correction subnetwork. The network predicts a sign correction factor (denoted as ) used to correct the vector direction. Then, the scalar is multiplied by the initial vector to obtain the translation parameter estimate after direction correction (denoted as ). ).

[0112] Step 42: Translation Parameter Scale Recovery: For the translation branch, due to the translation parameter estimation after direction correction... The modulus is still 1, lacking scale information, therefore scale recovery is required: Construct a scale recovery subnetwork (denoted as...) Its input includes global feature vectors. And the translation parameter estimation after sign correction .

[0113] This subnetwork outputs a positive real scalar (denoted as ) through regression operations. ), which is the true physical magnitude of the translation vector.

[0114] Apply the modulus length to This allows us to recover the translation vector with a true physical scale. ).

[0115] Step 43: Pose Output: Convert the corrected pose parameters into a standard pose representation and output them. Output rotation matrix: Estimates the corrected rotation parameters Remodeling into the standard Rotation matrix .

[0116] Output translation vector: the corrected translation vector. Output the translation vector.

[0117] Specifically, before performing absolute pose regression, the network model can be prepared through the following additional steps: network training and loss function construction.

[0118] To achieve stable network training, this invention designs a composite loss function to provide multi-stage supervision of the network's intermediate states (initial analytical solution, symbol prediction) and final result (final pose). The specific implementation steps are as follows: Step 51: Data Preparation: Obtain the real camera pose label corresponding to the query image, denoted as... Simultaneously, the actual rotation matrix and translation vector are converted into their corresponding vector forms (denoted as ). and ,in Numerically, that is This symbol is introduced for the convenience of subsequent descriptions and for the uniformity of formula symbols, serving as a benchmark for intermediate supervision.

[0119] Step 52: Construct the initial estimated loss ( This loss is used to supervise the standardized initial estimate vector calculated in step three (analytic solution). and Since the ambiguity of the symbol has not yet been eliminated (e.g., and Geometrically, they all point to the same straight line. A direction alignment loss is used as a constraint, ignoring the difference between positive and negative directions. Using cosine similarity The square of the square is used to measure the degree of collinearity between the predicted vector and the true vector. The formula is as follows:

[0120] Initial estimated loss This forces the calculated initial pose vector to align with the true pose vector on a straight line.

[0121] Step 53: Construct the symbolic correction loss ( ): This loss is used to supervise the symbolic correction subnetwork in step four (pose correction).

[0122] First, based on the initial estimated vector ( and ) and truth vector ( and The dot product symbol generates binary labels. A value of 1 indicates that the initial estimated vector is in the same direction as the true vector, meaning no sign correction is needed; otherwise, it is 0. Using binary cross-entropy loss to supervise the prediction probability values ​​of rotation and translation symbol subnetworks .

[0123] Step 54: Construct the final pose loss ( This loss is used to supervise the pose of the final output. .

[0124] The Euclidean distance between the predicted pose and the true pose label can be directly calculated using the mean square error:

[0125] Specifically, for translation branches, this loss implicitly supervises the scale-recovery subnetwork. The learning process forces the network to revert to the correct physical scale.

[0126] Step 55: Total Loss Optimization The total loss function is constructed by weighted summing of the three losses mentioned above. :

[0127] in , The hyperparameters for balancing the weights are defined. The total loss is minimized using the backpropagation algorithm. This allows for end-to-end updates of parameters for the backbone network and all subnetworks.

[0128] In summary, the technical solution of this invention implements a "geometric perception proxy" mechanism. Its core lies in breaking the "black box" mode of traditional absolute pose regression methods. Instead of directly regressing pose values, it allows a neural network to predict a specific Gram matrix that encodes the scene's geometric structure and projection geometric constraints. High-precision camera pose is then obtained through analytical solving and post-processing correction. This invention innovatively achieves the following technical means: 1. Core Technology Architecture: An absolute pose regression method based on geometric perception agents. Its core features are: 1) constructing parallel rotation and translation branches in the neural network, and regressing the corresponding specific Gram matrices as intermediate representations; 2) using an analytical solver (such as a differentiable direct linear transformation algorithm) to solve for the initial translation and rotation parameters from the above specific Gram matrices.

[0129] 2. Key Correction Mechanism: The post-processing correction network is used in conjunction with the analytical solution. Its core features are: 1) To address the sign ambiguity problem introduced by the analytical solution, a sign correction network is designed to determine the sign direction of the pose parameters; 2) To address the scale loss problem of the translation vector introduced by the analytical solution, a scale recovery network is designed to regress the magnitude of the translation vector.

[0130] Compared with the prior art, the embodiments of the present invention have at least the following beneficial effects: High positioning accuracy: This invention utilizes a specific Gram matrix to replace traditional direct pose numerical regression. This matrix encodes projective geometric information between 3D scene geometry and 2D image features, guiding the neural network to learn the scene's geometric structure and projective geometric constraints, thus effectively solving the accuracy limitation problem caused by the lack of geometric perception in existing technologies.

[0131] High efficiency: While improving accuracy, this invention fully retains the end-to-end inference architecture, avoiding the introduction of cumbersome feature extraction and matching processes or complex post-processing iterative optimization steps. The geometry perception proxy module of this invention consists of only a small number of network parameters and efficient analytical solution steps, perfectly preserving the core advantage of the real-time performance of the absolute pose regression method with almost no increase in inference latency (only a millisecond-level increase).

[0132] like Figure 5 As shown, this embodiment of the invention also provides an absolute pose regression device 900, which can implement the above-described method. This device may include: The first module 901 is used to obtain the query image to be located; The second module 902 is used to extract global feature vectors based on the query image using a deep neural network. The third module 903 is used to input the global feature vector into the proxy regression subnetwork and regress to obtain the geometry-aware proxy; wherein, the geometry-aware proxy adopts the Gram matrix, which is used to encode the projective geometric constraint information between the geometry of the 3D scene and the features of the 2D image; the geometry-aware proxy includes rotation geometry-aware proxy and translation geometry-aware proxy. The fourth module 904 is used to extract the original estimates of pose parameters through an analytical solver based on a geometrically perceptual agent, and then obtain standardized initial estimates through standardization processing; wherein, the standardized initial estimates include standardized rotation initial estimates and standardized translation initial estimates; The fifth module 905 is used to obtain the camera pose based on the standardized initial estimate, by using the symbolic correction subnetwork to eliminate the symbolic ambiguity of the standardized initial estimate, and by using the scale recovery subnetwork to recover the true physical scale of the translation parameters; wherein, the camera pose includes the rotation matrix and the translation vector.

[0133] In some embodiments, the apparatus further includes a sixth module for performing the following operations: Obtain the query image sample and its corresponding real camera pose label; wherein, the real camera pose label includes the vector form of the real rotation matrix and the real translation vector; Based on the query image samples, the global feature vector of the samples is extracted through the original deep neural network; Input the global feature vector of the sample into the original agent regression subnetwork to obtain the geometrically perceptual agent of the sample; Based on the sample geometry-aware agent, the original estimates of the sample pose parameters are extracted by the original analytical solver, and then the sample standardized initial estimates are obtained through standardization processing; wherein, the sample standardized initial estimates include sample standardized rotation initial estimates and sample standardized translation initial estimates; Based on the real camera pose labels and sample standardization initial estimates, the initial estimated loss is obtained by constructing the square of the cosine similarity. Binary labels are generated based on the dot product of the initial standardized estimate of the samples and the actual camera pose labels; Based on the initial sample standardization estimate, the symbolic ambiguity of the initial sample standardization estimate is eliminated by the original symbolic correction subnetwork to obtain the symbolic parameter estimate, and the true physical scale of the translation parameter is recovered by the original scale recovery subnetwork to obtain the sample camera pose; wherein, the symbolic parameter estimate includes the sample rotation parameter estimate and the sample translation parameter estimate, and the sample camera pose includes the sample rotation matrix and the sample translation vector; Based on binary classification labels and symbol parameter estimation, a symbol correction loss is constructed using binary cross-entropy. Based on real camera pose labels and sample camera poses, the final pose loss is constructed using mean squared error. The total loss function is obtained by weighted summing of the initial estimated loss, sign correction loss, and final pose loss. Based on the total loss function, the backpropagation algorithm is used to optimize and update the network parameters of the original deep neural network, the original surrogate regression subnetwork, the original analytical solver, the original sign correction subnetwork, and the original scale recovery subnetwork, so as to obtain the deep neural network, the surrogate regression subnetwork, the analytical solver, the sign correction network, and the scale recovery subnetwork.

[0134] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0135] This invention also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0136] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0137] like Figure 6 As shown, Figure 6 The hardware structure of an electronic device 1000 according to another embodiment is illustrated. The electronic device 1000 includes: The processor 1001 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (aSIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention. The memory 1002 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RaM). The memory 1002 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called and executed by the processor 1001. Input / output interface 1003 is used to implement information input and output; The communication interface 1004 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1005 transmits information between various components of the device (e.g., processor 1001, memory 1002, input / output interface 1003, and communication interface 1004); The processor 1001, memory 1002, input / output interface 1003 and communication interface 1004 are connected to each other within the device via bus 1005.

[0138] The electronic device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0139] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0140] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0141] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0142] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0143] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0144] This invention provides an absolute pose regression method, apparatus, electronic device, storage medium, and program product. It acquires a query image to be located; based on the query image, it extracts a global feature vector using a deep neural network; the global feature vector is input into a proxy regression subnetwork to obtain a geometric perception proxy; wherein the geometric perception proxy uses a Gram matrix, which encodes the projective geometric constraint information between the geometry of the 3D scene and the features of the 2D image; the geometric perception proxy includes a rotation geometric perception proxy and a translation geometric perception proxy; based on the geometric perception proxy, an analytical solver extracts the original estimate of the pose parameters, and then a standardized initial estimate is obtained through standardization processing; wherein the standardized initial estimate includes a standardized rotation initial estimate and a standardized translation initial estimate; based on the standardized initial estimate, a sign correction subnetwork is used to eliminate the sign ambiguity of the standardized initial estimate, and a scale recovery subnetwork is used to recover the true physical scale of the translation parameters to obtain the camera pose; wherein the camera pose includes a rotation matrix and a translation vector. This invention abandons the traditional approach of directly regressing pose values ​​and innovatively introduces a geometric perception proxy—a Gram matrix of a specific dimension. This matrix encodes the projective geometric constraint information between the geometry of the 3D scene and the features of the 2D image. Specifically, by having the neural network regress this proxy instead of directly outputting the pose, and combining analytical solving with post-processing correction, the network can implicitly learn the geometric structure of the scene, thereby significantly improving the localization accuracy and effectively enhancing the accuracy of the absolute pose regression method. Furthermore, the overall scheme of this invention still retains the end-to-end inference process, which can meet the requirements of high real-time performance.

[0145] The embodiments described in this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0146] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present invention, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0147] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0148] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0149] The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and spirit of the present invention should be within the scope of the claims of the present invention.

Claims

1. An absolute pose regression method, characterized in that, The method includes the following steps: Obtain the query image to be located; Based on the query image, a global feature vector is extracted using a deep neural network; The global feature vector is input into the proxy regression subnetwork to obtain a geometry-aware proxy; wherein, the geometry-aware proxy adopts a Gram matrix, which is used to encode the projective geometric constraint information between the geometry of the 3D scene and the features of the 2D image, and the geometry-aware proxy includes a rotation geometry-aware proxy and a translation geometry-aware proxy. Based on the geometric perception agent, the original pose parameter estimates are extracted by the analytical solver, and then standardized initial estimates are obtained through standardization processing; wherein, the standardized initial estimates include standardized rotation initial estimates and standardized translation initial estimates; Based on the standardized initial estimate, the symbolic ambiguity of the standardized initial estimate is eliminated by using a symbolic correction subnetwork, and the true physical scale of the translation parameters is recovered by using a scale recovery subnetwork to obtain the camera pose; wherein, the camera pose includes a rotation matrix and a translation vector.

2. The method according to claim 1, characterized in that, The step of extracting feature vectors based on the query image using a deep neural network includes the following steps: The query image is input into a deep neural network, and forward inference is used to map the query image into a high-dimensional feature map; The deep neural network includes a residual network or a self-attention-based network architecture; The high-dimensional feature map is compressed into a one-dimensional global feature vector through a global average pooling layer.

3. The method according to claim 1, characterized in that, The surrogate regression subnetwork employs a multilayer perceptron composed of fully connected layers. The surrogate regression subnetwork includes a rotation surrogate regression subnetwork and a translation surrogate regression subnetwork. The step of inputting the global feature vector into the surrogate regression subnetwork to regress and obtain a geometric perceptual surrogate includes the following steps: The global feature vector is input into the rotated surrogate regression subnetwork so that the rotated surrogate regression subnetwork outputs a rotated lower triangular vector; wherein the rotated lower triangular vector includes a first number of lower triangular elements; By utilizing the symmetry property of the matrix, the rotated lower triangular vector is reshaped and filled to obtain a real symmetric Gram matrix in the first dimension as the rotational geometry perception proxy. The global feature vector is input into the translation surrogate regression subnetwork so that the translation surrogate regression subnetwork outputs a translation lower triangular vector; wherein the translation lower triangular vector includes a second number of lower triangular elements; By utilizing the symmetry properties of the matrix, the translated lower triangular vector is reshaped and filled to obtain a second-dimensional real symmetric Gram matrix as the translation geometry perception proxy.

4. The method according to claim 1, characterized in that, The process of extracting the original pose parameter estimates using an analytical solver based on the geometrically aware agent, and then obtaining standardized initial estimates through standardization, includes the following steps: Based on the aforementioned rotational geometry-aware agent, the eigenvector corresponding to the minimum eigenvalue is obtained by processing the eigenvalue decomposition algorithm based on inverse power iteration as the original estimate of the rotation parameters. The original estimates of the rotation parameters are subjected to Schmitt orthogonalization to obtain standardized initial estimates of the rotation. Based on the aforementioned translation geometry-aware agent, the eigenvector corresponding to the minimum eigenvalue is obtained by processing the eigenvalue decomposition algorithm based on inverse power iteration as the original estimate of the translation parameters. The original translation parameter estimate is subjected to L2 normalization to obtain the standardized initial translation estimate as a unit direction vector.

5. The method according to claim 1, characterized in that, The sign correction subnetwork includes a rotation sign correction subnetwork and a translation sign correction subnetwork. The step of eliminating the sign ambiguity of the standardized initial estimate using the sign correction subnetwork, based on the standardized initial estimate, includes the following steps: The standardized rotation initial estimate is concatenated with the global feature vector to obtain the first concatenated vector; The first concatenated vector is input into the rotation sign correction subnetwork so that the rotation sign correction subnetwork outputs a rotation sign correction factor; Multiplying the rotation sign correction factor by the standardized initial rotation estimate yields the rotation parameter estimate; The rotation parameter estimate is reshaped into a matrix of a preset dimension to obtain the rotation matrix; The standardized translation initial estimate is concatenated with the global feature vector to obtain the second concatenated vector; The second concatenated vector is input into the translation sign correction sub-network so that the translation sign correction sub-network outputs the translation sign correction factor; Multiply the translation sign correction factor by the standardized initial translation estimate to obtain the translation parameter estimate.

6. The method according to claim 5, characterized in that, The method of recovering the true physical scale of translation parameters using a scale recovery subnetwork includes the following steps: The translation parameter estimate is concatenated with the global feature vector to obtain a third concatenated vector; The third concatenated vector is input into the scale recovery subnetwork so that the scale recovery subnetwork outputs a positive real scalar; wherein the positive real scalar represents the true physical magnitude of the translation vector; The translation vector is recovered by applying the real physical modulus to the translation parameter estimate.

7. The method according to claim 1, characterized in that, The method further includes the following steps: Obtain the query image sample and its corresponding real camera pose label; wherein, the real camera pose label includes the vector form of the real rotation matrix and the real translation vector; Based on the query image sample, the global feature vector of the sample is extracted through the original deep neural network; The global feature vector of the sample is input into the original agent regression subnetwork, and the sample geometric perception agent is obtained by regression. Based on the sample geometric perception agent, the original estimate of the sample pose parameters is extracted by the original analytical solver, and then the sample standardized initial estimate is obtained through standardization processing; wherein, the sample standardized initial estimate includes the sample standardized rotation initial estimate and the sample standardized translation initial estimate; Based on the real camera pose labels and the standardized initial estimate of the samples, the initial estimated loss is obtained by constructing the square of the cosine similarity. A binary classification label is generated based on the dot product of the standardized initial estimate of the sample and the real camera pose label; Based on the initial standardized estimate of the samples, the symbolic ambiguity of the initial standardized estimate is eliminated using the original symbolic correction subnetwork to obtain the symbolic parameter estimate, and the true physical scale of the translation parameters is recovered using the original scale recovery subnetwork to obtain the sample camera pose; wherein, the symbolic parameter estimate includes the sample rotation parameter estimate and the sample translation parameter estimate, and the sample camera pose includes the sample rotation matrix and the sample translation vector; Based on the binary classification labels and the symbol parameter estimation, a symbol correction loss is constructed using binary cross-entropy; Based on the real camera pose labels and the sample camera poses, the final pose loss is constructed using mean squared error. The initial estimated loss, the sign correction loss, and the final pose loss are weighted and summed to obtain the total loss function; Based on the total loss function, the backpropagation algorithm is used to optimize and update the network parameters of the original deep neural network, the original surrogate regression subnetwork, the original analytical solver, the original sign correction subnetwork, and the original scale recovery subnetwork, so as to obtain the deep neural network, the surrogate regression subnetwork, the analytical solver, the sign correction subnetwork, and the scale recovery subnetwork.

8. An absolute pose regression device, characterized in that, The device includes: The first module is used to obtain the query image to be located; The second module is used to extract a global feature vector based on the query image using a deep neural network; The third module is used to input the global feature vector into the proxy regression sub-network to obtain a geometry-aware proxy; wherein, the geometry-aware proxy adopts a Gram matrix, the Gram matrix is ​​used to encode the projective geometric constraint information between the geometry of the three-dimensional scene and the features of the two-dimensional image, and the geometry-aware proxy includes a rotation geometry-aware proxy and a translation geometry-aware proxy. The fourth module is used to extract the original estimates of pose parameters through an analytical solver based on the geometric perception agent, and then obtain standardized initial estimates through standardization processing; wherein, the standardized initial estimates include standardized rotation initial estimates and standardized translation initial estimates; The fifth module is used to eliminate the symbolic ambiguity of the standardized initial estimate using a symbolic correction subnetwork and to recover the true physical scale of the translation parameters using a scale recovery subnetwork, thereby obtaining the camera pose; wherein the camera pose includes a rotation matrix and a translation vector.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images