A camera relocalization system based on a lightweight Transformer model

Abstract

This invention belongs to the fields of image processing and camera relocalization, and more particularly to a camera relocalization system based on a lightweight Transformer model. The system comprises an image acquisition module, a feature extraction module, a sequence group attention (SGA) module, an EffLoc downsampling module, an average pooling module, and a camera pose regression module. This invention proposes EffLoc, a novel lightweight end-to-end visual transformer architecture, for 6-DOF camera relocalization using a single image, which can be generalized to large-scale real-world environments.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and camera repositioning, and in particular relates to a camera repositioning system based on a lightweight Transformer model. Background Art

[0002] Camera relocalization, also known as camera pose regression, focuses on retrieving the 3D position and orientation (6-DoF) of a camera based on an input image. It plays a key role in intelligent systems, ranging from augmented reality (AR) [1] / mixed reality (MR) to delivery drones, robotics, and autonomous driving [2].

[0003] Camera localization methods have historically relied primarily on image structure and features, establishing dense correspondences between 2D pixels and 3D points within the scene by matching visual observations to a map [3]. Subsequently, the camera pose is estimated using a perspective N-point (PnP) solver [4] or the Kabsch algorithm with RANSAC [5]. These traditional relocalization methods essentially rely on a matching process that involves comparing a query image to a database of reference images [6]. The computational and storage requirements of these techniques are directly related to the number of sample points in the database. Furthermore, the effectiveness of these methods is closely related to the quality of the matching process, which in turn depends on the similarity score. Deep learning-based camera relocalization methods can achieve end-to-end pose estimation directly from images using deep neural networks. For example, PoseNet uses a convolutional neural network (CNN)-based encoder to extract features from a single image as vector embeddings, which are then converted into 6-degrees-of-freedom (6-DoF) poses [7]. Other end-to-end learning methods, such as

[15] , utilize an implicit map database to store scene information, eliminating the need for complex manual feature engineering [8].

[0004] Traditionally, deep learning-based pose estimation has heavily relied on convolutional neural networks (CNNs) for feature extraction, which operate within local pixel neighborhoods. However, visual transformers (ViTs) are a recent breakthrough that divide images into multiple patches and utilize position embeddings to capture global dependencies. Unlike CNNs, ViTs establish meaningful correlations between spatially distant image regions, which is crucial for real-world relocalization tasks with large datasets. In addition, CNN-based visual localization models suffer from accuracy limitations and lack robustness due to challenges such as illumination variations, occlusions, and dynamic objects. In contrast, when trained using image-pose pairs, ViTs better match the map library, eliminating scale drift and accumulated errors. The emerging lightweight visual transformers

[10] offer computational efficiency and improved robustness in complex, resource-limited real-world scenarios.

[0005] Camera relocalization is a key task in computer vision, used in fields such as augmented reality, delivery drones, robotics, and autonomous driving. It aims to estimate the 3D position and orientation (6-DoF) of a camera from an input image. Traditional camera pose estimation methods, such as simultaneous localization and mapping (SLAM), rely on structure-based methods, while recent developments leverage deep learning to generate camera poses directly from images in an end-to-end manner.

[0006] References:

[0007] [1] R.Castle, G.Klein, and DWMurray, "Video-rate localization in multiple maps for wearable augmented reality." 2008 12th IEEE InternationalSymposium on Wearable Computers, Pittsburgh, PA, USA, 2008, pp.15-22.

[0008] [2] Royer, E., Lhuillier, M., Dhome, M. et al. "Monocular Vision for MobileRobot Localization and Autonomous Navigation." Int J Comput Vision 74, 2007, pp.237–260.

[0009] [3] C. Campos, R. Elvira, JJ GRodríguez, JMM Montiel and JDTardós, "ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM." In IEEE Transactions on Robotics, vol.37, no.6, Dec.2021, pp.1874-1890.

[0010] [4]R.Elvira,J.D.Tardósand J.M.M.Montiel,“ORBSLAM-Atlas:a robust andaccurate multi-map system.”2019IEEE / RSJ International Conference onIntelligent Robots and Systems(IROS),Macau,China,2019,pp.6253-6259[5]E.Brachmann et al.,“DSAC—Differentiable RANSAC for Camera Localization.”2017IEEE Conference on Computer Vision and Pattern Recognition(CVPR),USA,2017,pp.2492-2500.

[0011] [6]Wang,S.,Kang,Q.,She,R.,Tay,W.P.,Hartmannsgruber,A.,\&NavarroNavarro,D.“RobustLoc:Robust Camera Pose Regression in Challenging DrivingEnvironments.”Proceedings of the AAAI Conference on Artificial Intelligence,2023,pp.6209-621.

[0012] [7]A.Kendall,M.Grimes and R.Cipolla,“PoseNet:A Convolutional Networkfor Real-Time 6-DOF Camera Relocalization.”2015IEEE International Conferenceon Computer Vision(ICCV),Santiago,Chile,2015,pp.2938-2946.

[0013] [8] S. Wang, R. Clark, H. Wen and N. Trigoni, "DeepVO: Towards end-to-endvisual odometry with deep Recurrent Convolutional Neural Networks." 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, 2017, pp. 2043-2050.

[0014] [9] Y.Shavit, R.Ferens and Y.Keller, "Learning Multi-Scene Absolute PoseRegression with Transformers." 2021IEEE / CVF International Conference on Computer Vision (ICCV), Canada, 2021, pp.2713-2722.

[10] J.Liet al., "Next-ViT: Nextgeneration vision transformer for efficient deployment in realistic industrial scenarios." In Proceedings of the European conference on computer vision (ECCV), 2022. Summary of the Invention

[0015] To address the challenges faced by existing CNN-based visual relocalization methods, we introduce EffLoc, a lightweight visual transformer framework for efficient 6-degrees-of-freedom (6-DoF) camera relocalization. Our layered architecture integrates memory-bound self-attention and inter-channel communication to improve memory efficiency. We propose a Sequential Group Attention (SGA) module to improve computational efficiency by distributing input features across attention heads and redistributing parameters. Our model surpasses previous techniques in image-based 6-DoF camera relocalization, achieving a good balance between accuracy and efficiency, and performs well even on large-scale outdoor datasets. Its advantages are characterized by its simplicity, end-to-end trainability, and elimination of hand-crafted loss functions.

[0016] The present invention provides a camera relocalization system based on a lightweight Transformer model, which is constructed by the following method steps:

[0017] S1, Image acquisition: Get the input image and divide the image into multiple overlapping image patches, with the size of each patch fixed to 16×16 pixels;

[0018] S2, Feature Extraction: Perform feature extraction on multiple overlapping input image patches to accurately capture shared information and subtle details in adjacent image patches;

[0019] Among them, the image is divided into overlapping patches and each patch is embedded using a traditional ConvNet with layered convolutions;

[0020] S3, Sequence Group Attention Processing: Focuses on different parts of the image and understands the key information of the image scene. Specifically, the following steps are performed: In the first stage, each channel of the feature map input to the SGA basic module is passed through a deep convolutional layer to extract and process local textures and features; in the second stage, a feedforward network is used to perform nonlinear changes on the features and achieve higher-level feature abstraction; in the third stage, the Sequence Group Attention (SGA) mechanism is introduced to model the input sequence, associating it with multiple sequence multi-head attention heads to learn the relationships and dependencies within the sequence; in the fourth stage, the first and second stages are repeated sequentially according to the size of the input image to learn feature information at more abstract levels, increasing the depth and generalization ability of the model;

[0021] S4, EffLoc downsampling processing: reduces the resolution of input features, reduces computational complexity, increases the receptive field, and improves the model's understanding of the context in the image;

[0022] S4, average pooling processing: Reduce the dimension of the input data, calculate the average value for each pooling window, and use the average value to replace all values ​​in the window, reducing model redundant information and improving model robustness;

[0023] S5. Camera pose regression: Regress the 6-DOF camera pose from the features passed through the perceptron MLPs.

[0024] Preferably, in step S2, a lightweight Vision Transformer is selected as the backbone network of EffLoc and initialized using a model pre-trained on the ImageNet-1K dataset, which is specifically used for image classification experiments. Given an image I∈R C×H×W , features X∈R can be extracted by overlapping patch embedding (Ope) C :

[0025] X=Ope(I) (1)

[0026] In addition, preferably, in step S3, a new attention module called sequence group attention SGA is introduced for the lightweight Vision Transformer, in which each attention head receives a different subset of the complete features, effectively decomposing the attention calculation into multiple heads of the attention module. The attention mechanism is expressed as:

[0027]

[0028] Among them, the jth attention head is on X ij Perform self-attention calculation; projection layer and The input features are divided into different subspaces; the self-attention of the input feature X is expressed as:

[0029]

[0030] Where n is the total number of attention heads, i.e. Xi = [X i1 ,X i2 ,…,X in ] where 1≤j≤n; linear layer The output features after projection connection are returned to the same dimension as the input to ensure dimensional consistency; the Softmax function is applied to normalize the attention scores and convert them into probability distributions in the range [0, 1]; each attention head is then combined with the split features learned by the subsequent attention heads, and it is iterated in a way that encourages Q, K, and V to learn more comprehensive feature information, thereby enhancing the inductive bias of local and global feature representations. The generated sequence group attention head SGH is represented as:

[0031]

[0032] Among them, SGH(X ij ) is the (j-1)th head output and the jth position input X ij The sum of .

[0033] Also preferably, in step S5, the camera position is estimated by the following method: from the ordered cascade group head guided feature SGH (X ij ) Regress the 6-DOF camera pose via multi-layer perceptrons (MLPs):

[0034] [p,q]=MLPs(SGH(X ij )) (5)

[0035] Where p represents the 3D camera position and q represents the 4D unit quaternion of the orientation. The parameters in the ordered cascade attention module are optimized using the L1 loss function

[41] :

[0036]

[0037] Where α and β are coefficients that balance the position loss and rotation loss. The logarithm of the unit quaternion, log(q), is a three-dimensional representation.

[0038] Further preferably, the unit quaternion q=(u, v) is represented by the scalar u as the real part of the quaternion and by the three-dimensional vector v as the imaginary part, and the quaternion is restricted to a single hemisphere, and is defined as follows:

[0039]

[0040] Specifically, the system includes the following modules:

[0041] The image acquisition module is used to obtain an input image and divide the image into multiple overlapping image patches, with the size of each patch fixed at 16×16 pixels.

[0042] A feature extraction module is used to extract features from multiple overlapping input image patches, accurately capturing shared information and subtle details in adjacent image patches;

[0043] The Sequence Group Attention (SGA) module is used to focus on different parts of an image and understand key information about the scene. Specifically, in the first stage, each channel of the feature map input to the SGA module is passed through a deep convolutional layer to extract and process local textures and features. In the second stage, a feedforward network is used to perform nonlinear transformations and achieve higher-level feature abstraction on the features. In the third stage, the Sequence Group Attention (SGA) mechanism is introduced to model the input sequence, associating it with multiple sequence multi-head attention heads to learn relationships and dependencies within the sequence. In the fourth stage, the first and second stages are repeated sequentially based on the size of the input image to learn feature information at more abstract levels, increasing the depth and generalization capabilities of the model.

[0044] The EffLoc downsampling module is used to reduce the resolution of input features, reduce computational complexity, increase the receptive field, and improve the model's understanding of context in the image;

[0045] The average pooling module is used to reduce the dimensionality of the input data. It calculates the average value for each pooling window and replaces all values ​​in the window with the average value, reducing model redundant information and improving model robustness.

[0046] The camera pose regression module is used to regress the 6-DOF camera pose from the features passed through the perceptron MLPs, that is, the 6D pose of the pointing object, which is achieved through 3D rotation + 3D translation.

[0047] The present invention also provides a device containing the system for camera relocalization based on the lightweight Transformer model, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program, and the computer program code implements the system for camera relocalization based on the lightweight Transformer model.

[0048] Furthermore, the system also includes a supporting camera for taking images. Specifically, the cameras are two BumblebeeXB3 stereo cameras.

[0049] The advantages of this invention are: We propose EffLoc, a novel lightweight end-to-end visual transformer architecture for 6-degree-of-freedom (6-DoF) camera relocalization using a single image, which can be generalized to large-scale real-world environments. We propose a simple yet effective sequential cascade module that significantly improves the latency / accuracy trade-off for image-based 6-DoF camera relocalization. By introducing different channel feature splits between attention heads, this module effectively reduces redundant attention computations, thereby achieving significant memory efficiency improvements. Furthermore, we introduce a novel parameterization method that involves reconfiguring the original VitQKV (query, key, value) ratio for camera pose estimation. This optimization significantly improves computational and memory efficiency, resulting in a significant reduction in FLOPs (86.8%) and memory usage (49.7%) compared to AtLoc. Notably, through this reconfiguration process, the influence of Q and K in the third sequential group attention module is significantly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is the full flow chart of EffLoc work. DETAILED DESCRIPTION

[0051] The present invention is further described below with reference to specific examples, which however do not limit the present invention.

[0052] Example 1

[0053] The construction method of the camera relocalization system based on the lightweight Transformer model is as follows:

[0054] S1. Image acquisition: Get the input image and divide the image into multiple overlapping image patches, with the size of each patch fixed to 16×16 pixels.

[0055] S2. Feature extraction: Feature extraction is performed on multiple overlapping input image patches to accurately capture the shared information and subtle details in adjacent image patches.

[0056] The method of the present invention involves dividing the image into overlapping patches and embedding each patch using a traditional ConvNet with layered convolutions. This overlapping patch embedding enhances fine-grained localization capabilities by capturing local details and spatial sensitivity obtained from neighboring patches. Past studies have demonstrated the effectiveness of classical convolutional networks. The efficacy of ResNet34, a 34-layer residual network, is emphasized as the basis for camera pose estimation. Residual networks like ResNet34 can train deeper neural networks, thereby solving the gradient vanishing and memory efficiency problems associated with cross-memory access, which lead to excessive computational costs. Therefore, the present invention selects a lightweight Vision Transformer (EfficientViT) as the backbone network of EffLoc. The weights of EfficientViT are initialized using a model pre-trained on the ImageNet-1K dataset, which is specifically designed for image classification experiments. Given an image I∈R C×H×W , features X∈R can be extracted by overlapping patch embedding (Ope) C :

[0057] X=Ope(I) (1)

[0058] S3, Sequence Group Attention Processing: This focuses on different parts of the image and understands key information about the scene. Specifically, the first stage passes each channel of the feature map input to the SGA basic module through a deep convolutional layer to extract and process local textures and features. The second stage utilizes a feedforward network to perform nonlinear transformations and higher-level feature abstraction on the features. The third stage introduces the Sequence Group Attention (SGA) mechanism to model the input sequence, associating it with multiple sequence multi-head attention heads to learn relationships and dependencies within the sequence. The fourth stage repeats the first and second stages sequentially, depending on the size of the input image, to learn feature information at more abstract levels, increasing the depth and generalization capabilities of the model.

[0059] Specifically, this paper introduces a new attention module for EfficientViT, named Sequence Group Attention (SGA). Each attention head receives a different subset of the complete features, effectively decomposing the attention computation into multiple heads of the attention module. Mathematically, this attention mechanism can be expressed as:

[0060]

[0061] The jth attention head is on X ij Perform self-attention calculation. Projection layer and The input features are divided into different subspaces. Finally, the self-attention of the input feature X can be expressed as:

[0062]

[0063] Where n is the total number of attention heads, i.e. Xi = [X i1 ,X i2 ,…,X in ] where 1≤j≤n. Linear layer The output features after projection concatenation are returned to the same dimension as the input to ensure dimensional consistency. The SoftMax function is applied to normalize the attention scores and convert them into probability distributions in the range [0, 1].

[0064] At this point, each attention head reduces computational cost by learning the split features. Each attention head is then combined with the split features learned by the subsequent attention heads, and iteratively encourages Q, K, and V to learn more comprehensive feature information, thereby enhancing the inductive bias of local and global feature representations. The resulting Sequence Group Attention Head (SGH) is represented as:

[0065]

[0066] Among them, SGH(X ij ) is the (j-1)th head output and the jth position input X ij The sum of .

[0067] S4, EffLoc downsampling: This reduces the resolution of the input features, reducing computational complexity, increasing the receptive field, and improving the model's understanding of image context. The output is the average of all values ​​in a 3x3 sliding window (reducing computational complexity from nine to one).

[0068] S4, average pooling processing: Reduce the dimension of the input data, calculate the average value for each pooling window, and use the average value to replace all values ​​in the window, reducing model redundant information and improving model robustness.

[0069] S5. Camera pose regression: Regress the 6-DOF camera pose based on the features passed through the perceptrons (MLPs).

[0070] The camera position estimation method is as follows: Based on the previous research of VidLoc pose estimation method, this method is derived from the ordered cascade group head guided feature SGH (X ij ) Regress the 6-DOF camera pose via multi-layer perceptrons (MLPs):

[0071] [p,q]=MLPs(SGH(X ij )) (5)

[0072] Here, p represents the 3D camera position and q represents the 4D unit quaternion of the direction. The parameters in the ordered cascade attention module are optimized using the L1 loss function:

[0073]

[0074] Here, α and β are coefficients that balance the position loss and rotation loss. The logarithm of the unit quaternion, log(q), is a three-dimensional representation that avoids over-parameterization. Therefore, we can directly use the L1 distance as the loss function without normalization. The L1 loss reduces the impact of outliers, making it better at handling data with atypical observations, and promotes sparsity of parameters and features, facilitating feature selection, and encouraging the assignment of zero weights to irrelevant or unimportant features. Specifically, the unit quaternion q = (u, v) is represented by the scalar u as the real part of the quaternion and by the three-dimensional vector v as the imaginary part, defined as follows:

[0075]

[0076] Quaternions are commonly used for camera pose regression because they provide a continuous and differentiable representation of orientation. By normalizing a 4D quaternion to unit length, any 3D rotation can be mapped to a valid unit quaternion. However, the non-uniqueness of quaternions can cause problems: -q and q can represent the same rotation because there are two hemispheres. To ensure uniqueness, this study restricts quaternions to a single hemisphere.

[0077] Example 2

[0078] Based on the method of Example 1, a camera relocalization system based on a lightweight Transformer model is obtained. The system completes 6-DOF camera pose estimation and implements camera relocalization by learning the current single image. The system includes the following modules corresponding to the steps of the method of Example 1:

[0079] The image acquisition module is used to obtain an input image and divide the image into multiple overlapping image patches, with the size of each patch fixed at 16×16 pixels.

[0080] The feature extraction module is used to extract features from multiple overlapping input image patches and accurately capture the shared information and subtle details in adjacent image patches.

[0081] The Sequence Group Attention (SGA) module focuses on different parts of an image and understands key information about the scene. Specifically, the first stage passes each channel of the feature map input to the SGA base module through a deep convolutional layer to extract and process local textures and features. The second stage utilizes a feedforward network to perform nonlinear transformations and achieve higher-level feature abstraction. The third stage introduces the Sequence Group Attention (SGA) mechanism to model the input sequence, linking it with multiple sequence multi-head attention heads to learn relationships and dependencies within the sequence. The fourth stage repeats the first and second stages sequentially, depending on the size of the input image, to learn feature information at a higher level of abstraction, increasing the model's depth and generalization capabilities.

[0082] The EffLoc downsampling module is used to reduce the resolution of input features, reduce computational complexity, increase the receptive field, and improve the model's understanding of the context in the image.

[0083] The average pooling module is used to reduce the dimension of the input data, calculate the average value for each pooling window, and replace all values ​​in the window with the average value, reducing model redundant information and improving model robustness.

[0084] The camera pose regression module is used to regress the 6-DOF camera pose from the features passed through the perceptrons (MLPs), that is, the 6D pose of the pointing object, which is achieved through 3D rotation + 3D translation.

[0085] The system is equipped with two Bumblebee XB3 stereo cameras with a resolution of 1280×960. The refresh rate is 16Hz, with images refreshed 16 times per second. The image sensor is a 1 / 3-inch Sony ICX445 CCD with global shutter technology. The lens is 3.8mm, providing a 66° horizontal field of view. The baseline is 12 / 24 cm, the distance between the two cameras. The cameras are located directly above the robot vehicle, facing the direction of travel, and at a height of 1.52 meters above the ground.

[0086] In practice, 6D pose is usually represented by a quaternion (describing rotation) and a translation parameter (describing position). The camera pose regression module completes 6-DOF pose regression by using perceptrons (MLPs) to decode feature parameters and convert camera hardware commands.

[0087] Example 3

[0088] The present invention uses the Oxford Robotics Institute to continuously travel 1,000 kilometers for one year, covering various weather environments, collecting dynamic elements such as moving and stationary vehicles, cyclists, and pedestrians on zebra crossings, totaling 3,438 cases, for model training and verification. By conducting comparative experiments with other methods (PoseNet+, AtLoc), the overall effectiveness of the method of the present invention is demonstrated. By comparing the 6-degree-of-freedom camera pose (displacement error, rotation error) after regression with the other two methods, the overall accuracy of the method of the present invention is demonstrated. The model comparison results are shown below:

[0089] Table 1 Method comparison results

[0090]

[0091] In Table 1, by comparing with PoseNet+ and AtLoc, EffLoc has achieved significant improvements on all four roads. EffLoc improves the average position accuracy of Road 1 from 25.39 meters to 7.58 meters, and the average position accuracy of Road 2 from 28.89 meters to 7.89 meters. EffLoc reduces the average rotation error of Road 1 from 17.49° to 3.72°, and the average rotation error of Road 2 from 19.65° to 4.19°. EffLoc achieved the largest performance improvement on the Road 3 and Road 4 datasets, improving by 78.3% and 65.8% respectively over PoseNet+. The performance of EffLoc on Road 3 and Road 4 routes is improved by 33.9% and 24.7% respectively over AtLoc. This fully demonstrates the effectiveness of the method of the present invention.

[0092] In addition, ablation experiments, i.e. internal comparative experiments, were conducted on different innovative modules in the model to demonstrate the effectiveness of each module and the impact of the hyperparameters in each module on the experimental results, and to find the optimal value of the hyperparameters within the trial range.

[0093] Table 2 Ablation experiment

[0094]

[0095] Where Di represents width, Li represents depth, and Hi represents the number of attention heads.

[0096] In the ablation experiments in Table 2, three models, EffLoc, EffLoc-XS, and EffLoc-XSS, were internally compared, each with varying width, depth, and attention heads. A uniform training size of 3000 epochs was maintained to investigate accuracy and computational efficiency. To ensure fairness, other modules remained the same. In the early stages (i), the model had limited channel widths (128, 240, and 320) and initially smaller channel depths (1, 2, and 3), reducing redundancy and memory usage. The layered structure of EffLoc-XXS facilitated faster convergence compared to EffLoc-XS, but slightly increased mean position error (from 70.72 meters to 36.89 meters) and rotation error (from 20.47° to 15.12°). EffLoc's optimized parameters achieved a balance between accuracy and efficiency in camera relocalization. Key modules had increased channels, preserving crucial feature information through high-dimensional learning. The smaller model of our invention demonstrated superior efficiency at a negligible loss in accuracy.

Claims

1. A method for constructing a camera relocalization system based on a lightweight Transformer model, characterized in that: The method comprises the following steps: S1, Image acquisition: Get the input image and divide the image into multiple overlapping image patches, with the size of each patch fixed to 16×16 pixels; S2, Feature Extraction: Perform feature extraction on multiple overlapping input image patches to accurately capture shared information and subtle details in adjacent image patches; Among them, the image is divided into overlapping patches and each patch is embedded using a traditional ConvNet with layered convolutions; S3, Sequence Group Attention Processing: Focuses on different parts of the image and understands the key information of the image scene. Specifically, the following steps are performed: In the first stage, each channel of the feature map input to the SGA basic module is passed through a deep convolutional layer to extract and process local textures and features; in the second stage, a feedforward network is used to perform nonlinear changes on the features and achieve higher-level feature abstraction; in the third stage, the Sequence Group Attention (SGA) mechanism is introduced to model the input sequence, associating it with multiple sequence multi-head attention heads to learn the relationships and dependencies within the sequence; in the fourth stage, the first and second stages are repeated sequentially according to the size of the input image to learn feature information at more abstract levels, increasing the depth and generalization ability of the model; S4, EffLoc downsampling processing: reduces the resolution of input features, reduces computational complexity, increases the receptive field, and improves the model's understanding of the context in the image; S5, average pooling processing: Reduce the dimension of the input data, calculate the average value for each pooling window, and use the average value to replace all values ​​in the window, reducing model redundant information and improving model robustness; S6, Camera Pose Regression: Regress the 6-DOF camera pose from the features passed through the perceptron MLPs.

2. The method for constructing a camera relocalization system based on a lightweight Transformer model according to claim 1, wherein: In step S2, a lightweight Vision Transformer is selected as the backbone network of EffLoc and initialized using a model pre-trained on the ImageNet-1K dataset, which is specifically used for image classification experiments. Given an image I∈R C×H×W , extract features X∈R by overlapping patch embedding (Ope) C : X=Ope(I)(1).

3. The method for constructing a camera relocalization system based on a lightweight Transformer model according to claim 1, wherein: In the S3 step, sequence group attention (SGA) is introduced for the lightweight Vision Transformer, where each attention head receives a different subset of the complete features, effectively decomposing the attention calculation into multiple heads of the attention module. The attention mechanism is expressed as: Among them, the jth attention head is on X ij Perform self-attention calculation; projection layer and The input features are divided into different subspaces; the self-attention of the input feature X is expressed as: Where n is the total number of attention heads, i.e. X i =[X i1 ,X i2 ,…,X in ], where 1≤j≤n; linear layer The output features after projection connection are returned to the same dimension as the input to ensure dimensional consistency; the Softmax function is applied to normalize the attention scores and convert them into probability distributions in the range [0, 1]; each attention head is then combined with the split features learned by the subsequent attention heads, and it is iterated in a way that encourages Q, K, and V to learn more comprehensive feature information, thereby enhancing the inductive bias of local and global feature representations. The generated sequence group attention head SGH is represented as: Among them, SGH(X ij ) is the (j-1)th head output and the jth position input X ij The sum of .

4. The method for constructing a camera relocalization system based on a lightweight Transformer model according to claim 1, wherein: In step S6, the camera position is estimated as follows: from the ordered cascade group head guided feature SGH (X ij ) Regress the 6-DOF camera pose via multi-layer perceptrons (MLPs): [p,q]=MLPs(SGH(X ij )) (5) Where p represents the 3D camera position, q represents the 4D unit quaternion of the direction, and the parameters in the ordered cascade attention module are optimized using the L1 loss function: Among them, α and β are coefficients that balance the position loss and rotation loss, and the logarithm of the unit quaternion log(q) is a three-dimensional representation.

5. The method for constructing a camera relocalization system based on a lightweight Transformer model according to claim 4, wherein: The unit quaternion q = (u, v) is represented by the scalar u as the real part of the quaternion and the three-dimensional vector v as the imaginary part, and is defined as follows: And restrict quaternions to a single hemisphere.

6. The method for constructing a camera relocalization system based on a lightweight Transformer model according to any one of claims 1 to 5, characterized in that: The camera relocation system includes the following modules: The image acquisition module is used to obtain the input image and divide the image into multiple overlapping image patches, with the size of each patch fixed at 16×16 pixels; A feature extraction module is used to extract features from multiple overlapping input image patches, accurately capturing shared information and subtle details in adjacent image patches; The Sequence Group Attention (SGA) module is used to focus on different parts of an image and understand key information about the scene. Specifically, in the first stage, each channel of the feature map input to the SGA module is passed through a deep convolutional layer to extract and process local textures and features. In the second stage, a feedforward network is used to perform nonlinear transformations and achieve higher-level feature abstraction on the features. In the third stage, the Sequence Group Attention (SGA) mechanism is introduced to model the input sequence, associating it with multiple sequence multi-head attention heads to learn relationships and dependencies within the sequence. In the fourth stage, the first and second stages are repeated sequentially based on the size of the input image to learn feature information at more abstract levels, increasing the depth and generalization capabilities of the model. The EffLoc downsampling module is used to reduce the resolution of input features, reduce computational complexity, increase the receptive field, and improve the model's understanding of context in the image; The average pooling module is used to reduce the dimensionality of the input data. It calculates the average value for each pooling window and replaces all values ​​in the window with the average value, reducing model redundant information and improving model robustness. The camera pose regression module is used to regress the 6-DOF camera pose from the features passed through the perceptron MLPs, that is, the 6D pose of the pointing object, which is achieved through 3D rotation + 3D translation.

7. A device comprising a method for constructing a system for camera relocalization based on a lightweight Transformer model according to any one of claims 1 to 6, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program, and the computer program code implements the method for constructing a system for camera relocalization based on a lightweight Transformer model according to any one of claims 1 to 6.

8. The device according to claim 7, characterized in that Also included is a matching camera for taking pictures.

9. The device according to claim 8, characterized in that The cameras are two stereo cameras.

Images