Visual relocation and pose estimation method based on space-time dual compression and related device

Abstract

The invention discloses a visual relocation and pose estimation method based on space-time double compression and a related device, and the method comprises the steps: carrying out the space compression and feature point number compression of a to-be-located image, and carrying out the time dimension self-adaptive compression, space compression and feature point number compression of an image in an image set matched with the to-be-located image; through the compression mechanism, the image processing amount is reduced, the processing speed is improved, both efficiency and precision can be considered, and the method is suitable for high-dynamic and large-scale scenes and resource-limited equipment.

Description

Technical Field

[0001] This invention relates to a visual relocalization and pose estimation method and related apparatus based on spatiotemporal dual compression, belonging to the field of navigation technology. Background Technology

[0002] Visual relocalization and pose estimation (i.e., position and orientation) are core technologies in computer vision and robot navigation, primarily applied in scenarios such as Simultaneous Localization and Mapping (SLAM), Augmented Reality (AR), and autonomous driving. Their fundamental principle relies on image sequences captured by a camera. By extracting key features from the images (such as SIFT or ORB feature points) and matching them, the current 3D position and orientation (pose) of the camera or robot can be determined within a pre-constructed scene map. Many estimation methods exist, such as CN120163877B, which discloses a visual relocalization method; CN114202701B, which discloses a UAV visual relocalization method based on object semantics; and CN120147419A, which discloses a visual relocalization method based on salient scene coordinate regression. While these methods have made progress in feature matching and pose estimation, they suffer from the problem of large image processing volumes when dealing with highly dynamic and large-scale scenes, making it difficult to balance relocalization efficiency and accuracy. Summary of the Invention

[0003] This invention provides a visual relocalization and pose estimation method and related apparatus based on spatiotemporal dual compression, which solves the problems disclosed in the background art.

[0004] According to one aspect of this application, a visual relocalization and pose estimation method based on spatiotemporal dual compression is provided, comprising:

[0005] Image A to be located is acquired from the image acquisition device. Image A is spatially compressed. Feature points of the spatially compressed image A are extracted and the number of feature points is compressed.

[0006] Based on the feature points of image A after compression, image B with the highest matching degree with image A is selected from the preprocessed image set; where the image set is the image set used to construct the scene map; the preprocessing of the image set includes time-dimensional adaptive compression of the images in the image set, spatial compression of the time-dimensional adaptively compressed images, extraction of feature points of the spatially compressed images, and number compression of the feature points.

[0007] Estimate the pose of the image acquisition device based on image A and image B.

[0008] Furthermore, time-dimensional adaptive compression of the images in the image set is performed using an interval sampling method. The interval sampling density is determined based on the acceleration at the time of image acquisition in the image set, as shown in the formula:

[0009] ;

[0010] In the formula, q j Let v be the sampling density at time j, ε be a constant greater than 0, and v j This represents the speed of the image acquisition device at time j. , a is the image acquisition interval. i Let be the acceleration at time i, which is the acceleration acquired by the image acquisition device or external IMU.

[0011] Furthermore, spatial compression involves performing multi-resolution processing on the temporally compressed image to generate an image pyramid; in the image pyramid, the original image is the 0th layer, and the higher the layer number, the lower the image resolution.

[0012] Based on the number of compressed feature points in image A, image B, which has the highest matching degree with image A, is selected from the preprocessed image set, including:

[0013] 1) Using feature point matching, images that initially match the nth layer of the second image pyramid are selected from all the nth layers of the first image pyramid; where the first image pyramid is the image pyramid corresponding to the image in the image set, and the second image pyramid is the image pyramid corresponding to image A;

[0014] 2) If the preliminary matching result of the nth layer meets the preset requirements, the image with the highest matching degree in the preliminary matching images shall be taken as image B;

[0015] If n is not equal to 0, and the initial matching result of the nth layer does not meet the preset requirements, n = n-1, go to 1).

[0016] If n equals 0 and the initial matching result of the nth layer does not meet the preset requirements, the image with the highest matching degree in the first image pyramid is taken as image B.

[0017] According to another aspect of this application, a visual relocalization and pose estimation device based on spatiotemporal dual compression is provided, comprising:

[0018] The acquisition and extraction module acquires the image A to be located from the image acquisition device, performs spatial compression on image A, extracts the feature points of the spatially compressed image A, and performs numerical compression on the feature points.

[0019] The filtering module selects the image B with the highest matching degree with image A from the preprocessed image set based on the number of compressed feature points of image A; where the image set is the image set used to construct the scene map; the preprocessing of the image set includes performing time-dimensional adaptive compression on the images in the image set, performing spatial compression on the time-dimensional adaptively compressed images, extracting feature points from the spatially compressed images, and performing number compression on the feature points.

[0020] The estimation module estimates the pose of the image acquisition device based on image A and image B.

[0021] Furthermore, in the filtering module, the images in the image set undergo time-dimensional adaptive compression using an interval sampling method. The interval sampling density is determined based on the acceleration at the time of image acquisition in the image set, as shown in the formula:

[0022] ;

[0023] In the formula, q j Let v be the sampling density at time j, ε be a constant greater than 0, and v j This represents the speed of the image acquisition device at time j. , a is the image acquisition interval. i Let be the acceleration at time i, which is the acceleration acquired by the image acquisition device or external IMU.

[0024] Furthermore, in the filtering module, spatial compression involves performing multi-resolution processing on the image after adaptive compression of the time dimension to generate an image pyramid; where the original image is the 0th layer in the image pyramid, and the higher the layer number, the lower the image resolution.

[0025] Based on the number of compressed feature points in image A, image B, which has the highest matching degree with image A, is selected from the preprocessed image set, including:

[0026] 1) Using feature point matching, images that initially match the nth layer of the second image pyramid are selected from all the nth layers of the first image pyramid; where the first image pyramid is the image pyramid corresponding to the image in the image set, and the second image pyramid is the image pyramid corresponding to image A;

[0027] 2) If the preliminary matching result of the nth layer meets the preset requirements, the image with the highest matching degree in the preliminary matching images shall be taken as image B;

[0028] If n is not equal to 0, and the initial matching result of the nth layer does not meet the preset requirements, n = n-1, go to 1).

[0029] If n equals 0 and the initial matching result of the nth layer does not meet the preset requirements, the image with the highest matching degree in the first image pyramid is taken as image B.

[0030] According to another aspect of this application, a computer-readable storage medium is provided that stores one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform a visual relocalization and pose estimation method.

[0031] According to another aspect of this application, a computer device is provided, including one or more processors and one or more memories, wherein one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, and the one or more programs include instructions for performing a visual relocalization and pose estimation method.

[0032] The beneficial effects achieved by this invention are as follows: 1. This invention performs spatial compression and feature point number compression on the image to be located, and performs temporal adaptive compression, spatial compression, and feature point number compression on the images in the image set that match the image to be located. Through this compression mechanism, the amount of image processing is reduced and the processing speed is improved, achieving a balance between efficiency and accuracy. It is suitable for high dynamic and large-scale scenes and for resource-constrained devices. 2. By using acceleration-driven temporal adaptive compression, image compression efficiency is maintained without sacrificing pose estimation accuracy, reducing the computational overhead of the algorithm and improving computational efficiency. Attached Figure Description

[0033] Figure 1 A flowchart for visual relocalization and pose estimation methods;

[0034] Figure 2 Here is a framework diagram of the SurperPoint model;

[0035] Figure 3 Flowchart for preprocessing an image set;

[0036] Figure 4 Detailed flowchart of image set preprocessing;

[0037] Figure 5 A flowchart for selecting the image with the highest matching degree;

[0038] Figure 6 Here is a diagram of the LightGlue network structure;

[0039] Figure 7 A detailed flowchart for selecting the image with the highest matching degree;

[0040] Figure 8 A flowchart for estimating the pose of an image acquisition device;

[0041] Figure 9 This is a block diagram of a visual relocalization and pose estimation device. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this application or its application or use. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0043] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application.

[0044] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0045] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0046] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0047] It should be noted that similar symbols and letters in the accompanying drawings represent similar items; therefore, once an item is defined in one accompanying drawing, it does not need to be discussed further in subsequent accompanying drawings.

[0048] This application provides a visual relocalization and pose estimation method, which aims to reduce image processing volume and improve processing speed through compression technology, making it suitable for the navigation field. The estimation method can be executed by an estimation device, which can be a terminal device or a server. The terminal device can include, but is not limited to, mobile phones, computers, smart wearable devices, smart vehicle devices, etc., as described in this application. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, big data, and artificial intelligence platforms, etc., as described in this application. Optionally, the estimation method can also be executed collaboratively by multiple electronic devices with computing power; for ease of explanation, subsequent embodiments will be described as being executed by an estimation device.

[0049] See Figure 1 , Figure 1 This is a flowchart of a visual relocalization and pose estimation method based on spatiotemporal dual compression provided in an embodiment of this application. The method can be executed by an estimation device and may include at least the following steps:

[0050] Step 1: Acquire the image A to be located from the image acquisition device, perform spatial compression on image A, extract the feature points of the spatially compressed image A, and perform numerical compression on the feature points.

[0051] It should be noted that feature points mainly include image keypoint machine 256-dimensional descriptors, which can be used... Figure 2 The SuperPoint model shown extracts feature points from an image. Specifically, it extracts image features through a convolutional network, selects key points and their 256-dimensional descriptors, and then standardizes the features around the key points. The resulting feature vector can adapt to rotation and scale changes.

[0052] To facilitate subsequent retrieval during filtering, a FAISS index needs to be constructed. This involves further organizing all feature points into a feature set and using a high-dimensional vector index type. The vectors are then grouped using clustering methods, which can accelerate the search process.

[0053] It should be noted that the singular value decomposition method can be used here to compress the number of feature points. Specifically, singular value decomposition (SVD) is performed on the feature points, and the top k principal components (top-k) are selected as the compressed feature points.

[0054] It should be noted that the spatial compression method for image A is the same as that for subsequent images, and the specific process will be described in detail later.

[0055] Step 2: Based on the number of compressed feature points of image A, select image B with the highest matching degree with image A from the preprocessed image set; where the image set is the image set for constructing the scene map; the preprocessing of the image set includes performing time-dimensional adaptive compression on the images in the image set, performing spatial compression on the time-dimensional adaptively compressed images, extracting feature points from the spatially compressed images, and performing number compression on the feature points.

[0056] It should be noted that the images stored in the image set are for constructing scene maps; that is, scene maps can be constructed from the images in the image set. To reduce the amount of image processing, the image set needs to be compressed beforehand. The specific process can be described as follows: Figure 3 As shown, it includes:

[0057] 21) Perform time-dimensional adaptive compression on the images in the image set.

[0058] In some embodiments, time-dimensional adaptive compression can employ an interval sampling method. The interval sampling density is determined based on the acceleration at the image acquisition time in the image set, where the acceleration is the acceleration acquired by the image acquisition device or an external IMU. Specifically, it employs a strategy of taking one frame every q frames. This interval sampling method can effectively compress the number of images and achieve time-domain compression, where q is the image sampling density in the time domain.

[0059] Assuming the images used for mapping (scene map) are and the subscripts represent image sequence numbers, in a single, relatively stable mapping process, the accelerations corresponding to the image sequence acquisition times are {a0, a1, ..., a...}. n-1}, then the speed v of the image acquisition device at time j. j It can be represented as:

[0060] ;

[0061] In the formula, a is the image acquisition interval. i Let v be the acceleration at time i. j The acceleration sequence acquired by the selected image acquisition device or external IMU {a i (From initial time 0 to time j), segment by segment and corresponding time interval The product is obtained by multiplying and then summing.

[0062] Then the interval sampling density at time j can be expressed as:

[0063] ;

[0064] In the formula, q j Let be the interval sampling density at time j, and ε be a constant greater than 0.

[0065] As can be seen from the above formula, when the image sensor moves very fast, the interval sampling density value decreases, indicating an increase in sampling density; conversely, when the sensor moves very slowly, the interval sampling density value increases, indicating a decrease in sampling density. By using acceleration-driven adaptive compression in the time dimension, image compression efficiency is maintained without sacrificing pose estimation accuracy, thus reducing the computational overhead and improving computational efficiency.

[0066] It should be noted that the above-mentioned time-dimensional adaptive compression can also be predicted using LSTM-based sequence models.

[0067] 22) Perform spatial compression on the image after adaptive compression of the time dimension.

[0068] Because the details of images vary greatly at different scales in different scenarios, a single resolution is insufficient to extract effective information comprehensively and efficiently. Therefore, spatial compression of images is necessary.

[0069] In some embodiments, spatial compression specifically involves performing multi-resolution processing on the temporally compressed image to generate an image pyramid; wherein, in the image pyramid, the original image is layer 0, and the higher the layer number, the lower the image resolution; for example, layer 0 is the original resolution layer, layer 1 is the 1 / 2 resolution layer, layer 2 is the 1 / 4 resolution layer, and layer 3 is the 1 / 8 resolution layer.

[0070] To achieve multi-resolution images, in some embodiments, the image can be Gaussian blurred and then downsampled. This allows the image to transition smoothly at different scales, better adapting to subsequent multi-resolution analysis. The formula can be expressed as:

[0071] ;

[0072] ;

[0073] In the formula, G(x,y) represents the response after applying a Gaussian filter to the image I(x,y), (x,y) represents the coordinate position of the image pixel (x is the horizontal coordinate, y is the vertical coordinate, used to locate points in the image), σ is the standard deviation of the Gaussian kernel, controlling the smoothness, * represents the convolution operation, e is the natural constant, and P l (x,y) represents the Gaussian response at scale l, where s is the scale factor used to control the spatial scaling of the Gaussian kernel, thereby enabling the extraction of image features at different scales. This represents the value of the Gaussian kernel function at scale l after scaling by a factor s, used to extract image features at different scales.

[0074] It should be noted that in some embodiments, wavelet transform (DWT) can be used instead of adaptive downsampling.

[0075] 23) Extract feature points from the spatially compressed image and compress the number of feature points.

[0076] It should be noted that the feature point extraction and number compression here are the same as in step 1, and will not be described again here.

[0077] Taking a time-dimensional adaptive compression method and a spatial compression method as examples, see [link to relevant documentation]. Figure 4The image set preprocessing process can be as follows: first, the images in the image set are adaptively compressed in the time dimension using the interval sampling method; then, the images are processed by Gaussian blurring and downsampled; further, the SuperPoint model is used to extract the feature points of each layer of the image and construct the FAISS index; finally, the singular value decomposition method is used to compress the number of feature points.

[0078] In some embodiments, see Figure 5 The process of selecting image B, which has the highest matching degree with image A, from the preprocessed image set may include:

[0079] S1) Using feature point matching, images that initially match the nth layer of the second image pyramid are selected from all the nth layers of the first image pyramid; where the first image pyramid is the image pyramid corresponding to the image in the image set, and the second image pyramid is the image pyramid corresponding to image A.

[0080] Further utilization such as Figure 6 The LightGlue network shown performs matching. First, for the image to be located (image A), and the image selected from the image set for matching with the image to be located each time the algorithm performs image matching (image D), local features are extracted from both, denoted as d. A p A With d D p D Secondly, in the first layer, the self-attention module captures the contextual relationships of features within a single graph, the cross-attention mechanism module establishes the interaction relationships between features between graphs, and a confidence module is introduced to ensure the reliability of feature matching. If the matching pair does not meet the exit condition, the LightGlue network will filter redundant features through pruning. Subsequently, after multiple iterations, when the exit condition is met, the matching module is entered. Based on the similarity matrix and matching degree analysis, the final feature allocation is completed, and the matching feature point pairs are output to achieve accurate cross-graph feature association.

[0081] After obtaining the feature matching point pairs output by LightGlue, the score of the matching point pairs is calculated, which can be expressed by the formula:

[0082] ;

[0083] In the formula, s pq Let be the matching score between the point p to be matched (i.e., the point in image A) and the map point q (i.e., the point in image D). For the sigmoid activation function, a phFor sparse attention weights (LightGlue Transformer layer), q p and k h Let T be the vector to be matched, T be the transpose, and H be the number of detector heads in the attention mechanism. The score threshold is used to prune and remove low-score candidates. Low-confidence candidates are pruned based on this score threshold to achieve space compression.

[0084] S2) If the preliminary matching result of the nth layer meets the preset requirements, the image with the highest matching degree in the preliminary matching images is taken as image B; if n is not equal to 0 and the preliminary matching result of the nth layer does not meet the preset requirements, n=n-1, go to S1); if n is equal to 0 and the preliminary matching result of the nth layer does not meet the preset requirements, the image with the highest matching degree in the first image pyramid is taken as image B.

[0085] It should be noted that the confidence level of the initial matching result of the current layer can be used as the judgment condition. That is, the confidence level is calculated based on the initial matching result of the current layer, and the formula can be expressed as:

[0086] ;

[0087] In the formula, For the first The confidence level of the initial matching results of the layer, where I is the number of interior points. For reprojection error, The standard deviation of the error (<2 pixels). This is the LightGlue attention score.

[0088] The preset requirement can be defined as a confidence level greater than a confidence threshold, such as 0.8.

[0089] See Figure 7 At the current resolution, after performing ANN search and LightGlue matching, if several images exist with a confidence score greater than 0.8, these images are directly output at the current resolution, and the image with the highest matching score is selected as the final matching result. If the current resolution cannot meet the preset requirements, the image resolution is increased, i.e., matching is performed on the next layer of images, and ANN search and LightGlue matching are repeated. However, if even after increasing the resolution to the maximum, an image that meets the score cannot be obtained, the one with the highest score is selected as the matching result.

[0090] At low resolution, image features are blurry and the number of matching features is small. This characteristic reduces the computational cost of the coarse matching stage, while gradually increasing the resolution is a key measure to provide more accurate image features for subsequent precise positioning.

[0091] Step 3: Estimate the pose of the image acquisition device based on image A and image B.

[0092] It should be noted that, see Figure 8 The pose estimation process involves first performing RANSAC filtering of outliers and PnP estimation, and then outputting the final optimized 6-DoF pose.

[0093] The first step is to perform interior point filtering on the selected layer, and then use RANSAC iterative filtering to filter out exterior points. The formula can be expressed as:

[0094] ;

[0095] In the formula, For interior point set, These are the observed 2D points and the reprojected points based on the assumed attitude, respectively. Let θ be the error threshold (<2 pixels), θ be the minimum inlier threshold, and N be the total number of initial matched point pairs. To calculate the Euclidean distance between two points, this step robustly matches the results, handles noise and dynamic scenes, and ensures reliable interior points are used for subsequent estimations.

[0096] The second step is to solve for the 6-DoF pose based on the interior point set and minimize the reprojection error. The formula can be expressed as:

[0097] ;

[0098] In the formula, The optimal camera pose estimate is obtained by minimizing the reprojection error, where R is a 3×3 rotation matrix, t is a 3×1 translation vector, and K is the camera intrinsic parameter matrix. For 3D map points, For the corresponding 2D projection points.

[0099] The above method compresses the number of feature points in the image to be located, and performs time-dimensional adaptive compression, spatial compression, and feature point number compression on the images in the image set that match the image to be located. Through this compression mechanism, the amount of image processing is reduced and the processing speed is improved. It can achieve a balance between efficiency and accuracy, and is suitable for high dynamic and large-scale scenes and resource-constrained devices.

[0100] See Figure 9 , Figure 9 This is a block diagram of a visual relocalization and pose estimation device provided in an embodiment of this application. This embodiment is a virtual device that can be loaded and executed by a computer device, which may include the aforementioned estimation device. Figure 9 The apparatus may include an acquisition / extraction module, a filtering module, and an estimation module, which, when performing the above estimation method, can:

[0101] The acquisition and extraction module acquires the image A to be located from the image acquisition device, performs spatial compression on image A, extracts the feature points of the spatially compressed image A, and performs numerical compression on the feature points.

[0102] The filtering module selects the image B with the highest matching degree with image A from the preprocessed image set based on the number of compressed feature points of image A; where the image set is the image set used to construct the scene map; the preprocessing of the image set includes performing time-dimensional adaptive compression on the images in the image set, performing spatial compression on the time-dimensional adaptively compressed images, extracting feature points from the spatially compressed images, and performing number compression on the feature points.

[0103] It should be noted that in the filtering module, the time-dimensional adaptive compression of the images in the image set adopts an interval sampling method. The interval sampling density is determined according to the acceleration at the time of image acquisition in the image set, and the formula is:

[0104] ;

[0105] In the formula, q j Let v be the sampling density at time j, ε be a constant greater than 0, and v j This represents the speed of the image acquisition device at time j. , a is the image acquisition interval. i Let be the acceleration at time i, which is the acceleration acquired by the image acquisition device or external IMU.

[0106] By using acceleration-driven adaptive compression in the time dimension, image compression efficiency is achieved without sacrificing pose estimation accuracy, thereby reducing computational overhead and improving computational efficiency.

[0107] In the filtering module, spatial compression involves performing multi-resolution processing on the image after adaptive compression of the time dimension to generate an image pyramid. In the image pyramid, the original image is the 0th layer, and the higher the layer, the lower the image resolution.

[0108] From the preprocessed image set, select image B that has the highest matching degree with image A, including:

[0109] 1) Using feature point matching, images that initially match the nth layer of the second image pyramid are selected from all the nth layers of the first image pyramid; where the first image pyramid is the image pyramid corresponding to the image in the image set, and the second image pyramid is the image pyramid corresponding to image A;

[0110] 2) If the preliminary matching result of the nth layer meets the preset requirements, the image with the highest matching degree in the preliminary matching images shall be taken as image B;

[0111] If n is not equal to 0, and the initial matching result of the nth layer does not meet the preset requirements, n = n-1, go to 1).

[0112] If n equals 0 and the initial matching result of the nth layer does not meet the preset requirements, the image with the highest matching degree in the first image pyramid is taken as image B.

[0113] The estimation module estimates the pose of the image acquisition device based on image A and image B.

[0114] The aforementioned device compresses the number of feature points in the image to be located, and performs time-dimensional adaptive compression, spatial compression, and feature point number compression on the images in the image set that match the image to be located. Through this compression mechanism, the amount of image processing is reduced and the processing speed is improved, achieving a balance between efficiency and accuracy. It is suitable for high dynamic and large-scale scenarios and for resource-constrained devices.

[0115] This application also relates to a computer-readable storage medium that stores one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform a visual relocalization and pose estimation method.

[0116] This application also relates to a computer device including one or more processors and one or more memories, wherein one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, and the one or more programs include instructions for performing a visual relocalization and pose estimation method.

[0117] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxesFigure 1 A device that provides the functions specified in one or more boxes.

[0119] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0120] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0121] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of the claims of the present invention pending approval.

Claims

1. A visual relocalization and pose estimation method based on spatiotemporal dual compression, characterized in that, include: Image A to be located is acquired from the image acquisition device. Image A is spatially compressed. Feature points of the spatially compressed image A are extracted and the number of feature points is compressed. Based on the feature points of image A after compression, image B with the highest matching degree with image A is selected from the preprocessed image set; where the image set is the image set used to construct the scene map; the preprocessing of the image set includes time-dimensional adaptive compression of the images in the image set, spatial compression of the time-dimensional adaptively compressed images, extraction of feature points of the spatially compressed images, and number compression of the feature points. Estimate the pose of the image acquisition device based on image A and image B.

2. The method according to claim 1, characterized in that, Adaptive temporal compression of the images in the image set is performed using an interval sampling method. The interval sampling density is determined based on the acceleration at the time of image acquisition in the image set, as shown in the formula: ； In the formula, q j For the first j sampling density at time intervals, ε A constant greater than 0 v j Indicates the first j The speed of the image acquisition device at any given time. , Image acquisition interval a i For the first i The acceleration at any given moment is the acceleration captured by the image acquisition device or external IMU.

3. The method according to claim 1, characterized in that, Spatial compression involves performing multi-resolution processing on the temporally compressed image to generate an image pyramid; in the image pyramid, the original image is the 0th layer, and the higher the layer, the lower the image resolution. Based on the number of compressed feature points in image A, image B, which has the highest matching degree with image A, is selected from the preprocessed image set, including: 1) Using feature point matching, images that initially match the nth layer of the second image pyramid are selected from all the nth layers of the first image pyramid; where the first image pyramid is the image pyramid corresponding to the image in the image set, and the second image pyramid is the image pyramid corresponding to image A; 2) If the preliminary matching result of the nth layer meets the preset requirements, the image with the highest matching degree in the preliminary matching images shall be taken as image B; If n is not equal to 0, and the initial matching result of the nth layer does not meet the preset requirements, n = n-1, go to 1). If n equals 0 and the initial matching result of the nth layer does not meet the preset requirements, the image with the highest matching degree in the first image pyramid is taken as image B.

4. A visual relocalization and pose estimation device based on spatiotemporal dual compression, characterized in that, include: The acquisition and extraction module acquires the image A to be located from the image acquisition device, performs spatial compression on image A, extracts the feature points of the spatially compressed image A, and performs numerical compression on the feature points. The filtering module selects the image B with the highest matching degree with image A from the preprocessed image set based on the number of compressed feature points of image A; where the image set is the image set used to construct the scene map; the preprocessing of the image set includes performing time-dimensional adaptive compression on the images in the image set, performing spatial compression on the time-dimensional adaptively compressed images, extracting feature points from the spatially compressed images, and performing number compression on the feature points. The estimation module estimates the pose of the image acquisition device based on image A and image B.

5. The apparatus according to claim 4, characterized in that, In the filtering module, time-dimensional adaptive compression of the images in the image set is performed using an interval sampling method. The interval sampling density is determined based on the acceleration at the time of image acquisition in the image set, as shown in the formula: ； In the formula, q j For the first j sampling density at time intervals, ε A constant greater than 0 v j Indicates the first j The speed of the image acquisition device at any given time. , Image acquisition interval a i For the first i The acceleration at any given moment is the acceleration captured by the image acquisition device or external IMU.

6. The apparatus according to claim 4, characterized in that, In the filtering module, spatial compression involves performing multi-resolution processing on the image after adaptive compression of the time dimension to generate an image pyramid; in the image pyramid, the original image is the 0th layer, and the higher the layer number, the lower the image resolution. Based on the number of compressed feature points in image A, image B, which has the highest matching degree with image A, is selected from the preprocessed image set, including: 1) Using feature point matching, images that initially match the nth layer of the second image pyramid are selected from all the nth layers of the first image pyramid; where the first image pyramid is the image pyramid corresponding to the image in the image set, and the second image pyramid is the image pyramid corresponding to image A; 2) If the preliminary matching result of the nth layer meets the preset requirements, the image with the highest matching degree in the preliminary matching images shall be taken as image B; If n is not equal to 0, and the initial matching result of the nth layer does not meet the preset requirements, n = n-1, go to 1). If n equals 0 and the initial matching result of the nth layer does not meet the preset requirements, the image with the highest matching degree in the first image pyramid is taken as image B.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the method of any one of claims 1 to 3.

8. A computer device, characterized in that, include: One or more processors and one or more memories, one or more programs stored in one or more memories and configured to be executed by one or more processors, the one or more programs including instructions for performing the method of any one of claims 1 to 3.

Images