Visual localization method based on 3D LoD map and neural wireframe alignment

Abstract

The present application relates to a visual positioning method based on three-dimensional LoD map and neural wireframe alignment. The method comprises: uniformly sampling the four degrees of freedom with the initial posture as the center, generating posture hypotheses in four directions respectively; calculating the straight line alignment cost of the posture hypothesis and the pre-constructed three-dimensional wireframe points according to the neural wireframe alignment method, combining the straight line alignment costs in the four directions in a grid manner to calculate the posture cost volume and then calculating the probability distribution volume; determining the posture sampling range of the next layer according to the probability distribution volume to generate the posture hypothesis of the next layer, and taking the selected posture obtained at the last feature level as the candidate selected posture; mapping the multi-level features, using the mapped features and the three-dimensional wireframe points to design the optimization objective function of the candidate selected posture, and then solving it to obtain the final posture. The use of this method can improve the accuracy of visual positioning and reduce memory.

Description

Technical Field

[0001] The present application relates to the field of visual positioning technology, and in particular to a visual positioning method based on alignment of a three-dimensional LoD map with a neural wireframe. Background Art

[0002] Visual localization is the process of determining the position and orientation of a given image relative to a known map, i.e., the camera pose. This is a fundamental problem in many 3D computer vision applications, from autonomous driving and navigation of unmanned aerial vehicles (UAVs) to augmented reality. Current state-of-the-art visual localization methods typically involve matching pixels in a query image with points in a pre-built high-quality 3D map, typically derived from structure from motion (SfM) or a 3D texture model. Subsequently, a perspective n-point RANSAC technique is typically used to compute the camera pose.

[0003] However, building high-quality 3D maps worldwide using photogrammetry is expensive and requires frequent data updates to capture temporal changes in visual appearance. Furthermore, the storage cost of these 3D maps is high, posing a significant challenge for their deployment on mobile devices such as cell phones and drones. Furthermore, high-resolution 3D maps reveal detailed information about the location of a location, raising critical issues regarding homeland security and privacy. Summary of the Invention

[0004] Based on this, it is necessary to provide a visual positioning method based on alignment of three-dimensional LoD map and neural wireframe, which can improve the visual positioning accuracy and achieve low memory storage, in order to address the above technical problems.

[0005] A visual positioning method based on alignment of a three-dimensional LoD map with a neural wireframe, the method comprising:

[0006] Obtain the query image taken by the drone on the 3D city LoD map and the prior pose of the drone sensor; build a visual positioning model; the visual positioning model includes a feature extraction module, a pose selection module, and a pose optimization module;

[0007] In the feature extraction module, the query image is extracted at multiple levels based on the convolutional neural network to obtain multi-level features;

[0008] In the posture selection module, the initial posture of the current feature level is defined, the four degrees of freedom are uniformly sampled with the initial posture as the center, and posture hypotheses are generated in four directions respectively; the straight line alignment cost of the posture hypothesis and the pre-constructed three-dimensional wireframe points is calculated according to the neural wireframe alignment method, and the straight line alignment costs in the four directions are combined in a grid manner to obtain the posture cost volume; the posture cost volume is calculated according to the softmax function to obtain the probability distribution volume; the argmax operation is performed on the probability distribution volume to obtain the selected posture; the variance of the probability distribution volume of the current feature level at the current level is used to determine the posture sampling range of the next level to generate the posture hypothesis of the next level, and the selected posture obtained at the last feature level is used as the candidate selected posture; the initial posture on the first feature level is the prior posture;

[0009] In the pose optimization module, multi-level features are mapped, and the mapped features and 3D wireframe points are used to design the optimization objective function of the candidate selected pose. The optimization objective function is solved according to the Gauss-Newton method to obtain the final pose.

[0010] The posture selection module is trained according to a preset posture selection loss function to obtain a trained posture selection module; the posture optimization module is optimized using a preset posture optimization loss function to obtain a trained posture optimization module;

[0011] Use the trained visual localization model to perform visual localization on the input image.

[0012] In one embodiment, defining an initial pose at the current feature level includes:

[0013] The initial pose is defined at the current feature level as Where (x l ,y l ,z l ) represents the translation in three-dimensional space, They represent the yaw angle, pitch angle and roll angle respectively, and l represents the feature level number.

[0014] In one embodiment, the four degrees of freedom are uniformly sampled with the initial pose as the center, and pose hypotheses are generated in four directions, including:

[0015] The four degrees of freedom are uniformly sampled with the initial posture as the center, and the posture assumptions generated in the four directions are:

[0016]

[0017] Among them, d∈((x, y, z, θ), (x l ,y l ,z l) represents the translation in three-dimensional space, θ l represents the yaw angle, r l Indicates the sampling range, m l Indicates the number of samples, and l indicates the feature level number.

[0018] In one embodiment, calculating a straight line alignment cost between a pose hypothesis and pre-constructed 3D wireframe points according to a neural wireframe alignment method includes:

[0019] According to the neural wireframe alignment method, the straight line alignment cost of the pose hypothesis and the pre-constructed 3D wireframe points is calculated as

[0020]

[0021] in, represents the pose hypothesis, F l represents the features of layer l, P i Represents a 3D wireframe point.

[0022] In one embodiment, the process of calculating the variance of the probability distribution volume of the current feature level at the current level includes:

[0023] Due to the posture assumption Pose cost volume C l and the probability distribution volume P l With the same data structure, flatten them and index them with t, the variance v at level l l Calculated as

[0024]

[0025] Among them, l represents the current feature level number, P l-1 represents the probability distribution volume of the l-1 layer, represents the selected pose of layer l-1.

[0026] In one embodiment, the variance of the probability distribution volume of the current feature level at the current level is used to determine the gesture sampling range of the next level, including:

[0027] The standard deviation is calculated based on the variance: Use standard deviation to determine the attitude sampling range as r l =2λ·σ l , where λ is a hyperparameter that adjusts the length of the sampling range, and l represents the feature level number.

[0028] In one embodiment, the optimized objective function for candidate selection poses is designed using the mapped features and the 3D wireframe points, including:

[0029] The optimization objective function of the candidate selection pose is designed using the mapped features and 3D wireframe points:

[0030]

[0031] Among them, F rf Represents the mapped features, P i represents a 3D wireframe point, ξ * =(R * ,t * ) represents the final posture, and π is the projection operation.

[0032] In one embodiment, the preset posture selection loss function is

[0033]

[0034] Among them, l∈{1,2,3}, p l represents the probability distribution volume, represents the ground truth pose.

[0035] In one embodiment, the preset posture optimization loss function is

[0036]

[0037] Among them, the ground truth pose ρ represents the Huber robust kernel, ξ * =(R * ,t * ) represents the final posture, and π is the projection operation.

[0038] The above-mentioned visual positioning method based on alignment of three-dimensional LoD map with neural wireframe, this application realizes visual positioning by constructing a visual positioning model on the three-dimensional city LoD map, and using the query image taken by the drone and its sensor prior pose as input. Compared with the existing complex 3D representation, this application uses the level of detail LoD map to estimate the pose of the drone, providing a simple, accessible and privacy-friendly scene representation. In the constructed visual positioning model, multi-layer feature extraction is first performed on the query image, and a hierarchical pose estimation scheme is proposed, which uses multiple small pose volumes instead of large cost volumes to gradually calculate high-quality poses in a coarse-to-fine manner. In the hierarchical process, an adaptive sampling strategy is adopted, in which the uncertainty based on variance in the previous stage affects the sampling range of the next stage to construct the pose cost volume. This adaptive process induces a reasonable and fine-grained pose space division, which significantly improves the final pose output, improves the visual positioning accuracy and achieves low memory storage. In addition, after the coarse-to-fine pose estimation stage, multi-level features are mapped, and the mapped features and 3D wireframe points are used to design the optimization objective function of the candidate selection pose. The optimization objective function is solved according to the Gauss-Newton method to obtain the final pose, correcting the slight error in the initial gravity prior, thereby improving the overall pose accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 1 is a flow chart of a visual positioning method based on alignment of a three-dimensional LoD map with a neural wireframe in one embodiment;

[0040] Figure 2 Schematic diagram of a framework of a visual positioning method in one embodiment;

[0041] Figure 3 Schematic diagram of the uncertainty sampling range estimation process in one embodiment. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0043] In one embodiment, Figure 1 and Figure 2 As shown, a visual positioning method based on alignment of a three-dimensional LoD map with a neural wireframe is provided, comprising the following steps:

[0044] Step 102: Obtain a query image captured by the UAV on the 3D city LoD map and a priori posture of the UAV sensor; construct a visual positioning model; the visual positioning model includes a feature extraction module, a posture selection module, and a posture optimization module.

[0045] Step 104: In the feature extraction module, feature extraction is performed on the query image at multiple levels according to the convolutional neural network to obtain multi-level features.

[0046] A convolutional neural network with UNet is used to extract multi-level features from the query image, maintaining a high-dimensional feature map to encapsulate the rich visual information of each layer, abstracting the dimension of the feature map and reducing it to 1, where each pixel in the map represents the possibility of becoming a wireframe. The obtained multi-level features are used Represents, where l = {1, 2, 3} is the level index.

[0047] Step 106, in the posture selection module, define the initial posture on the current feature level, uniformly sample the four degrees of freedom with the initial posture as the center, and generate posture hypotheses in four directions respectively; calculate the straight line alignment cost of the posture hypothesis and the pre-constructed three-dimensional wireframe points according to the neural wireframe alignment method, and combine the straight line alignment costs in the four directions in a grid manner to obtain the posture cost volume; calculate the posture cost volume according to the softmax function to obtain the probability distribution volume; perform argmax operation on the probability distribution volume to obtain the selected posture; use the variance of the probability distribution volume of the current feature level at the current level to determine the posture sampling range of the next layer to generate the posture hypothesis of the next layer, and use the selected posture obtained at the last feature level as the candidate selected posture; the initial posture on the first feature level is the prior posture.

[0048] After feature extraction, a cost volume is established for various pose hypotheses sampled around the pose, and then the pose with the highest probability is selected at each level. In order to ensure effective sampling, the uncertainty of the pose selection at the current layer is used to determine the pose sampling range of the next layer. First, the initial pose on the current feature level is defined. Considering that the pitch and roll of the gravity direction of the previous data have high accuracy, only four degrees of freedom are uniformly sampled with the initial pose as the center, and pose hypotheses are generated in four directions respectively. The pose hypothesis is generated by sampling four degrees of freedom (including position and yaw angle) around the prior, assuming that the gravity direction provided by the inertial unit has a small error. Based on the generated pose hypothesis, the LoD building wireframe is projected onto the query image plane. Each pose hypothesis is then scored by the alignment between the predicted wireframe and the predicted wireframe. By combining these alignment costs in a grid manner in four different dimensions (x, y, Z, θ), a dimension of [m l (x)×m l (y)×m l (z)×m l (θ)] 4D pose cost volume C l . lExecute the softmax function to obtain the probability distribution volume Pl, and derive the probability density of the posture, which can be used for posture estimation through classification (similar to classification classification) or regression (similar to position expectation) operations. l Perform argmax operation to select the pose with the maximum probability The whole process interprets the output of pose estimation as a probability distribution parameterized by a learnable network to predict a 2D wireframe of the query image. In the uncertain sampling range estimation process, the pose selection uncertainty of the previous layer is used to determine the sampling range of the current layer in the coarse-to-fine process. The sampling range of the current level. This strategy allows for a progressive subdivision of the pose sampling space, thereby improving the accuracy of pose selection. More specifically, for l = 1, the pose sampling range is defined by carefully examining the maximum divergence between the prior and GT poses in the UAVD4L-LoD dataset. The sampling range of (x, y, z, θ) is established as [r1(x), r1(y), r1(z), r1(z)](=([r p (x), r p (y), r p (z), r p (z)]. For l = {2, 3}, use the probability distribution volume P l-1 The variance at l-1 determines the attitude sampling range r l , where, due to the pose assumption Pose cost volume C l and the probability distribution volume P l With the same data structure, flatten them and index them with t. The variance v at level l l Calculated as

[0049]

[0050] symbol Represents the subtraction operation applied to the (x, y, z, θ) directions respectively. The corresponding standard deviation is calculated as The calculated attitude sampling range is r l =2λ·σ l , where λ is a hyperparameter that adjusts the length of the sampling range. This uncertainty sampling range estimation process can be visualized as follows Figure 3 shown.

[0051] Our proposed layered pose estimation scheme utilizes multiple small pose volumes, rather than a large cost volume, to gradually compute high-quality poses in a coarse-to-fine manner. During the layering process, an adaptive sampling strategy is employed, where the variance-based uncertainty of the previous stage influences the sampling range of the next stage to construct the pose cost volume. This adaptive process induces a reasonable, fine-grained partitioning of the pose space, significantly improving the final pose output, enhancing visual localization accuracy, and enabling low-memory storage.

[0052] Step 108 , in the posture optimization module, the multi-level features are mapped, and the optimization objective function of the candidate selection posture is designed using the mapped features and the three-dimensional wireframe points. The optimization objective function is solved according to the Gauss-Newton method to obtain the final posture.

[0053] Based on the posture selected in the previous stage Use the post-processed convolutional network to extract the feature map F rf The refined wireframe probability map F3 is further extracted to optimize the pose ξ * =(R * , t * ) to align the 3D wireframe with the 2D predicted wireframe. The optimized objective function for candidate selected poses is designed using the mapped features and 3D wireframe points:

[0054]

[0055] Among them, P i is a 3D wireframe point, and Π is a projection operation. Minimizing this function allows the projected 3D wireframe point to move to a 2D position with a higher predicted probability. ξ is derived from the Gauss-Newton method. * The pose update formula is

[0056]

[0057] R * =R * ·exp(△ξ r )

[0058] Where, is the six-dimensional transformation vector, is the rotation component, is the translation component. Using Lie algebra The exponential mapping of the rotation component △ξ r Converted into a 3×3 rotation matrix. Among them, J i Represents the residual function f i The Jacobian matrix of the pose parameters.

[0059] Step 110: train the posture selection module according to a preset posture selection loss function to obtain a trained posture selection module; optimize the posture optimization module using a preset posture optimization loss function to obtain a trained posture optimization module; and perform visual positioning on the input image using the trained visual positioning model.

[0060] The pre-set pose selection loss function is used to constrain the negative log-likelihood loss of the pose selection module. This means that the closer the selected pose is to the true value, the greater the probability of that pose, which can improve the accuracy of pose selection. The pre-set pose optimization loss function is the loss function of the pose optimization module (Newton iterative optimization), which uses the reprojection error to constrain pose adjustments. By combining these two loss functions, the final pose result can be optimized from coarse to fine.

[0061] In the above-mentioned visual positioning method based on alignment of three-dimensional LoD map with neural wireframe, this application realizes visual positioning by constructing a visual positioning model on the three-dimensional city LoD map, and using the query image taken by the drone and its sensor prior pose as input. Compared with the existing complex 3D representation, this application uses the level of detail LoD map to estimate the pose of the drone, providing a simple, accessible and privacy-friendly scene representation. In the constructed visual positioning model, multi-layer feature extraction is first performed on the query image, and a hierarchical pose estimation scheme is proposed, which uses multiple small pose volumes instead of large cost volumes to gradually calculate high-quality poses in a coarse-to-fine manner. In the hierarchical process, an adaptive sampling strategy is adopted, in which the uncertainty based on variance in the previous stage affects the sampling range of the next stage to construct the pose cost volume. This adaptive process induces a reasonable and fine-grained pose space division, which significantly improves the final pose output, improves the visual positioning accuracy and achieves low memory storage. In addition, after the coarse-to-fine pose estimation stage, multi-level features are mapped, and the mapped features and 3D wireframe points are used to design the optimization objective function of the candidate selection pose. The optimization objective function is solved according to the Gauss-Newton method to obtain the final pose, correcting the slight error in the initial gravity prior, thereby improving the overall pose accuracy.

[0062] In one embodiment, defining an initial pose at the current feature level includes:

[0063] The initial pose is defined at the current feature level as Where (x l ,y l ,z l ) represents the translation in three-dimensional space, They represent the yaw angle, pitch angle and roll angle respectively, and l represents the feature level number.

[0064] In one embodiment, the four degrees of freedom are uniformly sampled with the initial pose as the center, and pose hypotheses are generated in four directions, including:

[0065] The four degrees of freedom are uniformly sampled with the initial posture as the center, and the posture assumptions generated in the four directions are:

[0066]

[0067] Among them, d∈((x, y, z, θ), (x l ,y l ,z l ) represents the translation in three-dimensional space, θ l represents the yaw angle, r l Indicates the sampling range, m l Indicates the number of samples, and l indicates the feature level number.

[0068] In one embodiment, calculating a straight line alignment cost between a pose hypothesis and pre-constructed 3D wireframe points according to a neural wireframe alignment method includes:

[0069] According to the neural wireframe alignment method, the straight line alignment cost of the pose hypothesis and the pre-constructed 3D wireframe points is calculated as

[0070]

[0071] in, represents the pose hypothesis, F l represents the features of layer l, P i represents a 3D wireframe point, and [·] is a sub-pixel interpolation lookup.

[0072] In a specific embodiment, the construction process of three-dimensional wireframe points is prior art and will not be described in detail in this application.

[0073] In one embodiment, the process of calculating the variance of the probability distribution volume of the current feature level at the current level includes:

[0074] Due to the posture assumption Pose cost volume C l and the probability distribution volume P l With the same data structure, flatten them and index them with t, the variance v at level l l Calculated as

[0075]

[0076] Among them, l represents the current feature level number, P l-1 represents the probability distribution volume of the l-1 layer, represents the selected pose of layer l-1.

[0077] In one embodiment, the variance of the probability distribution volume of the current feature level at the current level is used to determine the gesture sampling range of the next level, including:

[0078] The standard deviation is calculated based on the variance: Use standard deviation to determine the attitude sampling range as r l =2λ·σ l , where λ is a hyperparameter that adjusts the length of the sampling range, and l represents the feature level number.

[0079] In one embodiment, the optimized objective function for candidate selection poses is designed using the mapped features and the 3D wireframe points, including:

[0080] The optimization objective function of the candidate selection pose is designed using the mapped features and 3D wireframe points:

[0081]

[0082] Among them, F rf Represents the mapped features, P i represents a 3D wireframe point, ξ * =(R * ,t * ) represents the final posture, and π is the projection operation.

[0083] In one embodiment, the preset posture selection loss function is

[0084]

[0085] Among them, l∈{1,2,3}, p l represents the probability distribution volume, represents the ground truth pose.

[0086] In one embodiment, the preset posture optimization loss function is

[0087]

[0088] Among them, the ground truth pose ρ represents the Huber robust kernel, ξ * =(R * ,t * ) represents the final posture, and π is the projection operation.

[0089] In a specific embodiment, during the model training process, a random seed is set to retain the 3D wireframe points {P i} to 2000 points, and due to CUDA memory constraints, for level (l (=1, 2, 3, the number of pose samples ml (x), m l (y), m l (z), m l (θ) is uniformly assigned to [13,7,3]. The image size of the UAVD4L dataset is (512,480), and the image size of the Swiss-EPFL dataset is (720,480). The posture sampling range of level (1) is set to [10,10,30,7.5], which means [r p (x), r p (y), r p (z), r p (z)]. The hyperparameter λ is fixed to 1.5. For the UAVD4L-(lod dataset, a subset of synthetic images from UAVD4L

[64] is used as training data, which includes buildings. For Swiss-EPFL, the model is trained by combining synthetic images LHS from the CrossLoc project and real query images. During inference, the following changes are made to change the retrieved discrete points in the 3D wireframe at intervals of 1 meter. The number of pose samples is increased to [m l (x), m l (y), m l (z), m l (θ)](=[10,10,30,8]. λ is set to 0.8. The training and inference of the entire network are performed using two NVIDIA RTX 4090 GPUs.

[0090] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0091] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) (DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0092] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A visual positioning method based on alignment of three-dimensional LoD map and neural wireframe, characterized in that: The method comprises: Obtain a query image taken by a UAV on a three-dimensional city LoD map and a priori posture of the UAV sensor; construct a visual positioning model; the visual positioning model includes a feature extraction module, a posture selection module, and a posture optimization module; In the feature extraction module, feature extraction is performed on the query image at multiple levels according to a convolutional neural network to obtain multi-level features; In the posture selection module, the initial posture on the current feature level is defined, the four degrees of freedom are uniformly sampled with the initial posture as the center, and posture hypotheses are generated in four directions respectively; the straight line alignment cost of the posture hypothesis and the pre-constructed three-dimensional wireframe points is calculated according to the neural wireframe alignment method, and the straight line alignment costs in the four directions are combined in a grid manner to obtain the posture cost volume; the posture cost volume is calculated according to the softmax function to obtain the probability distribution volume; the argmax operation is performed on the probability distribution volume to obtain the selected posture; the variance of the probability distribution volume of the current feature level at the current level is used to determine the posture sampling range of the next level to generate the posture hypothesis of the next level, and the selected posture obtained at the last feature level is used as the candidate selected posture; the initial posture on the first feature level is the prior posture; In the posture optimization module, the multi-level features are mapped, an optimization objective function of a candidate selection posture is designed using the mapped features and the three-dimensional wireframe points, and the optimization objective function is solved according to the Gauss-Newton method to obtain a final posture; The posture selection module is trained according to a preset posture selection loss function to obtain a trained posture selection module; the posture optimization module is optimized using a preset posture optimization loss function to obtain a trained posture optimization module; Use the trained visual localization model to perform visual localization on the input image.

2. The method according to claim 1, characterized in that Define the initial pose at the current feature level, including: The initial pose is defined at the current feature level as Where (x l ,y l ,z l ) represents the translation in three-dimensional space, They represent the yaw angle, pitch angle and roll angle respectively, and l represents the feature level number.

3. The method according to claim 1, characterized in that The four degrees of freedom are uniformly sampled with the initial posture as the center, and posture hypotheses are generated in four directions, including: The four degrees of freedom are uniformly sampled with the initial posture as the center, and the posture assumptions generated in the four directions are: Among them, d∈(x, y, z, θ), (x l ,y l ,z l ) represents the translation in three-dimensional space, θ l represents the yaw angle, r l Indicates the sampling range, m l Indicates the number of samples, and l indicates the feature level number.

4. The method according to claim 1, wherein Calculating the straight line alignment cost of the pose hypothesis and the pre-constructed 3D wireframe points according to a neural wireframe alignment method, including: The straight line alignment cost of the pose hypothesis and the pre-constructed 3D wireframe points is calculated according to the neural wireframe alignment method: in, represents the pose hypothesis, F l represents the features of layer l, P i Represents a 3D wireframe point.

5. The method according to claim 1, characterized in that The calculation process of the variance of the probability distribution volume of the current feature level at the current level includes: Due to the posture assumption Pose cost volume C l and the probability distribution volume P l With the same data structure, flatten them and index them with t, the variance v at level l l Calculated as Among them, l represents the current feature level number, P l-1 represents the probability distribution volume of the l-1 layer, represents the selected pose of layer l-1.

6. The method according to claim 1, characterized in that The variance of the probability distribution volume of the current feature level at the current level is used to determine the posture sampling range of the next level, including: The standard deviation is calculated based on the variance: The standard deviation is used to determine the attitude sampling range as r l =2λ·σ l , where λ is a hyperparameter that adjusts the length of the sampling range, and l represents the feature level number.

7. The method according to claim 1, characterized in that The mapped features and the three-dimensional wireframe points are used to design an optimization objective function for candidate selection poses, including: The optimization objective function of the candidate selection pose designed using the mapped features and the three-dimensional wireframe points is: Among them, F rf Represents the mapped features, P i represents a 3D wireframe point, ξ * =(R * ,t * ) represents the final posture, and π is the projection operation.

8. The method according to claim 1, characterized in that The preset posture selection loss function is: Among them, l∈{1,2,3}, p l represents the probability distribution volume, represents the ground truth pose.

9. The method according to claim 1, characterized in that The preset posture optimization loss function is: Among them, the ground truth pose ρ represents the Huber robust kernel, ξ * =(R * ,t * ) represents the final posture, and π is the projection operation.

Images