A privacy protection line cloud map construction method for visual positioning

Abstract

The application discloses a privacy protection line cloud map construction method for visual positioning and belongs to the technical field of computer vision and visual positioning. The method performs local space division and small cluster merging on an original three-dimensional point map to form a local point cluster; a local reference system is independently established for each local point cluster, and the local direction of each three-dimensional point is determined through random and balanced direction label allocation; the local direction is mapped to a world coordinate system to generate a line direction, and a three-dimensional line passing through the original three-dimensional point is constructed to form a public line cloud map containing only visual descriptors and three-dimensional line associated data. The application eliminates the direction rule in a unified global framework through the local reference system, effectively reduces the risk of recovering the scene structure and content on the premise of completely hiding the original point coordinates, and retains the geometric constraint for visual positioning. The candidate local reference system can also be optimized based on an offline reference view to improve the geometric stability of pose solving.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and visual positioning technology, specifically to a privacy-preserving line cloud map construction method for visual positioning. Background Technology

[0002] Visual localization is used to estimate the camera pose of a query image within a known scene. It is widely applied in robot navigation, augmented reality, and autonomous systems, serving as a fundamental capability in these systems. Existing high-precision visual localization schemes typically employ a structured visual localization approach. This involves first reconstructing a sparse 3D point map of the scene offline using multi-view images and associating each 3D point with a corresponding visual descriptor. During the query phase, the 2D image features in the query image are matched with the visual descriptors in the map, establishing a correspondence between 2D image features and 3D points, and using this correspondence to estimate the camera pose. While this method achieves high localization accuracy, it usually relies on preserving a relatively clear scene geometry in the map.

[0003] In applications requiring shared maps to support visual positioning, directly publishing 3D point maps can easily lead to privacy risks. This is because publicly available maps typically retain explicit 3D structural information and may combine visual descriptive information to reflect the underlying scene content. Even without publishing the original images, scene geometry and appearance information can still be reconstructed, exposing sensitive content such as indoor and outdoor scene layouts. Therefore, how to reduce the amount of recoverable scene information in publicly available maps while ensuring the usability of visual positioning has become a crucial issue in privacy-preserving visual positioning.

[0004] To mitigate the risks associated with directly publishing point maps, existing technologies have proposed line cloud map representations. This involves elevating the original 3D points to 3D lines passing through those points and then publishing a line cloud map composed of visual descriptors and the 3D lines, instead of directly disclosing the 3D point coordinates. This approach hides the precise location of the original points to some extent, while still allowing pose estimation through the correspondence between 2D image features and 3D lines, thus providing a feasible approach for privacy-preserving visual localization.

[0005] However, existing line cloud map methods still have shortcomings. On the one hand, if the 3D line orientation is generated under a unified global reference framework, the published line cloud map may still retain exploitable statistical patterns, making it possible for the original 3D point distribution to be recovered, and making it difficult to fully suppress the risk of scene structure recovery. On the other hand, the orientation structure of the 3D lines also affects the geometric conditions for point-to-line pose estimation during the query phase. If the line orientation is not configured reasonably, it may lead to degradation, ill-conditioned, or unstable problems in the pose solution process. Therefore, how to construct a privacy-preserving line cloud map representation method that can reduce the scene recoverability in public maps while maintaining the geometric usability of visual positioning remains an urgent problem to be solved in the existing technology. Summary of the Invention

[0006] The technical problem this invention aims to solve is: in a map sharing scenario oriented towards visual positioning, how to construct a privacy-preserving line cloud map representation that can be used for visual positioning without directly disclosing the original 3D point coordinates, so as to reduce the risk of the scene geometry and scene content in the public map being restored.

[0007] Furthermore, the technical problem that this invention also aims to solve is: when existing line cloud map methods generate 3D line directions under a unified global reference framework, the published results may still retain usable directional patterns, which may lead to the possibility of the original 3D point distribution being recovered, and unreasonable line direction configuration may affect the stability of pose solving in the query stage.

[0008] To address this, the present invention proposes a method for constructing line cloud maps that can both reduce scene recoverability and maintain the geometric usability of visual positioning. To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a method for constructing a privacy-preserving linear cloud map for visual positioning, comprising: Obtain the original 3D point map and the visual descriptor corresponding to each 3D point in the original 3D point map; The original 3D point map is divided into local spatial segments to obtain multiple local point clusters; Establish a corresponding local reference system for each of the local point clusters; Within each of the local reference frames, a direction label is assigned to a 3D point in the corresponding local point cluster, and the local direction corresponding to each 3D point is determined from the three coordinate axis directions of the local reference frame based on the direction label. Mapping the local directions to the world coordinate system yields the line directions corresponding to each 3D point. Based on the point coordinates of each 3D point and the corresponding line direction, construct a 3D line passing through the corresponding 3D point; The visual descriptor is associated with the corresponding 3D line to form a public line cloud map. The public line cloud map only contains the association data between the visual descriptor and the 3D line, and does not contain the point coordinates of each 3D point in the original 3D point map.

[0009] In one embodiment of the present invention, the step of performing local spatial partitioning on the original three-dimensional point map to obtain multiple local point clusters includes: The original 3D point map is divided into voxels to obtain multiple initial local point clusters; Initial local point clusters with fewer than a preset point threshold are merged into the nearest local point cluster with a number of points not less than the preset point threshold, thus forming the local point cluster.

[0010] In one embodiment of the present invention, establishing a corresponding local reference system for each local point cluster includes: The mean value of the coordinates of each three-dimensional point within the local point cluster is used as the center of the local point cluster, and a corresponding local reference system is established with the center of the local point cluster as the origin. The local reference system is represented by a rotation matrix.

[0011] In one embodiment of the present invention, assigning direction labels to three-dimensional points in a corresponding local point cluster includes: For 3D points within the same local point cluster, directional labels are assigned randomly, and the difference in the number of 3D points corresponding to different directional labels within the same local point cluster does not exceed 1.

[0012] In one embodiment of the present invention, the three-dimensional line is represented using Plücker coordinates; for any three-dimensional point, its coordinates are denoted as... Its corresponding three-dimensional line Represented as: ,

[0013] in, The direction of the line corresponding to the three-dimensional point. The moment vector of this three-dimensional line, symbol This represents the cross product operation.

[0014] In one embodiment of the present invention, it further includes: For each local point cluster, multiple candidate local reference frames are pre-generated; Keeping the orientation labels unchanged, the geometric stability of the line orientation configurations corresponding to different candidate local reference systems is evaluated based on the evaluation index constructed from an offline reference view with known poses. Based on the evaluation results, candidate local reference frames are selected, and the line directions and corresponding three-dimensional lines of each three-dimensional point are re-determined based on the selected candidate local reference frames in order to update the public line cloud map. The publicly available line cloud map only includes the association results between visual descriptors and 3D lines, and does not include the local point cluster centers, the parameters of the local reference system, or the direction labels; The step of selecting candidate local reference frames based on evaluation results includes: A cluster-level iterative update method is adopted. In each update, the candidate local reference frame corresponding to a local point cluster is adjusted. The update is accepted when the overall map-level evaluation result after the update is better than that before the update, so as to output an optimized public line cloud map. The cluster-level iterative update method includes: A multiple initial value strategy is used to generate multiple different initial global configurations; Under each initial global configuration, multiple rounds of replacement and updates are performed on each local point cluster sequentially; The configuration with the highest overall map-level evaluation result is retained as the final local reference system configuration scheme; The multiple candidate local reference frames are obtained by sampling on SO(3) according to the angle dispersion criterion while keeping the center of the local point cluster unchanged.

[0015] In one embodiment of the present invention, the evaluation index constructed based on an offline reference view with a known pose, for evaluating the geometric stability of line direction configurations corresponding to different candidate local reference systems, includes: For any reference view Based on the correspondence between 2D image features and 3D lines in the reference view, the distance residual between 2D image feature points and image lines induced by the corresponding 3D lines is used as the point-to-line residual model. The Jacobian matrix of the residual with respect to the camera's six-DOF pose parameters is numerically calculated at the known camera pose in the reference view, denoted as... ; Multiple subsets containing six sets of correspondences are sampled from the correspondence between the two-dimensional image features corresponding to the reference view and the three-dimensional lines. And construct the Jacobian matrix of the corresponding subset. ; Based on the Jacobian matrix of the aforementioned subset, the following two statistics are defined:

[0016]

[0017] in, Indicate below Quantile statistics, and ; The regularization constant is greater than 0. It is the identity matrix. Represents the minimum singular value; For any reference view Its view-level evaluation score is defined as:

[0018] in, These are non-negative weights, used to balance the impact of the stability of the complete correspondence system on the stability of the minimum correspondence subset; After obtaining the view-level evaluation scores for each reference view, the global map-level evaluation objective is defined as follows:

[0019] in, This indicates the lowest score under the current configuration. A collection of reference views (0,1] is the preset scaling parameter. A weight parameter greater than 0, used to enhance the attention given to weaker reference views.

[0020] Secondly, the present invention provides a visual positioning method based on a privacy-preserving cloud map, comprising: Obtain a public line cloud map constructed using the aforementioned privacy-preserving line cloud map construction method for visual positioning; Extract two-dimensional image features from the query image; The two-dimensional image features are matched with the visual descriptors in the publicly available line cloud map; Establish the correspondence between two-dimensional image features and three-dimensional lines based on the matching results; Based on the correspondence between the two-dimensional image features and the three-dimensional lines, the camera pose corresponding to the query image is estimated.

[0021] In one embodiment of the present invention, estimating the camera pose corresponding to the query image based on the correspondence between the two-dimensional image features and the three-dimensional line includes: Construct point-to-line residuals based on the aforementioned correspondence; The camera pose corresponding to the query image is estimated by minimizing the point-to-line residual; The construction of point-to-line residuals based on the correspondence includes: For each pair of two-dimensional image features and three-dimensional lines, the image lines are induced using the Plücker coordinates of the three-dimensional lines; Calculate the distance from the two-dimensional image feature to the image line, and use it as the point-to-line residual; The step of estimating the camera pose corresponding to the query image by minimizing the point-to-line residual includes: using a robust loss function for minimization, and / or using a random sampling consistency framework to generate candidate poses and perform filtering.

[0022] Thirdly, the present invention provides a computer-readable storage medium storing publicly available line cloud map data constructed by the aforementioned privacy-preserving line cloud map construction method for visual positioning. The publicly available line cloud map data includes multiple sets of associated data, each set of associated data including a visual descriptor and parameters of a three-dimensional line, wherein the three-dimensional line passes through an original three-dimensional point, and the publicly available line cloud map data does not include the point coordinates of the original three-dimensional point.

[0023] The beneficial effects of this invention are: 1. Regarding privacy protection, this invention decomposes the original 3D point map into multiple local point clusters through local spatial partitioning, and establishes an independent local reference system for each local point cluster. This ensures that the 3D line directions within each point cluster are generated solely based on their respective local base directions. Unlike existing methods that retain exploitable directional statistical regularities when generating line directions within a unified global reference framework, the line directions of different local point clusters in this invention exhibit spatial inconsistency. From a global perspective, they lose a unified geometric regularity, significantly increasing the difficulty for attackers to recover the original 3D point distribution and scene structure information from a publicly available line cloud map. Furthermore, the publicly available line cloud map only contains the association data between visual descriptors and 3D line parameters, without disclosing any original 3D point coordinates, local point cluster centers, local reference system parameters, or direction labels. This cuts off the key geometric constraints upon which the recovery of the original point cloud depends from the information source, effectively reducing the risk of leakage of scene geometric and content privacy.

[0024] 2. Regarding the usability of visual positioning, although this invention elevates the original 3D point into a 3D line passing through that point and discloses it in the form of a line cloud, the precise location information of the original 3D point is implicitly contained in its Plücker coordinates because the 3D line is directly constructed from the coordinates of the original point and the line direction obtained by mapping through a local reference frame. After establishing a correspondence between the 2D image features of the query image and the 3D line through visual descriptors, an effective pose solution model can be constructed based on point-to-line geometric constraints. Therefore, even without disclosing the explicit point coordinates, the published line cloud map can still support high-precision camera pose estimation, achieving a balance between privacy protection and visual positioning usability, and avoiding the problem of significant performance degradation due to excessive blurring in existing privacy protection methods.

[0025] 3. Regarding positioning stability optimization, the preferred embodiment of this invention further optimizes the selection of local reference frames through offline reference views: While maintaining the direction labels unchanged, multiple candidate local reference frames are pre-generated for each local point cluster. Based on the reference view with known poses, evaluation indicators related to the stability of the geometric conditions for pose solving are established. A cluster-level iterative update method is used to search through multiple local reference frame configurations, ultimately selecting the combination that optimizes the overall map-level geometric stability. This optimization process does not change the basic representation of the publicly available line cloud map, nor does it add any additional public information. However, it effectively avoids pose solution degradation or ill-conditioned problems caused by unreasonable line direction configurations due to random selection of local reference frames. In particular, it improves positioning robustness and accuracy under difficult query conditions such as weak textures, repetitive structures, or drastic viewpoint changes. Furthermore, the iterative update mechanism based on a multi-initial-value strategy can, to some extent, avoid getting trapped in local optima, further ensuring the optimization effect. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 This is a schematic diagram illustrating how the original 3D point map in this invention is divided into local point clusters and a corresponding publicly available line cloud map is constructed. Figure 3 This is a schematic diagram comparing the existing original line cloud method with the line cloud construction mechanism based on the local reference frame of this invention; Figure 4 This is a schematic diagram illustrating the process of optimizing a local reference system based on an offline reference view according to the present invention. Figure 5 A schematic diagram illustrating the changing trend of point recovery error at different local point cluster scales; Figure 6 A schematic diagram comparing representative recovered images under different methods and different LFRD representative settings. Detailed Implementation

[0028] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0029] Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used in this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention; the terms “comprising” and “having”, and any variations thereof, in the specification, claims and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0030] In this invention, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least some embodiments of the invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this invention can be combined with other embodiments.

[0031] Example 1: This embodiment provides a privacy-preserving line cloud map construction method based on visual positioning. The following is in conjunction with... Figure 1 The overall process shown, combined with Figure 2 and Figure 3 The process by which this invention maintains visual positioning availability while protecting map privacy is described. Figure 1 This is a schematic diagram of the overall process of the present invention, used to illustrate the process of converting the original sparse three-dimensional point map into a public line cloud map without disclosing the original three-dimensional point coordinates, and realizing visual positioning based on the correspondence between two-dimensional image features and three-dimensional lines. Figure 2 This is a schematic diagram of how the original three-dimensional point map in this invention is divided into local point clusters through local spatial partitioning, and how the corresponding publicly disclosed line cloud map is further constructed. Figure 3 This diagram illustrates a comparison between existing primitive line cloud methods and the line cloud construction mechanism based on local reference frames of this invention. It demonstrates how this invention weakens global directional regularity by establishing separate local reference frames for different local point clusters. The method specifically includes the following steps: Step 1: Obtain the original 3D point map and the visual descriptor corresponding to each 3D point.

[0032] The original 3D point map can be represented as:

[0033] in, Indicates the first The coordinates of a three-dimensional point This represents the visual descriptor corresponding to the 3D point. This represents the total number of 3D points in the original 3D point map.

[0034] Step 2: Divide the original 3D point map into local spaces to obtain multiple initial local point clusters.

[0035] In this embodiment, as Figure 2 As shown, a voxel-based method is used to locally partition the 3D points in the original 3D point map. Specifically, a voxel mesh is applied to the coordinates of the 3D points, and 3D points falling into the same voxel are divided into the same initial local point cluster.

[0036] Step 3: Merge the initial local point clusters with fewer than a preset threshold to obtain the final local point cluster set.

[0037] To avoid instability in subsequent line direction construction due to excessively small local point clusters, initial local point clusters with fewer than a preset point count threshold are merged into the nearest local point cluster with a number of points not less than the preset point count threshold. After merging, the final set of local point clusters is obtained. ,in, Indicates the first The set of 3D point indices contained in a final local point cluster. This represents the final total number of local point clusters.

[0038] Step 4: Calculate the center of each local point cluster and establish the corresponding local reference frame.

[0039] For any local point cluster Its cluster center can be represented as:

[0040] in, Represents a local point cluster The number of three-dimensional points in the middle.

[0041] With local point cluster center Using the origin as the reference point, a corresponding local reference frame is established for each local point cluster. The local reference frame uses a rotation matrix. This indicates that its column vectors define the local base directions in this local reference frame. In the basic construction method of this embodiment, a rotation matrix is ​​independently sampled for each local point cluster as its local reference frame, and remains unchanged during this map construction process.

[0042] Step 5: Assign orientation labels to the 3D points in each local reference frame and determine the local orientation corresponding to each 3D point.

[0043] In this embodiment, a set of standard base directions is preset in the local reference frame. This set of standard base directions consists of the three coordinate axis directions of the local reference frame, and can be denoted as follows: For local point clusters Each three-dimensional point in Assign a direction label ,in Taken from the collection of direction labels Correspondingly, the local orientation of this 3D point in the local reference frame can be represented as:

[0044] The directional labels are randomly assigned, and preferably evenly distributed to different local base directions within the same local point cluster, so that the difference in the number of 3D points corresponding to different directional labels does not exceed 1.

[0045] Step 6: Map the local directions to the world coordinate system to obtain the line directions corresponding to each 3D point.

[0046] For local point clusters any three-dimensional point in The corresponding lower line direction in the world coordinate system can be represented as:

[0047] Therefore, 3D points in different local point clusters can generate corresponding line directions in their respective local reference frames and be mapped to a unified world coordinate system. For example... Figure 3 As shown, since different local point clusters correspond to different local reference frames, the generated line directions no longer uniformly depend on a single global reference frame, thus differing from the existing original line cloud methods that generate line directions under a single global reference frame.

[0048] Step 7: Construct a 3D line passing through the corresponding 3D point based on the 3D point coordinates and line direction.

[0049] In this embodiment, the three-dimensional line is represented using Plücker coordinates. For any three-dimensional point... Its corresponding three-dimensional It can be represented as: ,

[0050] in The direction of the line corresponding to the three-dimensional point (i.e., the unit direction vector). The moment vector of this three-dimensional line, symbol This represents the cross product operation. The 3D line constructed from this cross product passes through the original 3D point. However, they are no longer publicly available in the form of explicit point coordinates.

[0051] Step 8: Associate the visual descriptors with the corresponding 3D lines to form a public line cloud map.

[0052] For each 3D point, its visual descriptor With corresponding three-dimensional lines By linking the data, a public cloud map can be obtained:

[0053] The publicly available line cloud map is used to subsequently establish the correspondence between two-dimensional image features and three-dimensional lines to support visual positioning.

[0054] The publicly released data only includes the association results between visual descriptors and 3D lines, excluding the original 3D point coordinates, local point cluster centers, local reference system parameters, and direction labels. This embodiment converts the original 3D point map into a publicly available line cloud map representation that can be used for visual positioning. Without directly disclosing the original 3D point coordinates, it constructs 3D line directions using the local reference system corresponding to the local point clusters, thereby weakening the global directional regularity in the publicly available map and retaining the line cloud map representation used for subsequent visual positioning.

[0055] Example 2: This embodiment is a preferred implementation based on Embodiment 1. The following is in conjunction with... Figure 4 The offline optimization process for the local reference frame is explained. Figure 4 This is a flowchart illustrating the optimization of the local reference system based on offline reference views according to the present invention. It shows the processes of candidate local reference system generation, offline reference view evaluation, and cluster-level iterative updates. Compared to Embodiment 1, this embodiment, after completing the basic construction of the public line cloud map, further optimizes the local reference system offline to improve the geometric stability of the public line cloud map in subsequent visual positioning. Figure 4 As shown, the optimization process mainly includes candidate local reference frame generation, evaluation based on offline reference views, and cluster-level iterative updates, without changing the basic representation of the public line cloud map or adding additional public information.

[0056] In this embodiment, the direction label from Embodiment 1 is retained. Unchanged. For each local point cluster While keeping the center of the local point cluster unchanged, multiple candidate local reference frames are pre-sampled. To achieve more uniform coverage of the candidate space, it is preferable to... Sampling is performed according to the angular dispersion criterion. The candidate local reference frame set can be represented as:

[0057] in, Indicates a candidate local reference frame index. This represents the number of candidate local reference frames corresponding to each local point cluster. Each candidate local reference frame corresponds to a different local basis orientation configuration.

[0058] When it is the first A local point cluster selects a candidate local reference frame index. At that time, for three-dimensional points belonging to this local point cluster The optimized line direction can be represented as:

[0059] Therefore, the optimization object in this embodiment is not the direction label itself, but rather, while keeping the direction label unchanged, the line direction configuration corresponding to each local point cluster is optimized by selecting different candidate local reference frames.

[0060] To evaluate the line orientation configurations corresponding to different candidate local reference frames, this embodiment pre-selects a set of offline reference views with known camera poses, denoted as... The offline reference view is sampled once in each scene and remains fixed throughout the offline optimization process. In this embodiment, only reference views with no fewer than six sets of point-to-line correspondences are retained for subsequent evaluation. For any candidate local reference system configuration, while keeping the direction labels unchanged, the line directions of each 3D point are redefined, and a corresponding public line cloud map is formed. Then, a geometric stability score is performed based on the offline reference view.

[0061] For any reference view Based on the correspondence between 2D image features and 3D lines in the reference view, the distance residual between 2D image feature points and image lines induced by the corresponding 3D lines is used as the point-to-line residual model. The Jacobian matrix of the residual with respect to the camera's six-DOF pose parameters is numerically calculated at the known camera pose in the reference view, denoted as... To further evaluate the stability of the minimum corresponding subset, multiple subsets containing six sets of correspondences were sampled from the correspondence between the two-dimensional image features corresponding to the reference view and the three-dimensional lines. And construct the Jacobian matrix of the corresponding subset. .

[0062] Based on the subset Jacobian matrix, this embodiment defines the following two statistics:

[0063]

[0064] in, Indicate below Quantile statistics, and ; The regularization constant is greater than 0. It is the identity matrix. This represents the smallest singular value.

[0065] Furthermore, for any reference view Its view-level evaluation score is defined as:

[0066] in, These are non-negative weights used to balance the influence of the stability of the complete correspondence system and the stability of the minimum correspondence subset; the first two terms reflect the geometric conditions of the complete correspondence system, and the latter two terms reflect the stability of the weaker minimum correspondence subset.

[0067] After obtaining the view-level evaluation scores for each reference view, the global map-level evaluation objective is further defined as follows:

[0068] in, This indicates the lowest score under the current configuration. A collection of reference views (0,1] is the preset scaling parameter. A weight parameter greater than 0, used to enhance the attention given to weaker reference views.

[0069] In this embodiment, a multi-initial-value and cluster-level iterative update approach is used to optimize the candidate local reference frame. Specifically, firstly, a candidate local reference frame is selected for each local point cluster, forming multiple different initial global configurations. Then, under each initial global configuration, each local point cluster is updated sequentially in multiple rounds, attempting to replace its current candidate local reference frame with another candidate local reference frame. While keeping other local point cluster configurations unchanged, the line direction corresponding to the affected local point cluster is redefined, and the evaluation result is recalculated only for the affected reference view. If the overall map-level evaluation target after replacement is... If the current configuration is better than the previous one, then accept the update; otherwise, keep the original configuration unchanged.

[0070] By repeating the above iterative update process for each local point cluster and performing optimization under multiple initial global configurations, multiple optimized local reference system configuration results can be obtained; in each configuration result, the map-level evaluation objective is retained. The highest final configuration is used as the local reference system configuration scheme. This optimization process is an iterative optimization process performed successively on local point clusters, taking into account the common influence of different local point clusters on the overall geometric conditions while maintaining the public line cloud map representation unchanged. Using the optimized local reference system corresponding to the final configuration, the line direction of each three-dimensional point in Example 1 can be redefined, and an optimized public line cloud map can be formed.

[0071] This embodiment optimizes the local reference system offline without changing the basic representation of the publicly available line cloud map or disclosing the local reference system parameters and optimization evaluation scores. This is beneficial for improving the geometric stability of point-to-line pose estimation, and especially for enhancing the robustness of positioning under difficult query conditions.

[0072] Example 3 This embodiment provides a visual positioning method based on a privacy-preserving line cloud map. The method can be implemented based on a public line cloud map constructed in Embodiment 1, or on a public line cloud map optimized in Embodiment 2, and is used to estimate the corresponding camera pose based on a query image.

[0073] In this embodiment, the public line cloud map can be represented as:

[0074] in, Indicates the first Visual descriptors for each map element This represents the three-dimensional line associated with the visual descriptor. The direction of the line corresponding to the three-dimensional point (i.e., the unit direction vector). This represents a three-dimensional line moment vector.

[0075] The visual positioning method in this embodiment includes the following steps: Step 1: Obtain the two-dimensional image features in the query image.

[0076] Feature extraction is performed on the query image to obtain multiple two-dimensional image features. Each two-dimensional image feature contains the coordinates of a key point and the corresponding image feature descriptor.

[0077] Step 2: Match the 2D image features with the visual descriptors in the public line cloud map.

[0078] By matching image feature descriptors in a 2D image with visual descriptors in a public line cloud map, the matching results between query image features and map elements are obtained, thereby establishing a correspondence between 2D image features and 3D lines. This correspondence can be expressed as:

[0079] in, Indicates the first Two-dimensional coordinates of a query image feature. This represents the index of the map element that matches the features of the queried image. This indicates the number of matching relationships obtained.

[0080] Correspondingly, the set of correspondences between two-dimensional image features and three-dimensional lines can be obtained:

[0081] Step 3: Construct point-to-line residuals based on the correspondence between two-dimensional image features and three-dimensional lines.

[0082] In this embodiment, it is assumed that the camera has been calibrated and that normalized image coordinates and a pose representation from the world coordinate system to the camera coordinate system are used. Let the camera pose be... ,in, Represents the camera rotation matrix. This represents the camera translation vector. For any two-dimensional image feature... Its homogeneous coordinates can be expressed as:

[0083] Candidate camera pose Below, the three-dimensional lines corresponding to the features of the queried image. The corresponding image line can be induced on the image plane, and its expression is:

[0084] in, Represents three-dimensional lines The induced image lines, This represents the cross product operation.

[0085] set up:

[0086] The corresponding point-to-line residual can then be expressed as:

[0087] The above normalization form requires that the induced image lines satisfy... If the corresponding 3D line passes through the center of the camera but cannot form a valid image line, then this degradation case will not be included in the subsequent solution.

[0088] Step 4: Estimate the camera pose corresponding to the query image based on the point-to-line residual.

[0089] In this embodiment, the camera pose corresponding to the query image is estimated by minimizing the point-to-line residuals of all correspondences. The optimization objective can be expressed as:

[0090] in, This represents the robust loss function.

[0091] In practical implementation, candidate poses can be generated using the minimum six-group correspondence within the p6L-based random sampling consistency framework. The optimal pose results are then selected by combining the optimization objective with nonlinear optimization to refine the selected poses, thereby obtaining the final camera pose estimation result corresponding to the query image.

[0092] Through the above steps, this embodiment utilizes the publicly available line cloud map obtained in Embodiment 1 or Embodiment 2 to establish the correspondence between the two-dimensional image features of the query image and the three-dimensional line without disclosing the original three-dimensional point coordinates, and completes camera pose estimation, thereby achieving visual positioning based on the privacy-preserving line cloud map.

[0093] In some embodiments, the present invention also provides a computer-readable storage medium storing publicly available line cloud map data constructed by the aforementioned privacy-preserving line cloud map construction method for visual positioning. The publicly available line cloud map data includes multiple sets of associated data, each set of associated data including a visual descriptor and parameters of a three-dimensional line, wherein the three-dimensional line passes through an original three-dimensional point, and the publicly available line cloud map data does not include the point coordinates of the original three-dimensional point.

[0094] The effects of this invention can be further illustrated by the following experiments.

[0095] The following experimental results are only used to illustrate the effectiveness of the present invention and are not intended to limit the scope of protection of the present invention. Experiments were conducted on the indoor scene dataset Energy Landscape and the outdoor scene dataset Cambridge Landmarks, and compared with the original point cloud public method PC (Point Cloud), the original line cloud method OLC (Original Line Cloud), and the coordinate permutation method CP (Coordinate Permutation). The method of the present invention is denoted as LFRD (Local-Frame Randomized Direction Field). Here, PC represents the comparison method of directly publicizing the original 3D point cloud, serving as a reference upper bound for non-privacy public schemes; OLC represents the original line cloud comparison method; and CP represents the comparison method based on coordinate permutation. For the present invention, the local direction dictionary size is K=3, and the minimum local point cluster size is... .

[0096] First, the effectiveness of this invention in terms of positioning accuracy and online operation cost is verified. The positioning results are calculated using median rotation error, The evaluation was conducted using the area under the curve (AUC) at the threshold, the median shift error, the AUC at the 10cm threshold, and the online runtime for a single query.

[0097] Table 1. Comparison of localization results and running time of each method on the Energy Landscape dataset.

[0098] Table 2. Comparison of localization results and running time of each method on the Cambridge Landmarks dataset.

[0099] As shown in Tables 1 and 2, in indoor scenarios, the present invention maintains a similar level of positioning performance to OLC, and its online runtime for a single query is comparable to OLC, significantly lower than that of CP. In outdoor scenarios, the median rotation error of the present invention is better than both OLC and CP, and its online runtime is still significantly lower than that of CP. This indicates that the present invention maintains the availability of visual positioning without relying on the costly online recovery process required by CP.

[0100] Furthermore, the impact of local point cluster scale on the difficulty of point reconstruction was verified. For example... Figure 5 As shown, Figure 5 This diagram illustrates the trend of point recovery error variation at different local point cluster scales. It shows how the point recovery error of a public line cloud map changes from small clusters to transitional regions and then to large clusters as the local point cluster scale changes, and also demonstrates the impact of local point cluster scale on the difficulty of point recovery. As the local point cluster scale increases, the point recovery error shows a trend of changing from small clusters to transitional regions and then to large clusters. After the transitional region, the recovery error increases significantly, indicating that this invention, through local point cluster division and local reference frame orientation field construction, can effectively increase the difficulty of point recovery based on public line cloud maps.

[0101] To verify the effect of the offline optimization strategy in Example 2 on improving the stability of localization under difficult query conditions, the 12 query samples with the largest error of the basic LFRD method were selected as the difficult query subset H12 in two representative scenarios, and the half of the most difficult ones were selected as H6.

[0102] Table 3. Comparison of mean pose error between the basic LFRD and the optimized LFRD on the difficult query subset.

[0103] As shown in Table 3, after the offline optimization described in Example 2, both rotation and translation errors were reduced under difficult query conditions. Specifically, in the H6 subset of the Cambridge scene, the average translation error decreased from 23.71 cm to 12.68 cm, indicating that evaluating and selecting candidate local reference frames based on offline reference views can effectively improve the geometric stability of point-to-line pose estimation, and this improvement is more pronounced under difficult query conditions.

[0104] Furthermore, the effectiveness of this invention in scene content restoration and suppression was verified. The peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and perceptual difference index (LPIPS) between the restored image and the original image were used for evaluation. Lower PSNR and SSIM, and higher LPIPS, indicate that the scene image restored from the public map is more difficult to recognize. Table 4 presents the comparison results of restored image quality under three representative settings: OLC, CP, and this invention: LFRD-small, LFRD-post, and LFRD-large. LFRD-small, LFRD-post, and LFRD-large represent the restoration quality under the following settings: LFRD-small, LFRD-post, and LFRD-large. Figure 5 The local point cluster scale variation trend is shown by selecting representative settings for small cluster regions, transition regions, and large cluster regions.

[0105] Table 4. Comparison of restored image quality under three representative settings: OLC, CP, and the present invention, including LFRD-small, LFRD-post, and LFRD-large.

[0106] like Figure 6 As shown, Figure 6 This diagram illustrates the comparison of representative restored images under different methods and LFRD representative settings, showcasing the differences in discernibility of restored images from the original line cloud method, coordinate permutation method, and the present invention under different local point cluster scale representative settings. This demonstrates the suppressive effect of the present invention on scene content recoverability. In different representative scenes, the discernibility of the restored images from the present invention is significantly weaker than that from OLC and CP. Table 4 shows that in the EnergyLandscape dataset, the LPIPS of the three representative settings of the present invention are higher than those of OLC, and the LPIPS of LFRD-small, LFRD-post, and LFRD-large are also higher than those of CP. In the Cambridge Landmarks dataset, the LPIPS of LFRD-post and LFRD-large are higher than those of CP. This indicates that the present invention can effectively reduce the ability to recover scene content from publicly available line cloud maps, thereby enhancing map privacy protection.

[0107] In summary, the privacy-preserving line cloud map construction method for visual positioning provided by this invention is based on a technical approach of "local spatial partitioning - local reference system establishment - direction label allocation - line direction mapping". This method elevates the original 3D points into 3D lines associated only by visual descriptors, and replaces the explicit point cloud publication with a public line cloud map containing only the descriptors and 3D line parameters. By introducing a local reference system independently defined based on local point clusters, this method breaks the directional regularity brought about by the unified global reference framework in existing line cloud representations. It fundamentally weakens the statistical correlation between line directions and the spatial distribution of the original 3D points in the public map, making it difficult for attackers to recover scene structure and content using global geometric constraints, thus significantly improving privacy protection. Simultaneously, each 3D line is constructed from the original point coordinates and the corresponding line direction. The geometric information implicit in its Plücker coordinates can directly support the point-to-line residual construction and pose solving during the query phase. Even with completely hidden point coordinates, the geometric constraints required for visual positioning are still preserved, achieving an effective balance between privacy protection and positioning usability. Furthermore, in the preferred embodiment, this invention introduces a candidate local reference frame evaluation and cluster-level iterative optimization mechanism based on offline reference views. Without altering the public map representation or adding any additional public information, it automatically searches for the local reference frame configuration most favorable to the geometric conditions of pose solving. This makes the line direction distribution more conducive to stable positioning, particularly improving the robustness and accuracy of positioning in challenging scenarios such as weak textures, repetitive structures, and large viewpoint differences. Therefore, this invention, from the map representation construction mechanism to the optimization strategy, forms a complete privacy-preserving line cloud map generation scheme. It possesses both high security protection capabilities and maintains efficient and reliable visual positioning performance in complex environments, demonstrating broad application prospects.

[0108] Although the invention has been described with reference to preferred embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, the technical features mentioned in the various embodiments can be combined in any manner as long as there is no structural conflict. The invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A privacy-preserving linear cloud map construction method for visual positioning, characterized in that, include: Obtain the original 3D point map and the visual descriptor corresponding to each 3D point in the original 3D point map; The original 3D point map is divided into local spatial segments to obtain multiple local point clusters; Establish a corresponding local reference system for each of the local point clusters; Within each of the local reference frames, a direction label is assigned to a 3D point in the corresponding local point cluster, and the local direction corresponding to each 3D point is determined from the three coordinate axis directions of the local reference frame based on the direction label. Mapping the local directions to the world coordinate system yields the line directions corresponding to each 3D point. Based on the point coordinates of each 3D point and the corresponding line direction, construct a 3D line passing through the corresponding 3D point; The visual descriptor is associated with the corresponding 3D line to form a public line cloud map. The public line cloud map only contains the association data between the visual descriptor and the 3D line, and does not contain the point coordinates of each 3D point in the original 3D point map.

2. The method according to claim 1, characterized in that, The original 3D point map is divided into local spatial regions to obtain multiple local point clusters, including: The original 3D point map is divided into voxels to obtain multiple initial local point clusters; Initial local point clusters with fewer than a preset point threshold are merged into the nearest local point cluster with a number of points not less than the preset point threshold, thus forming the local point cluster.

3. The method according to claim 1, characterized in that, The step of establishing a corresponding local reference frame for each local point cluster includes: The mean value of the coordinates of each three-dimensional point within the local point cluster is used as the center of the local point cluster, and a corresponding local reference system is established with the center of the local point cluster as the origin. The local reference system is represented by a rotation matrix.

4. The method according to claim 1, characterized in that, The process of assigning directional labels to 3D points in a corresponding local point cluster includes: For 3D points within the same local point cluster, directional labels are assigned randomly, and the difference in the number of 3D points corresponding to different directional labels within the same local point cluster does not exceed 1.

5. The method according to claim 1, characterized in that, The three-dimensional lines are represented using Plücker coordinates; for any three-dimensional point, its coordinates are denoted as... Its corresponding three-dimensional line Represented as: ， in, The direction of the line corresponding to the three-dimensional point. The moment vector of this three-dimensional line, symbol This represents the cross product operation.

6. The method according to any one of claims 1 to 5, characterized in that, Also includes: For each local point cluster, multiple candidate local reference frames are pre-generated; Keeping the orientation labels unchanged, the geometric stability of the line orientation configurations corresponding to different candidate local reference systems is evaluated based on the evaluation index constructed from an offline reference view with known poses. Based on the evaluation results, candidate local reference frames are selected, and the line directions and corresponding three-dimensional lines of each three-dimensional point are re-determined based on the selected candidate local reference frames in order to update the public line cloud map. The publicly available line cloud map only includes the association results between visual descriptors and 3D lines, and does not include the local point cluster centers, the parameters of the local reference system, or the direction labels; The step of selecting candidate local reference frames based on evaluation results includes: A cluster-level iterative update method is adopted. In each update, the candidate local reference frame corresponding to a local point cluster is adjusted. The update is accepted when the overall map-level evaluation result after the update is better than that before the update, so as to output an optimized public line cloud map. The cluster-level iterative update method includes: A multiple initial value strategy is used to generate multiple different initial global configurations; Under each initial global configuration, multiple rounds of replacement and updates are performed on each local point cluster sequentially; The configuration with the highest overall map-level evaluation result is retained as the final local reference system configuration scheme; The multiple candidate local reference frames are obtained by sampling on SO(3) according to the angle dispersion criterion while keeping the center of the local point cluster unchanged.

7. The method according to claim 6, characterized in that, The evaluation index, constructed based on an offline reference view with a known pose, evaluates the geometric stability of line direction configurations corresponding to different candidate local reference frames, including: For any reference view Based on the correspondence between 2D image features and 3D lines in the reference view, the distance residual between 2D image feature points and image lines induced by the corresponding 3D lines is used as the point-to-line residual model. The Jacobian matrix of the residual with respect to the camera's six-DOF pose parameters is numerically calculated at the known camera pose in the reference view, denoted as... ; Multiple subsets containing six sets of correspondences are sampled from the correspondence between the two-dimensional image features corresponding to the reference view and the three-dimensional lines. And construct the Jacobian matrix of the corresponding subset. ; Based on the Jacobian matrix of the aforementioned subset, the following two statistics are defined: in, Indicate below Quantile statistics, and ; The regularization constant is greater than 0. It is the identity matrix. Represents the minimum singular value; For any reference view Its view-level evaluation score is defined as: in, These are non-negative weights, used to balance the impact of the stability of the complete correspondence system on the stability of the minimum correspondence subset; After obtaining the view-level evaluation scores for each reference view, the global map-level evaluation objective is defined as follows: in, This indicates the lowest score under the current configuration. A collection of reference views (0,1] is the preset scaling parameter. A weight parameter greater than 0, used to enhance the attention given to weaker reference views.

8. A visual positioning method based on a privacy-preserving cloud map, characterized in that, include: Obtain a publicly available line cloud map constructed using the method of any one of claims 1 to 7; Extract two-dimensional image features from the query image; The two-dimensional image features are matched with the visual descriptors in the publicly available line cloud map; Establish the correspondence between two-dimensional image features and three-dimensional lines based on the matching results; Based on the correspondence between the two-dimensional image features and the three-dimensional lines, the camera pose corresponding to the query image is estimated.

9. The method according to claim 8, characterized in that, The step of estimating the camera pose corresponding to the query image based on the correspondence between the two-dimensional image features and the three-dimensional lines includes: Construct point-to-line residuals based on the aforementioned correspondence; The camera pose corresponding to the query image is estimated by minimizing the point-to-line residual; The construction of point-to-line residuals based on the correspondence includes: For each pair of two-dimensional image features and three-dimensional lines, the image lines are induced using the Plücker coordinates of the three-dimensional lines; Calculate the distance from the two-dimensional image feature to the image line, and use it as the point-to-line residual; The step of estimating the camera pose corresponding to the query image by minimizing the point-to-line residual includes: using a robust loss function for minimization, and / or using a random sampling consistency framework to generate candidate poses and perform filtering.

10. A computer-readable storage medium storing thereon publicly available line cloud map data constructed by the method of any one of claims 1 to 7, characterized in that, The publicly available line cloud map data contains multiple sets of associated data. Each set of associated data contains a visual descriptor and parameters of a 3D line. The 3D line passes through an original 3D point, but the publicly available line cloud map data does not contain the point coordinates of the original 3D point.

Images