Ground map point cloud incremental construction method and device based on remote sensing image guidance

Abstract

This application discloses a method and apparatus for incremental construction of ground map point clouds based on remote sensing images. The method includes: extracting features from local point cloud data of a target to obtain feature vectors and fitting them; obtaining the relative acquisition path of the local point cloud based on the fitting results; cross-optimizing the features of the remote sensing image and the local point cloud; obtaining relative path coefficients and cross paths based on the cross-optimization results for weighted processing; performing local reprojection on the remote sensing image; generating an initial pose for local point cloud registration based on the projection results; optimizing the initial pose; constructing a point cloud local pose relationship with temporal information based on the optimization results; and performing geometric spatial transformation on the local point cloud data of the target to obtain the updated incremental point cloud of the target ground map. This solves the problem in related technologies where data loss from absolute and relative positioning devices can easily occur, thus reducing the accuracy of incremental point cloud construction.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method and apparatus for incremental construction of ground map point clouds based on remote sensing images. Background Technology

[0002] Traditional point cloud registration is mainly used to align point cloud data acquired from different perspectives or at different times. Point cloud data is a set of points in three-dimensional space, usually acquired by devices such as LiDAR and depth cameras. The purpose of point cloud registration is to accurately match corresponding points between different point clouds, thereby achieving alignment.

[0003] In related technologies, for incremental construction of map point clouds, in addition to the registration of the point cloud data itself, the absolute position information of the newly added map point cloud data can be provided by the Global Positioning System as a positioning aid for the incremental point cloud. At the same time, direction information can be provided by relative positioning devices such as inertial navigation as a rotation aid for the incremental point cloud.

[0004] However, in harsh or high-speed environments, the relevant technologies are prone to intermittent data loss in absolute and relative positioning devices, leading to failure or large errors in the incremental construction of ground map point clouds in real-world environments, thereby reducing the accuracy of incremental point cloud construction, which urgently needs to be addressed. Summary of the Invention

[0005] This application provides a method and apparatus for incremental construction of ground map point clouds based on remote sensing images, in order to solve the problem in related technologies that intermittent data loss of absolute and relative positioning devices is easily caused in harsh or high-speed environments, resulting in failure or large errors in incremental construction of ground map point clouds in real environments, thereby reducing the accuracy of incremental point cloud construction.

[0006] The first aspect of this application provides a method for incremental construction of ground map point clouds based on remote sensing images, comprising the following steps: acquiring point cloud data of a target ground map and a remote sensing image of the target ground map; collecting target local point cloud data with temporal information within a preset time period; extracting at least one feature from the target local point cloud data, and obtaining at least one feature vector based on the at least one feature; fitting the at least one feature vector to obtain a fitting result; obtaining a relative acquisition path of the target local point cloud based on the fitting result; and based on the relative acquisition path, performing cross-optimization on a first feature of the remote sensing image and a second feature of the target local point cloud to obtain a cross-optimization result, and then... The cross-optimization results yield relative path coefficients and cross paths. These relative path coefficients and cross paths are then weighted to obtain incremental point cloud acquisition paths. Based on these incremental point cloud acquisition paths, the remote sensing image is locally reprojected to obtain projection results. An initial pose for local point cloud registration is generated based on these projection results, and the initial pose is optimized to obtain optimized results. A point cloud local pose relationship with temporal information is constructed based on these optimized results. The point cloud local pose relationship is then geometrically transformed with the target local point cloud data to obtain transformation results. The point cloud data of the target ground map is updated based on these transformation results to obtain the updated target ground map point cloud increment.

[0007] Optionally, in one embodiment of this application, the step of extracting at least one feature of the target local point cloud data, obtaining at least one feature vector based on the at least one feature, fitting the at least one feature vector to obtain a fitting result, and obtaining the relative acquisition path of the target local point cloud based on the fitting result includes: extracting at least one feature of the target local point cloud data, obtaining at least one feature vector based on the at least one feature of the target local point cloud data; transforming the at least one feature vector based on MLP (Multilayer Perceptron) to obtain at least one matrix parameter; and concatenating the temporal information with the at least one matrix parameter to obtain the relative acquisition path fitted based on the temporal information.

[0008] Optionally, in one embodiment of this application, the step of cross-optimizing the first feature of the remote sensing image and the second feature of the target local point cloud based on the relative acquisition path to obtain a cross-optimization result, and obtaining relative path coefficients and cross paths according to the cross-optimization result, and weighting the relative path coefficients and cross paths to obtain an incremental point cloud acquisition path includes: extracting at least one feature of the remote sensing image and obtaining at least one remote sensing feature vector based on the at least one feature of the remote sensing image; performing local feature extraction on the point cloud data of the target ground map to obtain at least one point cloud feature vector; expanding and copying the at least one remote sensing feature vector and the at least one point cloud feature vector to concatenate with the at least one feature vector to obtain at least one feature matrix;

[0009] Based on a multilayer perceptron (MLP), the at least one feature matrix is ​​transformed to obtain at least one matrix parameter, relative path coefficient, and cross path; the relative path coefficient and the cross path are weighted and superimposed to obtain the optimized incremental point cloud acquisition path.

[0010] Optionally, in one embodiment of this application, the step of performing local reprojection on the remote sensing image based on the incremental point cloud acquisition path to obtain a projection result, generating an initial pose for the local point cloud registration based on the projection result, and optimizing the initial pose to obtain an optimized result, and constructing a point cloud local pose relationship with temporal information based on the optimized result includes: cropping corresponding points in the remote sensing image into a preset block of remote sensing image to obtain at least one remote sensing image block; performing projection transformation on the at least one remote sensing image block to obtain at least one projected ground point cloud corresponding to the remote sensing image block; constructing a system of equations for local geometric transformation using the at least one projected ground point cloud and the target local point cloud; solving the system of equations using the three-dimensional spatial coordinates of the target point between the projected ground point cloud and the target local point cloud to obtain a solution result; and combining the target ground map point cloud into an incremental point cloud as a whole, obtaining a geometric transformation relationship through a rotation matrix and a translation vector sequence to generate a point cloud local pose.

[0011] Optionally, in one embodiment of this application, the step of performing a geometrical spatial transformation on the local pose relationship of the point cloud and the target local point cloud data to obtain a transformation result, and updating the point cloud data of the target ground map according to the transformation result to obtain the updated target ground map point cloud increment includes: performing the geometrical spatial transformation on the target local point cloud based on the rotation matrix and the translation vector to obtain the transformation result, and stitching the transformation result onto the target ground map point cloud model to obtain the updated target ground map point cloud increment.

[0012] A second aspect of this application provides a ground map point cloud incremental construction device based on remote sensing image guidance, comprising: an acquisition module, configured to acquire point cloud data of a target ground map and a remote sensing image of the target ground map, and collect target local point cloud data with temporal information within a preset time period; a determination module, configured to extract at least one feature of the target local point cloud data, obtain at least one feature vector based on the at least one feature, fit the at least one feature vector to obtain a fitting result, and obtain a relative acquisition path of the target local point cloud based on the fitting result; and a first processing module, configured to perform cross-optimization on a first feature of the remote sensing image and a second feature of the target local point cloud based on the relative acquisition path, obtain a cross-optimization result, and determine the relative acquisition path of the target local point cloud based on the relative acquisition path. The first processing module obtains relative path coefficients and cross paths from the cross optimization results, and then weights the relative path coefficients and cross paths to obtain an incremental point cloud acquisition path. The second processing module performs local reprojection on the remote sensing image based on the incremental point cloud acquisition path to obtain a projection result, generates an initial pose for the local point cloud registration based on the projection result, and optimizes the initial pose to obtain an optimized result, so as to construct a point cloud local pose relationship with temporal information based on the optimized result. The third processing module performs geometric space transformation on the point cloud local pose relationship and the target local point cloud data to obtain a transformation result, and updates the point cloud data of the target ground map based on the transformation result to obtain an updated target ground map point cloud increment.

[0013] Optionally, in one embodiment of this application, the determining module includes: a first extraction unit, configured to extract at least one feature of the target local point cloud data and obtain at least one feature vector based on the at least one feature of the target local point cloud data; a first acquisition unit, configured to transform the at least one feature vector based on a multilayer perceptron (MLP) to obtain at least one matrix parameter; and a first determining unit, configured to concatenate the time-series information with the at least one matrix parameter to obtain the relative acquisition path fitted based on the time-series information.

[0014] Optionally, in one embodiment of this application, the first processing module includes: a second extraction unit, configured to extract at least one feature of the remote sensing image and obtain at least one remote sensing feature vector based on the at least one feature of the remote sensing image; a second acquisition unit, configured to perform local feature extraction on point cloud data of the target ground map to obtain at least one point cloud feature vector; a first processing unit, configured to expand and copy the at least one remote sensing feature vector and the at least one point cloud feature vector to concatenate with the at least one feature vector to obtain at least one feature matrix; a third acquisition unit, configured to transform the at least one feature matrix based on a multilayer perceptron (MLP) to obtain at least one matrix parameter, relative path coefficient, and cross path; and a fourth acquisition unit, configured to weight and superimpose the relative path coefficient and the cross path to obtain the optimized incremental point cloud acquisition path.

[0015] Optionally, in one embodiment of this application, the second processing module includes: a second processing unit, configured to extract a preset block of remote sensing image from corresponding points in the remote sensing image to obtain at least one remote sensing image block, and to perform projection transformation on the at least one remote sensing image block to obtain at least one projected ground point cloud corresponding to the remote sensing image block; a third processing unit, configured to construct a system of equations for local geometric transformation using the at least one projected ground point cloud and the target local point cloud, and to solve the system of equations using the three-dimensional spatial coordinates of the target point between the projected ground point cloud and the target local point cloud to obtain a solution result; and a generation unit, configured to combine the target ground map point cloud into an incremental point cloud based on the solution result, and to obtain a geometric transformation relationship through a rotation matrix and a translation vector sequence to generate a local pose of the point cloud.

[0016] Optionally, in one embodiment of this application, the updating module includes: a second determining unit, configured to perform the geometric space transformation on the target local point cloud based on the rotation matrix and the translation vector, obtain the transformation result, and stitch the transformation result onto the target ground map point cloud model to obtain the updated target ground map point cloud increment.

[0017] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for incremental construction of ground map point clouds based on remote sensing images as described in the above embodiments.

[0018] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for incremental construction of ground map point clouds guided by remote sensing images.

[0019] This application embodiment can acquire point cloud data of a target ground map and corresponding remote sensing images, collect target local point cloud data with temporal information over a certain period of time, extract features from the target local point cloud data to obtain feature vectors and fit them, obtain the relative acquisition path of the target local point cloud based on the fitting results, and then perform cross-optimization on the features of the remote sensing image and the features of the target local point cloud, and obtain relative path coefficients and cross paths based on the cross-optimization results for weighted processing, and then perform local reprojection on the remote sensing image, generate the initial pose for local point cloud registration based on the projection results, optimize the initial pose, construct the point cloud local pose relationship with temporal information based on the optimization results, and perform geometric space transformation on the target local point cloud data to obtain the updated target ground map point cloud increment, thereby effectively reducing the error of ground map point cloud increment construction in real environment and improving the accuracy of point cloud increment construction. Therefore, it solves the problem in related technologies where intermittent data loss of absolute and relative positioning devices can easily occur in harsh or high-speed environments, leading to failure or large errors in ground map point cloud increment construction in real environment, thus reducing the accuracy of point cloud increment construction.

[0020] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0021] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0022] Figure 1 This is a flowchart of a method for incremental construction of ground map point clouds based on remote sensing images, according to an embodiment of this application.

[0023] Figure 2 This is a schematic diagram of a ground map point cloud incremental construction device based on remote sensing image guidance provided in an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0025] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0026] The following describes, with reference to the accompanying drawings, a method and apparatus for incremental construction of ground map point clouds based on remote sensing images, according to embodiments of this application. Addressing the problem mentioned in the background art that, in harsh or high-speed environments, intermittent data loss in absolute and relative positioning equipment can easily occur, leading to failure or significant errors in incremental construction of ground map point clouds in real-world environments, thus reducing the accuracy of incremental point cloud construction, this application provides a method for incremental construction of ground map point clouds based on remote sensing images. In this method, point cloud data of a target ground map and corresponding remote sensing images can be acquired, target local point cloud data with temporal information can be collected within a certain time period, and features of the target local point cloud data can be extracted to obtain feature vectors and fitted. Based on the fitting results, the target local point cloud data can be obtained. The relative acquisition path of the local point cloud is used to cross-optimize the features of the remote sensing image and the features of the target local point cloud. Based on the cross-optimization results, relative path coefficients and cross paths are obtained for weighted processing, followed by local reprojection of the remote sensing image. Based on the projection results, an initial pose for local point cloud registration is generated and optimized. A point cloud local pose relationship with temporal information is constructed based on the optimization results. The target local point cloud data is then geometrically transformed to obtain an updated target ground map point cloud increment. This effectively reduces the error in ground map point cloud increment construction in real-world environments and improves the accuracy of point cloud increment construction. Therefore, this solves the problem in related technologies where intermittent data loss from absolute and relative positioning devices can easily occur in harsh or high-speed environments, leading to failure or large errors in ground map point cloud increment construction in real-world environments, thus reducing the accuracy of point cloud increment construction.

[0027] Specifically, Figure 1 This is a flowchart illustrating a method for incremental construction of ground map point clouds based on remote sensing images, provided in an embodiment of this application.

[0028] like Figure 1 As shown, the method for incremental construction of ground map point clouds based on remote sensing images includes the following steps:

[0029] In step S101, point cloud data of the target ground map and remote sensing images of the target ground map are acquired, and local point cloud data of the target with temporal information within a preset time period are collected.

[0030] It is understood that the embodiments of this application can acquire point cloud data of the target ground map and remote sensing images of the target ground map. For example, it can extract existing complete large-scale ground map point cloud data and corresponding remote sensing images of the region from the global positioning system, and collect several local point cloud data with time-series information within a certain period of time, thereby effectively improving the executability of incremental construction of ground map point cloud.

[0031] It should be noted that the preset time is set by those skilled in the art according to the actual situation, and no specific limitation is made here.

[0032] In step S102, at least one feature of the target local point cloud data is extracted, and at least one feature vector is obtained based on the at least one feature. The at least one feature vector is fitted to obtain the fitting result, and the relative acquisition path of the target local point cloud is obtained based on the fitting result.

[0033] It is understood that the embodiments of this application can extract at least one feature of the target local point cloud data in the following steps, obtain at least one feature vector based on at least one feature, fit at least one feature vector to obtain a fitting result, and obtain the relative acquisition path of the target local point cloud based on the fitting result, thereby effectively improving the robustness of incremental construction of ground map point cloud.

[0034] In one embodiment of this application, extracting at least one feature from the target local point cloud data and obtaining at least one feature vector based on the at least one feature, fitting the at least one feature vector to obtain a fitting result, and obtaining the relative acquisition path of the target local point cloud based on the fitting result includes: extracting at least one feature from the target local point cloud data and obtaining at least one feature vector based on the at least one feature of the target local point cloud data; transforming the at least one feature vector based on a multilayer perceptron (MLP) to obtain at least one matrix parameter; and concatenating temporal information with the at least one matrix parameter to obtain a relative acquisition path fitted based on temporal information.

[0035] In actual implementation, this embodiment of the application can input a complete map point cloud model P0 at time t0, with a size of n0×3, where n0 represents the number of points in the complete ground map point cloud. Then, it can input several local point cloud data p1, p2, ..., p1 from consecutive time sequences after time t0. k And record the temporal information t1, t2, ..., t corresponding to the collected local point cloud data. k The sizes of the local point cloud data are n1×3, n2×3, ..., n k ×3, where n1, n2, ..., n k Both represent the number of points in a local point cloud.

[0036] Furthermore, embodiments of this application can select a remote sensing image S covering a complete map point cloud from a pre-published remote sensing image database, and can perform feature extraction on several local point cloud data, classifying each local point cloud p1, p2, ..., p k Extract into a complete feature vector f1, f2, ..., f kEach feature vector has a size of 1×C, and the feature vectors are transformed based on MLP to finally obtain k matrix parameters of size 1×2. Each 1×2 binary represents the relative position of the local point cloud in the remote sensing image S, and the temporal information t1, t2, ..., t... k By concatenating the k×2 matrix parameters, the relative acquisition path L1, which is a k×3 matrix parameter, is obtained based on the fitting of time-series information, thereby effectively improving the accuracy of incremental construction of ground map point clouds.

[0037] In step S103, based on the relative acquisition path, the first feature of the remote sensing image and the second feature of the target local point cloud are cross-optimized to obtain the cross-optimization result. The relative path coefficient and cross path are obtained according to the cross-optimization result. The relative path coefficient and cross path are weighted to obtain the incremental point cloud acquisition path.

[0038] It is understood that, based on the relative acquisition path in the following steps, the first feature of the remote sensing image and the second feature of the target local point cloud can be cross-optimized to obtain the cross-optimization result. The relative path coefficient and cross path are obtained according to the cross-optimization result. The relative path coefficient and cross path are weighted to obtain the incremental point cloud acquisition path, thereby effectively improving the accuracy of incremental construction of ground map point cloud.

[0039] In one embodiment of this application, based on a relative acquisition path, a first feature of a remote sensing image and a second feature of a target local point cloud are cross-optimized to obtain a cross-optimization result. Relative path coefficients and cross paths are obtained based on the cross-optimization result. The relative path coefficients and cross paths are then weighted to obtain an incremental point cloud acquisition path. This includes: extracting at least one feature from a remote sensing image and obtaining at least one remote sensing feature vector based on that feature; extracting local features from the point cloud data of the target ground map to obtain at least one point cloud feature vector; expanding and copying the at least one remote sensing feature vector and the at least one point cloud feature vector to concatenate them with the at least one feature vector to obtain at least one feature matrix; transforming the at least one feature matrix based on a multilayer perceptron (MLP) to obtain at least one matrix parameter, relative path coefficients, and cross paths; and weighting and superimposing the relative path coefficients and cross paths to obtain the optimized incremental point cloud acquisition path.

[0040] As one possible implementation, embodiments of this application can perform feature extraction on the remote sensing image S to obtain a feature vector F of size 1×C. S Local feature extraction is performed on the ground map point cloud P0 to obtain a feature matrix F0 of size 1×C. The remote sensing feature vector F is then... sThe point cloud feature vector F0 is expanded and copied, and then concatenated with the feature vector group f1, f2, ..., f k Then, k feature vectors of size 1×3C are obtained. The feature matrix of size k×3C is transformed using MLP to obtain matrix parameters of size k×3, where the first column element k×1 and the last two column elements k×2 are used as relative path coefficients K. S And intersecting path L2, perform relative path coefficient K on relative path L1 and intersecting path L2. S The weighted summation yields the final optimized relative path L3, i.e., the incremental point cloud acquisition path. The specific calculation formula is as follows:

[0041] L3 = K S L1+(1-K S L2

[0042] Where L3 is the optimized relative path, K S L1 represents the relative path coefficient, L2 represents the relative path, and L3 represents the intersection path.

[0043] In step S104, based on the incremental point cloud acquisition path, the remote sensing image is locally reprojected to obtain the projection result, and an initial pose for local point cloud registration is generated based on the projection result. The initial pose is then optimized to obtain the optimized result, so as to construct a point cloud local pose relationship with temporal information based on the optimized result.

[0044] It is understood that, based on the incremental point cloud acquisition path in the following steps, the remote sensing image can be locally reprojected to obtain the projection result, and an initial pose for local point cloud registration can be generated based on the projection result. The initial pose can be optimized to obtain an optimized result, so as to construct a point cloud local pose relationship with temporal information based on the optimized result, thereby effectively improving the accuracy of incremental construction of ground map point cloud.

[0045] Optionally, in one embodiment of this application, based on the incremental point cloud acquisition path, the remote sensing image is locally reprojected to obtain the projection result, and an initial pose for local point cloud registration is generated based on the projection result. The initial pose is then optimized to obtain an optimized result. The construction of a point cloud local pose relationship with temporal information based on the optimized result includes: cropping corresponding points in the remote sensing image into a preset block of remote sensing image to obtain at least one remote sensing image block; performing projection transformation on the at least one remote sensing image block to obtain at least one projected ground point cloud corresponding to the remote sensing image block; constructing a system of equations for local geometric transformation using at least one projected ground point cloud and the target local point cloud; solving the system of equations using the three-dimensional spatial coordinates of the target point between the projected ground point cloud and the target local point cloud to obtain a solution result; and combining the target ground map point cloud into an incremental point cloud as a whole, obtaining the geometric transformation relationship through a rotation matrix and a translation vector sequence to generate the point cloud local pose.

[0046] In some embodiments, this application can extract small patches of remote sensing image data from corresponding points in the remote sensing image S based on the optimized relative path L3 in the remote sensing image S. The extracted remote sensing image patches are s1, s2, ..., s k Among them, the intrinsic parameters of the remote sensing cameras used to collect data in the remote sensing database are K. S The rotation matrix of the extrinsic parameter is R. S The translation vector is T S For remote sensing image patches s1, s2, ..., s k Perform projection transformation to obtain the projected ground point cloud q1,q2,...,q corresponding to the remote sensing image patch. k Construct a projection point cloud q1,q2,...,q from a remote sensing image. k And the actual collected local point cloud p1, p2, ..., p k The geometric transformation equations between them are:

[0047] r i p i +t i =q i

[0048] Where, r i and t i Let represent the rotation vector and translation matrix of the spatial transformation, respectively.

[0049] Next, the point cloud q1,q2,...,q can be projected using remote sensing images. k The three-dimensional spatial coordinates of several points between the actual collected local point cloud and the obtained data are used to solve a system of equations to calculate r1, r2, ..., r k and t1,t2,...,t k .

[0050] Based on the obtained local geometric transformation relationships, all ground point clouds are combined into an incremental point cloud p. 1k =r1p1+t1+...+r k p k +t k p is obtained through rotation matrix and translation vector sequence. 1k The geometric transformation relationship between P0 and P0 is as follows:

[0051] The rotation matrix is:

[0052] R 1k =(r1+r2+..+r k ) / k

[0053] The translation vector is:

[0054] T 4k =(t1+t2+..+t) k ) / k.

[0055] In step S105, the local pose relationship of the point cloud is geometrically transformed with the local point cloud data of the target to obtain the transformation result, and the point cloud data of the target ground map is updated according to the transformation result to obtain the updated point cloud increment of the target ground map.

[0056] It is understood that the embodiments of this application can perform geometric space transformation on the local pose relationship of the point cloud in the following steps and the local point cloud data of the target to obtain the transformation result, and update the point cloud data of the target ground map according to the transformation result to obtain the updated target ground map point cloud increment, which effectively reduces the error of ground map point cloud increment construction in real environment and improves the accuracy of point cloud increment construction.

[0057] Optionally, in one embodiment of this application, performing a geometrical spatial transformation on the local pose relationship of the point cloud and the local point cloud data of the target to obtain a transformation result, and updating the point cloud data of the target ground map according to the transformation result to obtain the updated target ground map point cloud increment includes: performing a geometrical spatial transformation on the local point cloud of the target based on a rotation matrix and a translation vector to obtain a transformation result, and stitching the transformation result onto the target ground map point cloud model to obtain the updated target ground map point cloud increment.

[0058] In some embodiments, the present application can be based on the rotation matrix R 4k Translation vector T 1k , p the overall point cloud 1k Perform a geometric transformation and then stitch the transformation result onto the complete ground map point cloud model P0. The specific stitching method is as follows:

[0059] P 0k=R 1k p 1k +T 1k ,

[0060] Among them, P 0k This is the updated, complete ground point cloud data.

[0061] In summary, the embodiments of this application can dynamically acquire local ground map point clouds, identify and dynamically track them, obtain motion trajectories and orientation information in large-scale remote sensing images, assist in the registration and alignment of the point cloud data itself, and finally obtain continuously updated ground map point clouds.

[0062] The method for incremental construction of ground map point clouds based on remote sensing images proposed in this application can acquire point cloud data of a target ground map and the corresponding remote sensing image, collect target local point cloud data with temporal information within a certain period of time, extract features from the target local point cloud data to obtain feature vectors and fit them, obtain the relative acquisition path of the target local point cloud based on the fitting results, and then perform cross-optimization on the features of the remote sensing image and the features of the target local point cloud. Based on the cross-optimization results, relative path coefficients and cross paths are obtained for weighted processing, and then the remote sensing image is locally reprojected. Based on the projection results, an initial pose for local point cloud registration is generated and optimized. Based on the optimization results, a point cloud local pose relationship with temporal information is constructed, and the target local point cloud data is geometrically transformed to obtain the updated target ground map point cloud increment. This effectively reduces the error in incremental construction of ground map point clouds in real environment and improves the accuracy of incremental point cloud construction. This solves the problem in related technologies where intermittent data loss of absolute and relative positioning devices can easily occur in harsh or high-speed environments, leading to failure or large errors in incremental point cloud construction of ground maps in real-world environments, thereby reducing the accuracy of incremental point cloud construction.

[0063] Next, referring to the accompanying drawings, a ground map point cloud incremental construction apparatus based on remote sensing image guidance proposed in an embodiment of this application is described.

[0064] Figure 2 This is a block diagram of a ground map point cloud incremental construction device based on remote sensing image guidance according to an embodiment of this application.

[0065] like Figure 2 As shown, the ground map point cloud incremental construction device 10 based on remote sensing image guidance includes: an acquisition module 100, a determination module 200, a first processing module 300, a second processing module 400, and an update module 500.

[0066] Specifically, the acquisition module 100 is used to acquire point cloud data of the target ground map and remote sensing images of the target ground map, and to collect local point cloud data of the target with time sequence information within a preset time period.

[0067] The determination module 200 is used to extract at least one feature from the target local point cloud data, obtain at least one feature vector based on the at least one feature, fit the at least one feature vector to obtain the fitting result, and obtain the relative acquisition path of the target local point cloud based on the fitting result.

[0068] The first processing module 300 is used to perform cross-optimization on the first feature of the remote sensing image and the second feature of the target local point cloud based on the relative acquisition path, to obtain the cross-optimization result, and to obtain the relative path coefficient and cross path based on the cross-optimization result, and to perform weighted processing on the relative path coefficient and cross path to obtain the incremental point cloud acquisition path.

[0069] The second processing module 400 is used to perform local reprojection on the remote sensing image based on the incremental point cloud acquisition path, obtain the projection result, generate the initial pose for local point cloud registration based on the projection result, optimize the initial pose to obtain the optimization result, and construct the local pose relationship of the point cloud with temporal information based on the optimization result.

[0070] The update module 500 is used to perform geometric space transformation between the local pose relationship of the point cloud and the local point cloud data of the target, obtain the transformation result, and update the point cloud data of the target ground map according to the transformation result to obtain the updated point cloud increment of the target ground map.

[0071] Optionally, in one embodiment of this application, the determining module 200 includes: a first extraction unit, a first acquisition unit, and a first determining unit.

[0072] The first extraction unit is used to extract at least one feature from the target local point cloud data and obtain at least one feature vector based on the at least one feature of the target local point cloud data.

[0073] The first acquisition unit is used to transform at least one feature vector based on a multilayer perceptron (MLP) to obtain at least one matrix parameter.

[0074] The first determining unit is used to concatenate time series information with at least one matrix parameter to obtain a relative acquisition path based on time series information fitting.

[0075] Optionally, in one embodiment of this application, the first processing module 300 includes: a second extraction unit, a second acquisition unit, a first processing unit, a third acquisition unit, and a fourth acquisition unit.

[0076] The second extraction unit is used to extract at least one feature of the remote sensing image and obtain at least one remote sensing feature vector based on the at least one feature of the remote sensing image.

[0077] The second acquisition unit is used to extract local features from the point cloud data of the target ground map to obtain at least one point cloud feature vector.

[0078] The first processing unit is used to expand and copy at least one remote sensing feature vector and at least one point cloud feature vector, and concatenate them with at least one feature vector to obtain at least one feature matrix.

[0079] The third acquisition unit is used to transform at least one feature matrix based on a multilayer perceptron (MLP) to obtain at least one matrix parameter, relative path coefficient, and cross path.

[0080] The fourth acquisition unit is used to weight and superimpose the relative path coefficients and intersection paths to obtain the optimized incremental point cloud acquisition path.

[0081] Optionally, in one embodiment of this application, the second processing module 400 includes: a second processing unit, a third processing unit, and a generation unit.

[0082] The second processing unit is used to extract at least one remote sensing image block from the corresponding points in the remote sensing image, and to perform projection transformation on the at least one remote sensing image block to obtain at least one projected ground point cloud corresponding to the remote sensing image block.

[0083] The third processing unit is used to construct a set of equations for local geometric transformation using at least one projected ground point cloud and a target local point cloud, and to solve the set of equations using the three-dimensional spatial coordinates of the target point between the projected ground point cloud and the target local point cloud to obtain the solution result.

[0084] The generation unit is used to combine the target ground map point cloud into an incremental point cloud based on the solution results. It obtains the geometric transformation relationship through the rotation matrix and translation vector sequence to generate the local pose of the point cloud.

[0085] Optionally, in one embodiment of this application, the update module 500 includes a second determining unit.

[0086] The second determining unit is used to perform geometric spatial transformation on the local point cloud of the target based on the rotation matrix and translation vector, obtain the transformation result, and stitch the transformation result onto the target ground map point cloud model to obtain the updated target ground map point cloud increment.

[0087] It should be noted that the foregoing explanation of the embodiment of the method for incremental construction of ground map point cloud based on remote sensing image guidance also applies to the device for incremental construction of ground map point cloud based on remote sensing image guidance in this embodiment, and will not be repeated here.

[0088] The ground map point cloud incremental construction device based on remote sensing image guidance proposed in this application can acquire point cloud data of a target ground map and the corresponding remote sensing image, collect target local point cloud data with temporal information within a certain period of time, extract features of the target local point cloud data to obtain feature vectors and fit them, obtain the relative acquisition path of the target local point cloud based on the fitting results, and then perform cross-optimization on the features of the remote sensing image and the features of the target local point cloud, and obtain relative path coefficients and cross paths based on the cross-optimization results for weighted processing, and then perform local reprojection on the remote sensing image, generate the initial pose for local point cloud registration based on the projection results, optimize the initial pose, construct the point cloud local pose relationship with temporal information based on the optimization results, and perform geometric space transformation on the target local point cloud data to obtain the updated target ground map point cloud increment, thereby effectively reducing the error of ground map point cloud incremental construction in real environment and improving the accuracy of point cloud incremental construction. This solves the problem in related technologies where intermittent data loss of absolute and relative positioning devices can easily occur in harsh or high-speed environments, leading to failure or large errors in incremental point cloud construction of ground maps in real-world environments, thereby reducing the accuracy of incremental point cloud construction.

[0089] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0090] The memory 301, the processor 302, and the computer program stored on the memory 301 and capable of running on the processor 302.

[0091] When the processor 302 executes the program, it implements the method for incremental construction of ground map point clouds based on remote sensing images provided in the above embodiments.

[0092] Furthermore, electronic devices also include:

[0093] Communication interface 303 is used for communication between memory 301 and processor 302.

[0094] The memory 301 is used to store computer programs that can run on the processor 302.

[0095] The memory 301 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0096] If the memory 301, processor 302, and communication interface 303 are implemented independently, then the communication interface 303, memory 301, and processor 302 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0097] Optionally, in a specific implementation, if the memory 301, processor 302, and communication interface 303 are integrated on a single chip, then the memory 301, processor 302, and communication interface 303 can communicate with each other through an internal interface.

[0098] Processor 302 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0099] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for incremental construction of ground map point clouds guided by remote sensing images.

[0100] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0101] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0102] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0103] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0104] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0105] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0106] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0107] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for incremental construction of a ground map point cloud based on remote sensing image guidance, characterized in that, Includes the following steps: Acquire point cloud data of the target ground map and remote sensing images of the target ground map, and collect local point cloud data of the target with temporal information within a preset time period; Extract at least one feature from the target local point cloud data, and obtain at least one feature vector based on the at least one feature. Fit the at least one feature vector to obtain a fitting result, and obtain the relative acquisition path of the target local point cloud based on the fitting result. Based on the relative acquisition path, the first feature of the remote sensing image and the second feature of the target local point cloud are cross-optimized to obtain the cross-optimization result. The relative path coefficient and the cross path are obtained according to the cross-optimization result. The relative path coefficient and the cross path are weighted to obtain the incremental point cloud acquisition path. Based on the incremental point cloud acquisition path, the remote sensing image is locally reprojected to obtain the projection result, and the initial pose of the local point cloud registration is generated based on the projection result. The initial pose is then optimized to obtain the optimization result, so as to construct a point cloud local pose relationship with temporal information based on the optimization result. as well as The local pose relationship of the point cloud is geometrically transformed with the local point cloud data of the target to obtain the transformation result. The point cloud data of the target ground map is then updated based on the transformation result to obtain the updated target ground map point cloud increment.

2. The method of claim 1, wherein, The steps of extracting at least one feature from the target local point cloud data, obtaining at least one feature vector based on the at least one feature, fitting the at least one feature vector to obtain a fitting result, and obtaining the relative acquisition path of the target local point cloud based on the fitting result include: Extract at least one feature from the target local point cloud data, and obtain at least one feature vector based on the at least one feature of the target local point cloud data; Based on a multilayer perceptron (MLP), the at least one feature vector is transformed to obtain at least one matrix parameter; The timing information is concatenated with the at least one matrix parameter to obtain the relative acquisition path fitted based on the timing information.

3. The method of claim 1, wherein, The step of cross-optimizing the first feature of the remote sensing image and the second feature of the target local point cloud based on the relative acquisition path to obtain the cross-optimization result, and obtaining the relative path coefficient and the cross path according to the cross-optimization result, and then weighting the relative path coefficient and the cross path to obtain the incremental point cloud acquisition path includes: Extract at least one feature from the remote sensing image, and obtain at least one remote sensing feature vector based on the at least one feature of the remote sensing image; Local feature extraction is performed on the point cloud data of the target ground map to obtain at least one point cloud feature vector; The at least one remote sensing feature vector and the at least one point cloud feature vector are expanded and copied, and then concatenated with the at least one feature vector to obtain at least one feature matrix; Based on a multilayer perceptron (MLP), the at least one feature matrix is ​​transformed to obtain at least one matrix parameter, relative path coefficient, and cross path; The relative path coefficients and the intersection paths are weighted and superimposed to obtain the optimized incremental point cloud acquisition path.

4. The method of claim 1, wherein, The step of performing local reprojection on the remote sensing image based on the incremental point cloud acquisition path to obtain a projection result, generating an initial pose for the local point cloud registration based on the projection result, and optimizing the initial pose to obtain an optimized result, and constructing a point cloud local pose relationship with temporal information based on the optimized result includes: The corresponding points in the remote sensing image are cropped into a preset block of remote sensing image to obtain at least one remote sensing image block. The at least one remote sensing image block is then projected and transformed to obtain at least one projected ground point cloud corresponding to the remote sensing image block. A system of equations for local geometric transformation is constructed using at least one projected ground point cloud and the target local point cloud. The system of equations is then solved using the three-dimensional spatial coordinates of the target point between the projected ground point cloud and the target local point cloud to obtain the solution result. Based on the solution results, the target ground map point cloud is combined into an incremental point cloud as a whole, and the geometric transformation relationship is obtained through the rotation matrix and translation vector sequence to generate the local pose of the point cloud.

5. The method of claim 4, wherein, The step of performing a geometric space transformation between the local pose relationship of the point cloud and the local point cloud data of the target to obtain a transformation result, and updating the point cloud data of the target ground map according to the transformation result to obtain the updated target ground map point cloud increment includes: Based on the rotation matrix and the translation vector, the geometric space transformation is performed on the local point cloud of the target to obtain the transformation result, and the transformation result is stitched onto the target ground map point cloud model to obtain the updated target ground map point cloud increment.

6. A device for incremental construction of a ground map point cloud based on remote sensing image guidance, characterized in that, include: The acquisition module is used to acquire point cloud data of the target ground map and remote sensing images of the target ground map, and to collect local point cloud data of the target with time sequence information within a preset time period. The determination module is used to extract at least one feature from the target local point cloud data, obtain at least one feature vector based on the at least one feature, fit the at least one feature vector to obtain a fitting result, and obtain the relative acquisition path of the target local point cloud based on the fitting result. The first processing module is used to perform cross-optimization on the first feature of the remote sensing image and the second feature of the target local point cloud based on the relative acquisition path to obtain the cross-optimization result, and obtain the relative path coefficient and the cross path according to the cross-optimization result, and perform weighted processing on the relative path coefficient and the cross path to obtain the incremental point cloud acquisition path. The second processing module is used to perform local reprojection on the remote sensing image based on the incremental point cloud acquisition path to obtain the projection result, generate the initial pose of the local point cloud registration based on the projection result, optimize the initial pose to obtain the optimization result, and construct the point cloud local pose relationship with temporal information based on the optimization result. as well as The update module is used to perform a geometric space transformation between the local pose relationship of the point cloud and the local point cloud data of the target, obtain the transformation result, and update the point cloud data of the target ground map according to the transformation result to obtain the updated target ground map point cloud increment.

7. The apparatus of claim 6, wherein, The determining module includes: The first extraction unit is used to extract at least one feature from the target local point cloud data and obtain at least one feature vector based on the at least one feature of the target local point cloud data. The first acquisition unit is used to transform the at least one feature vector based on a multilayer perceptron (MLP) to obtain at least one matrix parameter. The first determining unit is used to concatenate the timing information with the at least one matrix parameter to obtain the relative acquisition path fitted based on the timing information.

8. The apparatus of claim 6, wherein, The first processing module includes: The second extraction unit is used to extract at least one feature of the remote sensing image and obtain at least one remote sensing feature vector based on the at least one feature of the remote sensing image. The second acquisition unit is used to extract local features from the point cloud data of the target ground map to obtain at least one point cloud feature vector. The first processing unit is used to expand and copy the at least one remote sensing feature vector and the at least one point cloud feature vector, and concatenate them with the at least one feature vector to obtain at least one feature matrix; The third acquisition unit is used to transform the at least one feature matrix based on a multilayer perceptron (MLP) to obtain at least one matrix parameter, relative path coefficient, and cross path. The fourth acquisition unit is used to weight and superimpose the relative path coefficients and the intersection paths to obtain the optimized incremental point cloud acquisition path.

9. An electronic device, comprising: include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the method for incremental construction of ground map point clouds based on remote sensing image guidance as described in any one of claims 1-5.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method for incremental construction of ground map point clouds based on remote sensing images as described in any one of claims 1-5.

Images