Multi-pass data alignment method, device and equipment

Abstract

The invention discloses a multi-pass data alignment method, device and equipment, and the method comprises the steps: constructing a graph optimization constraint factor in each group of data in a to-be-optimized window list based on the alignment sampling points of a semantic object and a lane object included in a plurality of pieces of data in each group of data in the to-be-optimized window list; based on the anchor point data of the group data queue, determining a group data boundary of the to-be-optimized sliding window so as to construct a graph optimization constraint factor between group data included in the to-be-optimized sliding window; and performing graph optimization solution based on the graph optimization constraint factors in the group data and / or the graph optimization constraint factors among the group data so as to obtain a segment data graph optimization transformation matrix included in the group data, and aligning multiple times of data based on the segment data graph optimization transformation matrix. According to the method, the intra-group data can be aligned through the intra-group constraint, and the component data is aligned through the component constraint, so that relative errors among multiple data are eliminated, and the precision and reliability of map elements during reconstruction are ensured.

Description

Technical Field

[0001] The present invention relates to the field of geographic information technology, mainly to the field of high-precision map technology based on multi-source crowdsourcing data, and in particular to a method, device and equipment for aligning multiple passes of data. Background Art

[0002] High-precision maps can be used for autonomous driving, and the recognition and production of road signs are important components of generating high-precision maps. Road signs (semantic objects and vehicle lane objects) are generally recognized and extracted through road image information collected by cameras mounted on vehicles.

[0003] In the process of making high-precision maps, in order to solve the drawback of large resource investment in single-trip closed-loop collection (closed-loop single-trip collection) and optimize application scenarios, the inventor previously proposed a solution for making high-precision maps based on multi-source crowdsourcing data, that is, by using private cars as collection equipment, data collection is performed without interfering with the stable and fast driving of the vehicle. In the process of using private cars to collect data, since it is impossible to stipulate that a certain vehicle strictly follows a closed-loop trajectory, most vehicle trajectories are only one-way trajectories from the starting point to the destination, so multiple single-track data is used in the later production of high-precision maps, that is, multiple / multiple image collections are required for the same location, and then the road sign information in each collected image is extracted separately, and finally the same road sign collected in multiple trips is fused, and then a high-precision map is produced based on multi-source crowdsourcing data. Summary of the invention

[0004] In order to improve the ability of the same map data (vehicle lane objects or semantic objects, etc.) in actual roads to be successfully clustered during clustering, thereby enriching technical routes and increasing selection space, an embodiment of the present invention provides a method, device and equipment for aligning multiple data.

[0005] In a first aspect, an embodiment of the present invention provides a method for aligning multiple passes of data, which may include:

[0006] Based on the aligned sampling points of the semantic objects and the lane objects included in the plurality of segment data in each group data in the to-be-optimized window list, constructing a graph optimization constraint factor in each group data in the to-be-optimized window list;

[0007] Based on the anchor point data of the group data queue, determine the group data boundary of the sliding window to be optimized, so as to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized;

[0008] Graph optimization is solved based on the graph optimization constraint factors within the group data and / or the graph optimization constraint factors between the groups of data to obtain a graph optimization transformation matrix of the segment data included in the group data, and multiple passes of data are aligned based on the segment data graph optimization transformation matrix.

[0009] Optionally, the method may further include: determining, in a group data queue, the group data included in the window list to be optimized and the anchor point data in the group data queue based on whether the group data has been subjected to an alignment operation and the signal accuracy of the group data.

[0010] Optionally, the determining the group data included in the window list to be optimized and the anchor point data in the group data queue may include:

[0011] Initializing a group data queue, randomly selecting a group data from the group data queue as current group data, and initializing a search state of the current group data to a closed state;

[0012] Determine whether the current group of data has been subjected to an alignment operation, and if so, use the current group of data as anchor data of the group of data queue, and adjust the search state of the group of data to a closed state;

[0013] If the alignment operation has not been performed, determining the signal accuracy of the current set of data;

[0014] If the signal accuracy of the current group of data is poor, the current group of data is added to the list of windows to be optimized; if the signal accuracy of the current group of data is good, the current group of data is used as the anchor point data of the group of data queue, and the search state of the group of data is adjusted to a closed state;

[0015] The group data queue is traversed to determine the group data included in the to-be-optimized window list and the anchor point data in the group data queue.

[0016] Optionally, constructing the graph optimization constraint factor in each group of data in the window list to be optimized may include:

[0017] Determining an alignment error term of the segment data based on alignment sampling points of semantic objects and lane objects included in a plurality of segment data in the group data;

[0018] A graph optimization constraint factor within the set of data is constructed based on the alignment error term and the positions of the alignment sampling points.

[0019] Optionally, the determining of the group data boundary of the sliding window to be optimized based on the anchor point data of the group data queue to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized may include:

[0020] Based on the current group data in the sliding window to be optimized, perform a layer-order traversal, and determine the group data boundary of the sliding window to be optimized and the associated group data included in the sliding window to be optimized by using the anchor point data;

[0021] A graph optimization constraint factor between the group data included in the sliding window to be optimized is constructed based on the trajectory constraints between the segment data included in each group data in the associated group data.

[0022] Optionally, the constructing of a graph optimization constraint factor between the group data included in the sliding window to be optimized based on the trajectory constraints between the segment data included in each group data in the associated group data may include:

[0023] The preset trajectory points of the segment data included in each group data in the associated group data are used as edge constraint nodes between the associated group data, so as to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized based on the edge constraint nodes.

[0024] Optionally, the method may further include: performing clustering and aggregation based on the aligned multi-trip data to aggregate and obtain map data including semantic objects and lane objects.

[0025] In a second aspect, an embodiment of the present invention provides a multi-pass data alignment device, which may include:

[0026] An intra-group constraint factor construction module, used to construct a graph optimization constraint factor in each group of data in the to-be-optimized window list based on the aligned sampling points of the semantic objects and the vehicle lane objects included in the plurality of segment data in each group of data in the to-be-optimized window list;

[0027] An inter-group constraint factor construction module is used to determine the group data boundary of the sliding window to be optimized based on the anchor point data of the group data queue, so as to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized;

[0028] A solution module, used for performing graph optimization solution based on the graph optimization constraint factors within the group of data and / or the graph optimization constraint factors between the groups of data;

[0029] The alignment module is used to obtain a segment data graph included in the group data to optimize the transformation matrix, and align the multiple passes of data based on the segment data graph optimization transformation matrix.

[0030] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for aligning multiple passes of data as described in the first aspect.

[0031] In a fourth aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for aligning multiple passes of data as described in the first aspect is implemented.

[0032] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:

[0033] The embodiment of the present invention provides a method, device and equipment for aligning multiple passes of data. The method can align data within a group through intra-group constraints, and align component data through component constraints, thereby eliminating relative errors between multiple passes of multi-source data and ensuring the accuracy and reliability of map elements during reconstruction.

[0034] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0035] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0037] Figure 1 A schematic diagram for explaining relevant technical terms provided in an embodiment of the present invention;

[0038] Figure 2 A flowchart of a method for aligning multiple passes of data provided in an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of a method for aligning multiple passes of data provided in an embodiment of the present invention;

[0040] Figure 4 A flowchart of a detailed multi-pass data alignment method provided in an embodiment of the present invention;

[0041] Figure 5 A flowchart of determining a list of windows to be optimized provided in an embodiment of the present invention;

[0042] Figure 6 This is an example of two segments in group data provided in an embodiment of the present invention;

[0043] Figure 7 This is a sampling example of a lane object in a segment in the group data provided in an embodiment of the present invention;

[0044] Figure 8 This is a sampling example of semantic objects in a segment within group data provided in an embodiment of the present invention;

[0045] Fig. 9An example of building a graph model for a segment in group data provided in an embodiment of the present invention;

[0046] Fig.10 This is a diagram showing the effect of adjusting the constraint factors constructed based on the ICP objective function after solving the problem in the embodiment of the present invention;

[0047] Fig.11 This is a diagram showing the effect of solving and adjusting the constraint factors constructed based on the GICP objective function according to an embodiment of the present invention;

[0048] Fig.12 This is the effect between bunches that occurs after the bunches are individually segment aligned in the embodiment of the present invention;

[0049] Fig.13 A schematic diagram of the construction of a window to be optimized in an embodiment of the present invention;

[0050] Fig.14 A schematic diagram of a graph structure constructed in an embodiment of the present invention;

[0051] Fig.15 This is a diagram showing the effect of alignment between data groups in an embodiment of the present invention;

[0052] Fig.16 It is a schematic diagram of the structure of a multi-pass data alignment device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0054] In the process of making a high-precision map based on multiple passes of data, the same map data in the actual road, such as lane objects, semantic objects of the road surface, etc., must be clustered and then aggregated into a semantic object. The inventors found that due to the limited acquisition accuracy of multi-source data during acquisition, and the density-based clustering has certain requirements on the distance of the same semantic object, it is necessary to align the multiple passes of data so that the multiple passes of data are roughly in the same position to facilitate clustering and aggregation. In the process of aligning the multiple passes of data, the inventors also found that due to the multiple collections of the multiple passes of visual data itself, the data accuracy is limited, and there is a lack of effective scale calibration information between the passes of data. Therefore, how to align the multiple passes of data is a necessary step in the subsequent construction of a high-precision map. In view of the above problems, the present invention is proposed to provide a method, device and equipment for aligning multiple passes of data to overcome the above problems or at least partially solve the above problems.

[0055] In order to facilitate a better understanding of the embodiments of the present invention, some technical terms that may be involved in the embodiments of the present invention are explained here. Figure 1 As shown in the figure, a tile is the basic unit of a high-precision map, that is, there are several tile data in the map in the form of raster data; a tile can include multiple groups of data (bunch), a bunch is a collection of segment data (segment), and the areas passed by these segments are required to be roughly the same; a segment represents the data generated by each vehicle passing through a certain area. For example, a road between two intersections is passed by multiple vehicles, and multiple vehicles generate multiple segment data between the two intersections. These different segment data belong to the same bunch; and after a vehicle passes through a certain area, it will collect the vehicle line, semantic objects, etc. in the area. A segment includes vehicle line objects and semantic objects.

[0056] The embodiment of the present invention provides a method for aligning multiple data passes. Figure 2 to Figure 4 As shown, the method may include the following steps:

[0057] Step S21: construct a graph optimization constraint factor in each group of data in the window list to be optimized based on the aligned sampling points of the semantic objects and the vehicle lane objects included in the multiple segment data in each group of data in the window list to be optimized.

[0058] Step S22: Based on the anchor point data of the group data queue, determine the group data boundary of the sliding window to be optimized, so as to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized.

[0059] It should be noted that, when executing the above-mentioned steps S21 and S22, step S21 may be executed first and then step S22, or step S22 may be executed first and then step S21, or step S21 and step S22 may be executed at the same time, or only one of step S21 and step S22 may be executed, and the embodiment of the present invention does not make any specific limitation on this.

[0060] Step S23: performing graph optimization based on the graph optimization constraint factors within the group data and / or the graph optimization constraint factors between the group data to obtain the segment data graph optimization transformation matrix included in the group data, and aligning the multiple passes of data based on the segment data graph optimization transformation matrix.

[0061] In the embodiment of the present invention, when performing graph optimization solution, the solution may be performed based only on the graph optimization constraint factors within the group data, or it may be performed based only on the graph optimization constraint factors between the group data. Of course, the optimal solution may also be to use the graph optimization constraint factors within and between groups to solve at the same time, and the embodiment of the present invention does not make any specific limitation on this.

[0062] The above-mentioned multi-pass data alignment method provided in an embodiment of the present invention can align the data within the group through the intra-group constraints, and align the component data through the component constraints, thereby eliminating the relative errors between multi-pass multi-source data and ensuring the accuracy and reliability of map elements during reconstruction.

[0063] In an alternative embodiment, referring to Figure 2 to Figure 4 As shown, before constructing the graph optimization constraint factors within the group data and the graph optimization constraint factors between the group data, the following steps need to be performed, namely: step S20, based on whether the group data has been aligned and the signal accuracy of the group data, determine the group data included in the window list to be optimized and the anchor data in the group data queue in the group data queue. In this step, due to the large volume of data, it is impossible to perform alignment operations at the same time, and the aligned group data can be used as a benchmark for the unaligned data. Therefore, this step needs to determine which data in a series of data needs to be aligned first, so as to facilitate the alignment step.

[0064] In a specific embodiment, the group data included in the window list to be optimized and the anchor point data in the group data queue are determined, referring to Figure 5 and Figure 4 As shown, the following steps may be included:

[0065] Step S51, initialize the group data queue, randomly select a group data from the group data queue as the current group data, and initialize the search state of the current group data to the closed state.

[0066] This step is to initialize a bunch queue (i.e., clear the bunch queue), randomly add a current bunch 1 to the bunch queue, and initialize the search state of the current bunch 1 to False (including forward search state and backward search state). It should be noted that the bunch queue mentioned below is the group data queue.

[0067] Step S52, determine whether the current group of data has been aligned. If so, execute step S53; if not, execute step S54.

[0068] This step is to execute the queue and pop up the above-mentioned bunch 1. If the alignment operation has been performed, no further alignment is required, and it can be used as anchor data to align other groups of data in the queue. That is, the segment that has been aligned is set to visited (check whether the search status of this bunch 1 has been activated), that is, set as an anchor point, so that the segment will not be adjusted when it enters optimization again.

[0069] Step S53: Use the current group data as the anchor data of the group data queue, and adjust the search state of the group data to a closed state.

[0070] Step S54, determine the quality of the signal accuracy of the current set of data; if the signal accuracy of the current set of data is poor, execute step S55; if the signal accuracy of the current set of data is good, execute step S53.

[0071] This step is to judge whether the signal of the segment in this bunch 1 is good or bad (excellent or inferior). The judgment is made based on the signal quality of the track points in the segment. When the signals of the track points are all good, the signal of the segment is evaluated as good. If the signals of all the segments in a bunch are good, the signal of the group of data is evaluated as good).

[0072] Step S55: Add the current group data to the list of windows to be optimized.

[0073] In the specific implementation of this step, if the signal is not good, add this bunch 1 to the sliding window to be optimized, and check whether this bunch 1 has activated the search state; if not, activate the search state for it, and adjust the forward and backward search states of this bunch 1 to True. Adjust the forward search state to True, and add the forward connection Bunch 2 to the bunch queue, adjust the backward search state to True, and add the backward connection Bunch 3 to the bunch queue.

[0074] In this step, when the signal is good, check whether the search status of bunch 1 has been activated. If it has been activated, adjust the current search direction to False, that is, the boundary of the current window (list of windows to be optimized) in one direction is found, so no more search is performed in this direction.

[0075] Step S56: traverse the group data queue to determine the group data included in the window list to be optimized and the anchor point data in the group data queue.

[0076] This step is to check whether the bunch queue is empty. If not, return to step S51 and continue to execute the dequeue. If it is empty, determine whether the list of windows to be optimized is empty. If it is empty, end this round of alignment process. If not, continue with the following alignment steps.

[0077] In a specific example, initialize the bunch queue, join bunch 1, pop bunch 1,

[0078] ① The signal of bunch 1 is not good, so this bunch 1 is added to the sliding window to be optimized, and the forward search is performed at the same time, and the forward-connected bunch 2 is added to the bunch queue. The backward search is performed, and the backward-connected bunch 3 is added to the bunch queue. At this time, there are two bunches (Bunch 2 and Bunch 3) in the bunch queue;

[0079] ② The signal of bunch 1 is good and has been activated. Adjust the forward search state and no longer search forward, but only search backward.

[0080] At this time, the bunch queue is not empty, so bunch 2 continues to be popped out, and the above operation is repeated to continuously add bunches with poor signals to the sliding window to be optimized, until the bunch queue is empty and no more bunches are popped out.

[0081] In an optional embodiment, a graph optimization constraint factor is constructed within each group data in the window list to be optimized, so that alignment optimization is performed within each bunch. In a specific implementation, this embodiment can achieve the top, bottom, left, and right alignment within the group data by constructing a line-to-line objective function, which may specifically include: first, based on the alignment sampling points of the semantic objects and the vehicle line objects included in the multiple segment data in the group data, determining the alignment error term of the segment data; then, based on the alignment error term and the position of the alignment sampling point, constructing the graph optimization constraint factor within the group data (the line-to-line GICP objective function is used in this embodiment).

[0082] In the embodiment of the present invention, the optimization constraint construction of the bunch inner graph is mainly divided into two types of constraints, one is based on the point-to-point error between semantic object sampling points, and the other is based on the line-to-line error between lane line sampling points.

[0083] The constraint construction between semantic objects is consistent with the traditional ICP algorithm. The ICP algorithm transforms the points to be registered in the point cloud into the same coordinate system and performs the least squares method on the Euclidean distance of all sample points to obtain the optimal transformation matrix T * , now suppose there are two sets of point clouds A={a i},B={b i}, for each matching point cloud sample {a i ,b i}, the loss function is as follows:

[0084]

[0085] In the above formula and For point a i and b i The transformation matrix from the origin of the world coordinate system to its own position, based on the maximum a posteriori estimation, has the following objective function, where w i Weights for sampling point registration:

[0086] argmin T {∑ i w i ‖b i -Ta i ‖ 2} (2) After adding the plane normal vector η i After that, we get the objective function of the point-to-plane case:

[0087] argmin T {∑ i w i ‖η i (b i -Ta i )‖ 2} (3)

[0088] In the g2o framework, the observed trajectory points of the point cloud sets A and B are vertices (in the embodiment of the present invention, the trajectory points are found by using the timestamps of the vehicle-side fusion frames), and each loss function e i is an edge connecting two vertices.

[0089] In a specific example, the raw data input before alignment processing refers to Figure 6 As shown, now sample these two segments, refer to Figure 7 and Figure 8 As shown in the figure, 32 sets of matching sampling points are obtained. Among them, 30 sets of matching relationships come from lane lines and 2 sets come from semantic objects. The above sampling points are used as alignment sampling points to build the model. There are 32 sets of matching relationships between two observation points. Because the two observation points point to each other, that is, every two matching points point to each other, refer to Fig. 9 As shown in , there are 64 error edges (alignment error terms), which are different when pointing from the upper point to the lower point and from the lower point to the upper point. Each edge represents the error between this set of matching relationships, because the two matching points are the same point, but the poses of the same point observed at different observation points are inconsistent, which indicates that an error has occurred. Using the above formula (3) as the objective function, the least squares method is used to solve and obtain the optimal transformation matrix T * (displacement and object), adjust the two matching segments separately, and get the result as follows Fig.10 shown.

[0090] Reference Fig.10 As shown in the figure, after adjustment, the two segments are not completely aligned. The inventor found that the reason is that the lane line sampling provides a longitudinal error, but the longitudinal error in the lane line sampling is wrong. Therefore, the longitudinal error provided by the lane line sampling point needs to be ignored. In order to solve the above problem, the plane-to-plane GICP algorithm is used in the embodiment of the present invention, and the line-to-line GICP algorithm is proposed to optimize and improve it. The details are as follows:

[0091] In general SLAM scenarios, it is assumed that the corner points of the image have offset errors on the plane, so the distance from point to point or point to surface becomes the error distance from surface to surface. The objective function of surface-to-surface GICP is as follows: is the covariance of point cloud A and B, and has undergone a normal projection:

[0092]

[0093] Use the maximum likelihood estimate to iteratively calculate T

[0094]

[0095]

[0096] e i =‖b i -Ta i ‖2 (7)

[0097]

[0098]

[0099]

[0100]

[0101] ui and vi represent b i and a i The normal vector of and By rotating the above covariance matrix, p i Let R denote a rotation that transforms the basis vector e1 to some normal vector.

[0102] In the actual application process of map learning, the embodiment of the present invention densely samples semantic objects, restores them to the effect of point cloud to a certain extent, and converts the matching relationship between semantic objects into the matching relationship between sampling points. Since the lane model has a rich road line topology structure, it is more inclined to judge the errors between lines and points, and between lines, compared with the surface-to-surface projection error. In this implementation, the projection information matrix is ​​redesigned, and the GICP is improved in point-to-line and line-to-line calculations, that is, all vectors in the 3D space are projected to a certain projection line segment m, and m is in a polar coordinate system with the polar axis as the positive direction of the x-axis. Then the normal vector of the polar coordinate plane is,

[0103]

[0104] Let the rotation angle from x to m be θ, then we have the Lie algebra The rotation matrix can be obtained:

[0105]

[0106] Here is a specific example, select a simple projection matrix P i (The normal vector represented by this projection matrix should be the same as the above point-to-plane), projecting the xyz coordinate components onto the xz plane, where ξ is an infinitesimal quantity:

[0107]

[0108] The objective function of point-to-line GICP is as follows, where ∑ is the covariance matrix of the GICP after a point-to-plane projection. The following equation degenerates to point-to-plane when the z-axis is completely ignored:

[0109]

[0110] Furthermore, we can get the line-to-line objective function, P ia With P ib The composition of the projection matrix P refers to the above information i However, since the covariance is calculated, the projection matrix P i Take the inverse. Because the projection matrix itself is a singular matrix and cannot be inverted, the infinitesimal introduced plays a role in this step, which ensures the non-singularity of P:

[0111]

[0112]

[0113]

[0114]

[0115] The sampling point group on the lane line is constructed using the new line-to-line GICP objective function, and the sampling point group on the semantic object is constructed using the GICP objective function. The two objective functions are constructed in the same graph model and solved to obtain a new transformation matrix. The segment is adjusted according to the transformation matrix. After adjustment, refer to Fig.11 As shown in the figure, the two segments in the final group data are aligned.

[0116] In another optional embodiment, the above step S12 determines the group data boundary of the sliding window to be optimized based on the anchor point data of the group data queue, so as to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized, which may specifically include: first, based on the current group data in the sliding window to be optimized, a layer-order traversal is performed to determine the group data boundary of the sliding window to be optimized and the associated group data included in the sliding window to be optimized with the anchor point data; then, the graph optimization constraint factor (GICP edge constraint objective function) between the group data included in the sliding window to be optimized is constructed with the trajectory constraints between the segment data included in each group data in the associated group data.

[0117] Reference Fig.12As shown, when each Bunch realizes intra-segment alignment separately, there will be unevenness between the bunches. The reason for this phenomenon is that the constraint of trajectory continuity between the previous and next segments of each segment in the bunch is not taken into account when performing intra-segment alignment. In order to solve the above problems, several bunches need to be put together for modeling and optimization in the embodiment of the present invention. It should be noted that not all the bunches in the Tile are associated and solved. The reason why all the Bunches in the Tile are not modeled and optimized together is from the perspective of the time complexity of the algorithm.

[0118] In the specific implementation, the need to optimize several bunches together becomes a key consideration. In real business scenarios, not all bunches need to be aligned, that is, there are bunches with good signals, and there may be no relative error inside, or the relative error is very small. These bunches can be used as anchor data, that is, fixed points, and they can be used as the boundaries of the window to eliminate the relative error within the window. Therefore, the construction rule of the optimized sliding window is to start with a bunch, search forward and backward for each segment in this bunch according to the trajectory, and put the bunch where the previous and next segments are located into the optimized sliding window until the front and back bunches of the window are all anchor bunches (in addition to the bunch with good signal, there is another bunch that can be used as an anchor, which is an optimized bunch). Stop the forward and backward search and complete the construction of the optimized sliding window.

[0119] Reference Fig.13 As shown, the optimized sliding window is constructed as Fig.13 As shown in (a), the dotted line track in the forward search needs to search four bunches to find the anchor point, while the solid line track needs to search two windows to find the anchor point. Therefore, the forward search stops after finding the anchor point window of the solid line track, and the backward search is the same. It should be noted that the layer-order traversal method is used for forward and backward search in this embodiment.

[0120] After building the sliding window, you can build a joint objective function for multiple bunches in the window. Based on the constraint objective function within the bunch, add the trajectory constraints between the previous and next segments of each segment in the bunch to the objective function to build a new joint objective function. The formula is as follows:

[0121]

[0122] Where T=[T0,T1,...,T K ] represents the K Bunches in the sliding window, T Krepresents the transformation matrix of the Kth bunch, v k represents the observation point of the Kth bunch, Represents the constraint between the previous and next trajectory points.

[0123] The sliding window obtained in the previous example is modeled using a new objective function 20. The graph structure in the model is as follows Fig.14 As shown in the figure, the next bunch can be known based on each segment trajectory point in the bunch. When one set of trajectory points finds a bunch that has been visited or has a good signal, the search downwards stops.

[0124] In an embodiment of the present invention, preset trajectory points of segment data included in each group data in the associated group data are used as edge constraint nodes between the associated group data, so as to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized based on the edge constraint nodes.

[0125] Reference Fig.14 As shown, in the implementation of the present invention, each trajectory point is the first trajectory point (graph node) of each segment in this bunch. The dashed edges in the figure represent the constraints between trajectories, and the straight edges represent the GICP edge constraints constructed by the matched sampling points. Since the semantic objects in the intersections are not matched with the lane lines, there are no GICP edges connecting the observation points at the two intersections. Solving this graph model can obtain the transformation matrix of each segment. After adjustment based on the transformation matrix, the relative error between the segments in each bunch in the window can be eliminated, and the Bunches can be made relatively smooth (refer to Fig.15 effect shown).

[0126] The transformation matrix of each segment calculated in the graph optimization solution step is used to adjust the semantic objects, lane lines, etc. in this segment. After the adjustment, the same semantic objects and lane lines in multi-source data will be located at approximately the same longitude and latitude, thereby eliminating the relative error between the same semantic objects and lane lines.

[0127] In another alternative embodiment, referring to Figure 2 As shown, the method may further include the following steps: Step S24, clustering and aggregating based on the aligned multi-pass data to aggregate and obtain map data including semantic objects and lane objects. The alignment in the embodiment of the present invention provides a data basis for the subsequent steps of clustering and aggregation, making it easier to cluster the semantic objects and lane objects collected by the multi-pass data, and providing a data source for the subsequent aggregation into high-definition map elements.

[0128] Based on the same inventive concept, an embodiment of the present invention further provides a multi-pass data alignment device, referring to Fig.16As shown, it may include:

[0129] The intra-group constraint factor construction module 161 is used to construct a graph optimization constraint factor in each group of data in the to-be-optimized window list based on the aligned sampling points of the semantic objects and the vehicle lane objects included in the plurality of segment data in each group of data in the to-be-optimized window list;

[0130] The inter-group constraint factor construction module 162 is used to determine the group data boundary of the sliding window to be optimized based on the anchor point data of the group data queue, so as to construct the graph optimization constraint factor between the group data included in the sliding window to be optimized;

[0131] The solution module 163 is used to perform graph optimization solution based on the graph optimization constraint factors within the group of data and / or the graph optimization constraint factors between the groups of data;

[0132] The alignment module 164 is used to obtain a segment data graph included in the group data to optimize the transformation matrix, and align the multiple passes of data based on the segment data graph to optimize the transformation matrix.

[0133] In an optional embodiment, the parameter Fig.16 As shown, the device may also include a determination module 160, which is used to determine the group data included in the window list to be optimized and the anchor point data in the group data queue in the group data queue based on whether the group data has performed an alignment operation and the signal accuracy of the group data.

[0134] In another optional embodiment, the determination module 160 is specifically configured to:

[0135] Initializing a group data queue, randomly selecting a group data from the group data queue as current group data, and initializing a search state of the current group data to a closed state;

[0136] Determine whether the current group of data has been subjected to an alignment operation, and if so, use the current group of data as anchor data of the group of data queue, and adjust the search state of the group of data to a closed state;

[0137] If the alignment operation has not been performed, determining the signal accuracy of the current set of data;

[0138] If the signal accuracy of the current group of data is poor, the current group of data is added to the list of windows to be optimized; if the signal accuracy of the current group of data is good, the current group of data is used as the anchor point data of the group of data queue, and the search state of the group of data is adjusted to a closed state;

[0139] The group data queue is traversed to determine the group data included in the to-be-optimized window list and the anchor point data in the group data queue.

[0140] In another optional embodiment, the intra-group constraint factor construction module 161 is specifically used for:

[0141] Determining an alignment error term of the segment data based on alignment sampling points of semantic objects and lane objects included in a plurality of segment data in the group data;

[0142] A graph optimization constraint factor within the set of data is constructed based on the alignment error term and the positions of the alignment sampling points.

[0143] In another optional embodiment, the inter-group constraint factor construction module 162 is specifically used for:

[0144] Based on the current group data in the sliding window to be optimized, perform a layer-order traversal, and determine the group data boundary of the sliding window to be optimized and the associated group data included in the sliding window to be optimized by using the anchor point data;

[0145] A graph optimization constraint factor between the group data included in the sliding window to be optimized is constructed based on the trajectory constraints between the segment data included in each group data in the associated group data.

[0146] In another optional embodiment, the inter-group constraint factor construction module 162 is specifically used to: use the preset trajectory points of the segment data included in each group data in the associated group data as the edge constraint nodes between the associated group data, so as to construct the graph optimization constraint factors between the group data included in the sliding window to be optimized based on the edge constraint nodes.

[0147] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned multi-pass data alignment method is implemented.

[0148] Based on the same inventive concept, a computer device is also provided in an embodiment of the present invention, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the above-mentioned multi-pass data alignment method when executing the program.

[0149] The principles of the problems solved by the above-mentioned devices, clients, media, related equipment and systems in the embodiments of the present invention are similar to those of the above-mentioned methods, so their implementation can refer to the implementation of the above-mentioned methods, and the repeated parts will not be repeated.

[0150] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0151] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0154] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for aligning multiple passes of data, characterized in that: include: Based on the aligned sampling points of the semantic objects and the lane objects included in the plurality of segment data in each group data in the to-be-optimized window list, constructing a graph optimization constraint factor in each group data in the to-be-optimized window list; Based on the anchor point data of the group data queue, determine the group data boundary of the sliding window to be optimized, so as to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized; Graph optimization is solved based on the graph optimization constraint factors within the group data and / or the graph optimization constraint factors between the groups of data to obtain a graph optimization transformation matrix of the segment data included in the group data, and multiple passes of data are aligned based on the segment data graph optimization transformation matrix.

2. The method according to claim 1, characterized in that Also includes: Based on whether the group data has been subjected to an alignment operation and the signal accuracy of the group data, the group data included in the to-be-optimized window list and the anchor point data in the group data queue are determined in the group data queue.

3. The method according to claim 2, characterized in that The determining of the group data included in the to-be-optimized window list and the anchor point data in the group data queue comprises: Initializing a group data queue, randomly selecting a group data from the group data queue as current group data, and initializing a search state of the current group data to a closed state; Determine whether the current group of data has been subjected to an alignment operation, and if so, use the current group of data as anchor data of the group of data queue, and adjust the search state of the group of data to a closed state; If the alignment operation has not been performed, determining the signal accuracy of the current set of data; If the signal accuracy of the current group of data is poor, the current group of data is added to the list of windows to be optimized; if the signal accuracy of the current group of data is good, the current group of data is used as the anchor point data of the group of data queue, and the search state of the group of data is adjusted to a closed state; The group data queue is traversed to determine the group data included in the to-be-optimized window list and the anchor point data in the group data queue.

4. The method according to claim 1, characterized in that: Constructing the graph optimization constraint factors in each group of data in the window list to be optimized includes: Determining an alignment error term of the segment data based on alignment sampling points of semantic objects and lane objects included in a plurality of segment data in the group data; A graph optimization constraint factor within the set of data is constructed based on the alignment error term and the positions of the alignment sampling points.

5. The method according to claim 1, characterized in that The anchor point data based on the group data queue determines the group data boundary of the sliding window to be optimized to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized, including: Based on the current group data in the sliding window to be optimized, perform a layer-order traversal, and determine the group data boundary of the sliding window to be optimized and the associated group data included in the sliding window to be optimized by using the anchor point data; A graph optimization constraint factor between the group data included in the sliding window to be optimized is constructed based on the trajectory constraints between the segment data included in each group data in the associated group data.

6. The method according to claim 5, characterized in that The step of constructing a graph optimization constraint factor between the group data included in the sliding window to be optimized based on the trajectory constraints between the segment data included in each group data in the associated group data includes: The preset trajectory points of the segment data included in each group data in the associated group data are used as edge constraint nodes between the associated group data, so as to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized based on the edge constraint nodes.

7. The method according to any one of claims 1 to 6, characterized in that: Also includes: Clustering and aggregation are performed based on the aligned multi-trip data to aggregate and obtain map data including semantic objects and lane objects.

8. A device for aligning multiple passes of data, characterized in that: include: An intra-group constraint factor construction module, used to construct a graph optimization constraint factor in each group of data in the to-be-optimized window list based on the aligned sampling points of the semantic objects and the vehicle lane objects included in the plurality of segment data in each group of data in the to-be-optimized window list; An inter-group constraint factor construction module is used to determine the group data boundary of the sliding window to be optimized based on the anchor point data of the group data queue, so as to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized; A solution module, used for performing graph optimization solution based on the graph optimization constraint factors within the group of data and / or the graph optimization constraint factors between the groups of data; The alignment module is used to obtain a segment data graph included in the group data to optimize the transformation matrix, and align the multiple passes of data based on the segment data graph optimization transformation matrix.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the multi-pass data alignment method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the multi-pass data alignment method according to any one of claims 1 to 7 is implemented.

Images