Multi-source data processing method, device and equipment

Abstract

The invention discloses a multi-source data processing method, device and equipment, and the method comprises the steps: carrying out the segmentation processing of each single-trip data in multi-source data, so as to divide the single-trip data into a plurality of pieces of segment data; constructing group data corresponding to the multi-source data based on the segment data; matching segment data included in the group data to determine a matching relationship of the segment data; on the basis of the matching relation of the segment data, the lane objects and the semantic objects included in the segment data are sampled to obtain aligned sampling points of the segment data; and aligning the segment data based on the alignment sampling points to realize multi-source data alignment. According to the method, the technical problems that multi-source data errors are different, absolute positions of semantic objects of the same road identification are dispersed, large-range excessive clustering is needed in the later clustering process, and the data matching credibility is low are solved. The processing rate during data processing is improved, and a foundation is laid for making a high-definition map based on multi-source data subsequently.

Description

Technical Field

[0001] The present invention relates to the field of geographic information technology, mainly to the field of high-precision map technology based on multi-source crowdsourcing data, and in particular to a multi-source data processing method, device and equipment. Background Art

[0002] High-precision maps can be used for autonomous driving, and the recognition and production of road signs are important components of generating high-precision maps. Road signs (semantic objects) are generally identified and extracted through road image information collected by cameras mounted on vehicles.

[0003] In the process of making high-precision maps, in order to solve the drawback of large resource investment in single-trip closed-loop collection (closed-loop single-trip collection) and optimize application scenarios, the inventor previously proposed a solution for making high-precision maps based on multi-source crowdsourcing data, that is, by using private cars as collection equipment, data collection is performed without interfering with the stable and fast driving of the vehicle. In the process of using private cars to collect data, since it is impossible to stipulate that a certain vehicle strictly follows a closed-loop trajectory, most vehicle trajectories are only one-way trajectories from the starting point to the destination, so multiple single-track data is used in the later production of high-precision maps, that is, multiple / multiple image collections are required for the same location, and then the road sign information in each collected image is extracted separately, and finally the same road sign collected in multiple trips is fused, and then a high-precision map is produced based on multi-source crowdsourcing data. Summary of the invention

[0004] In order to facilitate the fusion of multi-source data to form a high-precision map, further improve data processing efficiency, and thereby enrich technical routes and increase selection space, a multi-source data processing method, device and equipment are provided in an embodiment of the present invention.

[0005] In a first aspect, an embodiment of the present invention provides a method for processing multi-source data, which may include:

[0006] Performing segmentation processing on each single-pass data included in the multi-source data to divide the single-pass data into a plurality of segment data;

[0007] constructing group data corresponding to the multi-source data based on the segment data;

[0008] Matching the segment data included in the group data to determine a matching relationship between the segment data;

[0009] Based on the matching relationship of the segment data, sampling the vehicle lane object and the semantic object included in the segment data to obtain an alignment sampling point of the segment data;

[0010] The segment data are aligned based on the alignment sampling points to achieve the multi-source data alignment.

[0011] In a second aspect, an embodiment of the present invention provides a multi-source data processing device, which may include:

[0012] A segmentation module, used for performing segmentation processing on each single-pass data included in the multi-source data, so as to divide the single-pass data into a plurality of segment data;

[0013] A grouping module, used for constructing group data corresponding to the multi-source data based on the segment data;

[0014] a matching module, configured to match the segment data included in the group data to determine a matching relationship between the segment data;

[0015] A sampling module, configured to sample the vehicle lane objects and semantic objects included in the segment data based on the matching relationship of the segment data, so as to obtain aligned sampling points of the segment data;

[0016] An alignment module is used to align the segment data based on the alignment sampling points to achieve the multi-source data alignment.

[0017] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-source data processing method as described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the multi-source data processing method as described in the first aspect is implemented.

[0019] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:

[0020] The embodiments of the present invention provide a method, device and equipment for processing multi-source data. The method mainly segments and groups data collected at different times from multi-source data, and provides stable segmented processing data for matching and aligning segment data in subsequent steps. At the same time, the present invention can establish a segment data topological network structure through the above-mentioned segmentation processing, and provide an effective and reasonable data structure for segment data matching and alignment.

[0021] The matching method solves the technical problem that the characteristics of segment data in the group data constructed by multi-source data are not obvious and cannot be matched normally or are matched incorrectly. By matching the segment data in the group data, it lays the foundation for aligning the semantic objects in the subsequent segment data, and thus can produce high-precision maps more accurately. At the same time, this method solves the technical problem that the errors of multi-source data are different, the absolute positions of semantic objects of the same road sign are scattered, and a large-scale over-clustering is required in the later clustering process, resulting in low data matching credibility.

[0022] Furthermore, by sampling objects in multi-source data, the alignment errors between objects are determined by sampling points, and the objects in the multi-source data are aligned based on the alignment errors, providing a basis for subsequent clustering of multi-source data. In specific implementation, the group data divided by multi-source data is used as a processing unit, which avoids the inconsistency of the first and last points of the multi-source data and improves the processing rate during data processing. The above method samples the vehicle line objects and semantic objects to determine the longitudinal error and lateral error between the multi-source data, which can effectively correct the absolute errors between the multi-source data due to sampling accuracy and multiple and multiple acquisitions, laying the foundation for the subsequent production of high-definition maps based on multi-source data.

[0023] Furthermore, this method can align the data within a group through intra-group constraints, and align the component data through component constraints, thereby eliminating the relative errors between multi-source data and ensuring the accuracy and reliability of map elements during reconstruction.

[0024] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0025] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0027] Figure 1 is a flowchart of a method for processing multi-source data;

[0028] Figure 2 It is a flowchart of segment processing;

[0029] Figure 3 It is a schematic diagram of the results of segmentation and grouping of multi-source data;

[0030] Figure 4 A flowchart for screening feature objects at intersections;

[0031] Figure 5 This is an example of an intersection segment feature object;

[0032] Figure 6 Flow chart for determining segmentation points and segmentation radius;

[0033] Figure 7 A schematic diagram of a set of intersection segment feature objects constructed in an embodiment of the present invention;

[0034] Figure 8 for Figure 7 Clustering effect diagram;

[0035] Fig. 9 for Figure 5 Schematic diagram of segmentation points obtained by clustering;

[0036] Fig.10 It is a schematic diagram of the segmentation effect;

[0037] Fig.11 This is an example of the segment set formed after segmentation;

[0038] Fig.12 This is the segment grouping effect diagram;

[0039] Fig.13 The renderings generated for bunch;

[0040] Fig.14 This is one of the schematic diagrams of projection matching between groups of data in an embodiment of the present invention;

[0041] Fig.15 The second schematic diagram of projection matching between groups of data in an embodiment of the present invention

[0042] Fig.16 Schematic diagram of inserting vector points for bunch;

[0043] Fig.17 A schematic diagram of the topological structure relationship between the established multi-source data;

[0044] Fig.18 It is a schematic diagram of segment data;

[0045] Fig.19 Schematic diagram made for line;

[0046] Fig. 20 A schematic diagram of grouping multi-source data;

[0047] Fig.21 A simplified diagram of the matching method of perception data in multi-source data;

[0048] Fig. 22 A flow chart of a method for matching perception data in multi-source data;

[0049] Fig.23 A specific process flow chart for determining the offset of the current segment data to be matched in the segment data set;

[0050] Fig.24 Determine a first offset projection diagram for the boundary attribute;

[0051] Fig.25 A flowchart for processing segment data in different groups of data;

[0052] Fig.26 One of the schematic diagrams for verifying the matching relationship;

[0053] Fig. 27 The second schematic diagram for verifying the matching relationship;

[0054] Fig.28 The third schematic diagram for verifying the matching relationship;

[0055] Fig.29 A flowchart of a method for sampling objects in multi-source data;

[0056] Fig.30 One of the flowcharts for sampling the car line objects based on the car line object set;

[0057] Fig.31 The second flowchart of sampling the car line objects based on the car line object set;

[0058] Fig.32 A detailed flowchart of sampling the car line objects based on the car line object set;

[0059] Fig.33 A schematic diagram for determining whether two lane objects intersect;

[0060] Fig.34 A schematic diagram for determining whether a lane object has a curved part;

[0061] Fig.35 This is a flow chart of performing curve sampling on the matched lane object in step S305;

[0062] Fig.36 This is the second flow chart of performing curve sampling on the matched lane object in step S305;

[0063] Fig.37 This is one of the flowcharts for determining the matching interval in step S351;

[0064] Fig.38This is the second flowchart of determining the matching interval in step S351;

[0065] Fig.39 A schematic diagram of determining a matching interval in step S351;

[0066] Fig.40 Flowchart for direct sampling of straight line objects;

[0067] Fig.41 It is one of the direct sampling processes of the straight line object;

[0068] Fig.42 This is the second direct sampling process for the straight line object;

[0069] Fig.43 The third is the direct sampling process of the straight line object;

[0070] Fig.44 Flowchart for sampling reference lines for straight line objects;

[0071] Fig.45 This is a schematic diagram of the reference line sampling of the straight line object;

[0072] Fig.46 One of the flow charts for sampling semantic objects;

[0073] Fig.47 Flowchart 2 for sampling semantic objects;

[0074] Fig.48 Execute the flow chart to determine the highest similarity region;

[0075] Fig.49 A schematic diagram for determining the outer bounding box of a graphic;

[0076] Fig.50 A schematic diagram of constructing a feature matrix of a semantic object;

[0077] Fig.51 A schematic diagram for determining a sliding search window;

[0078] Fig.52 Schematic diagram of semantic object feature matrix similarity calculation;

[0079] Fig.53 A schematic diagram is provided to explain the relevant technical terms;

[0080] Fig.54 A flowchart of the method for aligning multi-source data is provided;

[0081] Fig.55 A simplified diagram of the alignment method for the provided multi-source data;

[0082] Fig.56A flowchart of the detailed multi-source data alignment method is provided;

[0083] Fig.57 A flowchart for determining a list of windows to be optimized is provided;

[0084] Fig.58 This is an example of two segments within the provided group data;

[0085] Fig.59 Sample examples for the lane objects in the segment within the provided group data;

[0086] Fig.60 Sample examples of semantic objects in the segments within the provided group data;

[0087] Fig.61 Build a graph model instance for the segments in the provided group data;

[0088] Fig.62 This is the effect diagram of the adjustment after solving the constraint factors built based on the ICP objective function;

[0089] Fig.63 This is a diagram showing the effect of solving and adjusting the constraint factors constructed based on the GICP objective function according to an embodiment of the present invention;

[0090] Fig.64 The effect between bunches that occurs after segment alignment is performed for each bunch separately;

[0091] Fig.65 A schematic diagram constructed for the window to be optimized;

[0092] Fig.66 Schematic diagram of the constructed graph structure;

[0093] Fig.67 This is a diagram showing the effect of alignment between data groups in an embodiment of the present invention;

[0094] Fig.68 It is a structural schematic diagram of a multi-source data processing device. DETAILED DESCRIPTION

[0095] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0096] The technical terms described below in the embodiments of the present invention are explained as follows:

[0097] Object data (semantic): a collection of vector objects such as road markings and road entities collected from lanes (semantic objects such as markings, arrows, sidewalks, and lane objects).

[0098] Trajectory data: a collection of vehicle trajectory shape points.

[0099] Lane line data (line): A collection of line objects made based on a single lane semantic object.

[0100] Segment node definition (cross): Generally, intersections and divergence and confluence points in reality are used as segment nodes (segment points).

[0101] Bunch data: In reality, a section of road that passes through two (cross) nodes is defined as a road segment.

[0102] Segment data: The road segment information collected through a bunch at different time points is taken as an object.

[0103] The present invention provides a method for processing multi-source data. Figure 1 As shown, the following steps may be included:

[0104] Step S10: Segment each single-pass data included in the multi-source data to divide the single-pass data into a plurality of segment data.

[0105] Step S20: construct group data corresponding to the multi-source data based on the segment data.

[0106] Step S30: Match the segment data included in the group data to determine a matching relationship between the segment data.

[0107] Step S40: based on the matching relationship of the segment data, sampling is performed on the vehicle lane objects and semantic objects included in the segment data to obtain aligned sampling points of the segment data.

[0108] Step S50: align the segment data based on the alignment sampling points to achieve multi-source data alignment.

[0109] The above method provided in the embodiment of the present invention is mainly for segmenting and grouping data collected at different times from multiple sources, and providing stable segmented processing data for matching and aligning segment data in subsequent steps; at the same time, the present invention can establish a segment data topological network structure through the above segmentation processing, and provide an effective and reasonable data structure for segment data matching and alignment.

[0110] In an optional embodiment, the method may further include: step S60, clustering and aggregating based on the aligned multi-source data to aggregate and obtain map data including semantic objects and lane objects.

[0111] In another optional embodiment, before segmentation and grouping, it is also necessary to construct a topological relationship for the single-pass data, as follows:

[0112] (1) Obtain the trajectory data and semantic data of the vehicle, and obtain the arrangement order of the semantic data based on the observation time of a single trip of semantic data; (2) Fit each semantic data through multiple terms to obtain the vector arrow of each semantic data; (3) Select any semantic data as the starting semantic data, and explore from the end point of the vector arrow obtained by fitting the semantic data, and determine the first connectable semantic range by presetting a first distance range and a first angle range; specifically, based on the acquired trajectory data, obtain the shortest distance from the trajectory centroid to the trajectory: ① If it is greater than the preset shortest distance threshold, the trajectory is judged to be a curve. At this time, at the end point of the vector arrow, explore along the direction of the observation point of the trajectory point; ② If it is less than or equal to the preset shortest distance threshold, the trajectory is judged to be a non-curve. At this time, at the end point of the vector arrow, explore along the direction of the vector arrow. (4) Identify whether there is semantic data within the first connectable semantic range; if so, determine the semantic data as the semantic data to be connected; (5) If not, determine the second connectable semantic range by presetting the second distance range and the second angle range; (6) Identify whether there is semantic data within the second connectable semantic range; if so, determine the semantic data as the semantic data to be connected; specifically, if there are more than or equal to two semantic data to be connected identified at this time, the semantic data to be connected that is closest to the starting semantic data is preferred; further, if there are two or more semantic data to be connected that are closest and equal in distance, the semantic data to be connected that has the smallest angle with the starting semantic data is preferred; further, if there are still more than or equal to two semantic data after distance and angle identification, at this time, obtain the lane information of the current position based on the initial semantic data, and determine the center line of the lane information, preferably the semantic data farthest from the center line. (7) Determine whether it can be connected by the data type of the current semantic data and the data type of the target semantic data. If the types are the same, determine the connection relationship between the two, and establish the topological relationship between the starting semantic data and the target semantic data.

[0113] The method for processing multi-source data by segmentation and grouping in the embodiment of the present invention refers to Figure 2 and Figure 3 As shown, the following steps may be included:

[0114] Step S11: Determine the segmentation points and segmentation radius of the multi-source data grouping based on the intersection segmentation feature objects respectively identified in the multiple single-trip data.

[0115] In this step, we first screen out semantic objects of intersection feature types based on the attribute information of the semantic objects input from the vehicle side, then clean and screen the initially screened intersection feature objects to find feature objects that can represent intersection segmentation, and finally group and cluster the intersection segmentation feature objects to obtain segmentation points and corresponding segmentation radii for multiple groupings.

[0116] Step S12: segment the multiple single-trip data according to the segmentation points and the segmentation radius, so as to divide the single-trip data into multiple segment data.

[0117] This step is to segment multiple single-trip data based on segmentation points and segmentation radius, so that each single-trip data is divided into multiple segment data, preparing for grouping in subsequent steps.

[0118] The above method provided in the embodiment of the present invention is mainly for segmenting and grouping data collected at different times from multiple sources, and providing stable segmented processing data for matching and aligning segment data in subsequent steps; at the same time, the present invention can establish a segment data topological network structure through the above segmentation processing, and provide an effective and reasonable data structure for segment data matching and alignment.

[0119] In an optional embodiment, the identification of intersection segmentation feature objects in the single-trip data in the above-mentioned step S11 includes: first, determining the intersection feature objects in the single-trip data based on the attribute information of the semantic objects included in the single-trip data; then, based on the object type of the intersection feature objects and / or the distance between adjacent intersection feature objects, screening the intersection feature objects to obtain intersection segmentation feature objects.

[0120] In this embodiment, the semantic objects are screened out as intersection feature objects through the attribute information of the semantic objects, such as sidewalks, stop lines, sidewalks or arrow-type semantic objects near the stop line as intersection feature objects. For example, there are several types of intersection position determination: ① sidewalk (zebra crossing), ② stop line, ③ stop line + sidewalk at the same time, ④ stop line + nearby arrow, ⑤ sidewalk + nearby arrow. (Among them, the confidence of the combination type is higher). After processing multi-source data many times, the inventors have found that not all intersection feature objects can be used as intersection segmentation feature objects, and the selected intersection feature objects need to be further screened before the intersection segmentation feature objects can be determined.

[0121] In a specific embodiment, the intersection feature objects are screened based on the object type of the intersection feature object and / or the distance between adjacent intersection feature objects. Figure 4 As shown, the following steps may be included:

[0122] Step S41, determine whether the distance between adjacent intersection feature objects is less than a preset first distance threshold; if so, execute step S42; otherwise, end.

[0123] This step sorts the intersection feature objects obtained in the above steps by time, so that the approximate location of the intersection can be determined. Further screening is required to delete the intersection feature objects that may interfere with the segmentation. When this step is implemented, the distance is used for judgment. If the distance between two adjacent intersection feature objects is less than the preset first distance threshold, it is determined that the distance is too close, and the intersection feature objects are deleted (feature types include arrows, sidewalks, stop lines, etc., and the same type that is too close is considered to be in the same position, and only one feature is retained. When input on the vehicle side, each data will have a confidence level, and data with high confidence levels will be retained first).

[0124] Step S42, determine whether the object types of the feature objects of adjacent intersections are consistent; if consistent, execute step S43; if inconsistent, execute step S44.

[0125] Step S43: Screening the intersection feature objects based on the existence confidence of the adjacent intersection feature objects.

[0126] Step S44, screening intersection feature objects based on object type weight values ​​of adjacent intersection feature objects; wherein, in the embodiment of the present invention, the object type weight values ​​are arranged in the following order: sidewalk + stop line combination object, sidewalk + arrow combination object, stop line + arrow combination object, sidewalk object, stop line object, arrow object.

[0127] In this embodiment, for intersection feature objects that have both sidewalks and multiple types of arrows and stop lines, if the distance between these intersection feature objects and the intersection is within a preset distance threshold, they are considered to be features of the same intersection, and only one feature can be retained. At this time, the confidence value of each intersection feature object is obtained, and intersection feature objects with high confidence are retained first. For two intersection feature objects whose confidence difference is within the preset confidence threshold, they are deleted according to the pre-defined object type weight value rules. For example, the order of retained weights can be: sidewalk>stop line>arrow. Reference Figure 5 As shown in the figure, the solid circles are the intersection segment feature objects obtained after screening.

[0128] After selecting the intersection segmentation feature objects, it is necessary to determine the segmentation points and segmentation radius of the multi-source data grouping based on these intersection segmentation feature objects. Figure 6 As shown, the following steps may be included:

[0129] Step S60, based on the intersection segmentation feature object and the trajectory line in the single-trip data, the intersection segmentation feature object is vector-converted to convert the intersection segmentation feature object into an intersection segmentation vector point object. In this step, in order to more easily or quickly determine the segmentation point and segmentation radius, and the intersection segmentation feature object is not convenient for data processing as surface data or line data, in this embodiment, the intersection segmentation feature object obtained in the above steps must first be vector-converted. Since each single-trip data contains a trajectory line, the vector conversion can be performed based on the direction of the trajectory point in the trajectory line. Also refer to Figure 5 As shown, Figure 5 The solid circle in the middle is the converted intersection segment vector point object, which will be processed as a vector point object later.

[0130] Step S61, traverse the intersection segment feature objects in pairs, and determine whether the distance between two intersection segment feature objects is less than a preset second distance threshold; if so, execute step S62; otherwise, end.

[0131] In this embodiment, the distance of the intersection segmentation vector point objects in the multi-source data is determined, and the distance between any two vector point objects is confirmed in a pairwise traversal manner, and then the vector point objects that meet the distance threshold requirements are retained and density clustered (the DBSCAN density clustering algorithm is used). After clustering is successful, the vector point objects with distances within the threshold range are divided into one group. The second distance threshold in this embodiment is to divide the intersection segmentation vector points that are actually the same intersection into one group. The second distance threshold can be 5-12m, and this embodiment does not specifically limit the threshold.

[0132] Step S62: construct a set of intersection segment feature objects based on the intersection segment feature objects. Figure 7 As shown, a set of constructed intersection segment feature objects is shown.

[0133] Step S63: Density clustering is performed on the intersection segment feature objects included in the intersection segment feature object set, so as to divide a plurality of intersection segment feature objects that meet the third distance threshold requirement after successful clustering into a group of object clusters.

[0134] This step will cluster into several object clusters during clustering. The distance requirement during clustering is the third distance threshold. Vector point objects that meet the third distance threshold are grouped into one group as an object cluster. Figure 8 As shown, Figure 7 Clustering effect diagram. At the same time, multiple object clusters may correspond to the same intersection in the actual road, so the object clusters at the same intersection need to be fused. This step can also be determined based on the third distance threshold, and multiple object clusters within the third distance threshold range are fused.

[0135] Step S64: determining a segmentation point based on the cluster center position of the plurality of intersection segmentation feature objects in the object cluster, and determining a segmentation radius based on the segmentation point and the distribution range of the plurality of intersection segmentation feature objects in the object cluster. Fig. 9 As shown, in the embodiment of the present invention, the object cluster can be aggregated into a specific segmentation point. Fig. 9 The points in the diamond are used as segmentation points, that is, the center points of the cross object.

[0136] In another optional embodiment, the present invention also needs to delete segmentation points that are too close, as follows: determine whether the distance between adjacent object clusters is less than a preset fourth distance threshold; if so, delete the object cluster with fewer intersection segmentation feature objects in the adjacent object clusters; if the number of intersection segmentation feature objects in the adjacent object clusters is the same, delete any one of the adjacent object clusters; and / or determine whether the number of intersection segmentation feature objects included in the object cluster is less than a preset number threshold; if so, delete the object cluster.

[0137] In the embodiment of the present invention, deletion is performed for the case of being too close. Usually, there can be two groups of features within a range of 50 meters (when the distance is too close, it is not convenient to use the subsequent matching alignment. When deleting, it can be determined according to the number of objects in each cross group. The group with more objects in the group is retained first. If the number of objects in two groups is the same, one of them is randomly deleted); in the clustering results, the group with too few objects in the group is deleted. Usually, the group with less than two objects in the group is considered unreliable. Usually, the vehicle passes by many times and only recognizes that the number of segment vector points is too small (refer to Figure 6 If the group is not large enough to generate a cross, such group will be deleted. The threshold is usually used to determine whether it meets the standard.

[0138] In an optional embodiment, also refer to Figure 2 As shown, the above method may further include the following steps:

[0139] Step S13: Generate group data corresponding to the multi-source data based on the segment data divided by the single-pass data, and establish an association relationship between the group data and the segment data.

[0140] In the embodiment of the present invention, by inputting whether the vehicle-side single-trip driving trajectory passes through the cross position, the cross set of the trajectory passing through the intersection is screened (with each cross as the center, the trajectory passing within the radius is considered to pass through the cross, and the trajectory not passing through the cross is not split, refer to Fig.10 The figure shows a schematic diagram of segmentation based on segmentation points (cross / intersection vector points). Fig.11As shown in the figure, in an actual example, the segment set is formed after segmentation, where the segment set is between two dots.

[0141] In this step, the segment data corresponding to the same section of the actual road are divided into the same group data (bunch). When processing multi-source data later, the bunch is used as the minimum processing unit to produce a high-precision map. When this step is implemented specifically, first, the multiple segment data corresponding to the same road section are divided into a group data according to the direction and position of the segment data; then, an association relationship between the group data and the segment data it includes is established, so that the segment data carries the corresponding group data attributes. That is, the division principle in this embodiment is that the segment vectorization direction is the same (single vehicle driving direction), and the segment spatial distance is within a certain range (height, plane distance, etc.); after the bunch object is generated, an association relationship is established with the segment, so that the segment carries the corresponding bunch attribute. In the embodiment of the present invention, a group data topological network structure is established based on segmentation and grouping, which provides an effective and reasonable data structure for subsequent data matching and alignment. Refer to Fig.12 and Fig.13 As shown, after segmentation and segment processing, segment data and group data are obtained.

[0142] In another optional embodiment, also refer to Figure 2 As shown, after the group data are generated, the following steps may be further included: Step S14, comparing the projection distance between the group data with a preset projection distance threshold, so as to merge at least two group data into one group data based on the comparison result.

[0143] The inventors found that when multiple trips are collected on the same road, due to the lateral error in the driving direction, duplication occurs when generating a bunch. Therefore, this embodiment matches the bunches to find such repeated bunches and merge them into one bunch object. Fig.14 and Fig.15 As shown, in this embodiment, the average difference is calculated by segmented equal intervals. If the distance between the equally spaced bunches meets the preset distance threshold, they are considered to be the same bunch; if the distance between the equally spaced bunches does not meet the preset distance threshold, they are considered to be different bunches. In specific implementation, the standard deviation can be used for judgment.

[0144] In another optional embodiment, after generating the group data, the following steps may also be included: determining whether the length of the group data is greater than a preset length threshold; if so, inserting an intra-road segmentation feature object between adjacent intersection segmentation feature objects of the single-trip data, and determining the segmentation point and segmentation radius of the multi-source data grouping based on the intra-road segmentation feature object.

[0145] In this embodiment, after the bunch is generated, if the bunch is too long, it will affect the use of subsequent data, affect the matching and alignment efficiency, and the correct matching and alignment effect. Therefore, the overlong bunch objects are identified, and segment features (for example, high-speed scenes) are inserted equidistantly and added to the sub-features, and then the above-mentioned steps of filtering the grouped feature objects are performed. Fig.16 As shown in the figure, if the obtained bunch exceeds a certain length, it will be segmented by inserting points directly. Fig.16 The dot on the center line), after insertion, these segment features are filtered and deleted as above to get the point, and then get the cross and other steps to identify the trajectory, segment it, and finally match the bunch.

[0146] In another optional embodiment, the method may further include: based on the association relationship between the group data, segment data and segmentation points in the multi-source data, associating the semantic objects, vehicle line objects and track lines in the multi-source data to establish a topological structure relationship of the multi-source data. Fig.17 As shown, a schematic diagram of the topological structure relationship between the multi-source data is established, so that the front and back relationships of the bunch, as well as the segments within the bunch, the semantic objects and lane objects in the segments, etc. can be quickly located.

[0147] In another optional embodiment, the method may also include: a topological structure relationship of multi-source data, constructing a group data endpoint set, and determining the matching priority of the group data obtained by grouping the multi-source data based on the discreteness of the group data endpoint set. In this embodiment, multiple cross objects output a cross set, and the cross set refers to a set of node objects generated by grouping for segmenting the road section. In subsequent steps, a topological relationship of the cross object can be established, and the matching priority when the signal is not good can be determined based on the discreteness during matching alignment.

[0148] Reference Fig.18 As shown, a single trip of data consists of several connected segments, each of which contains a semantic object and a lane object. Multiple trips of segment data at the same position can be combined into a bunch of data, that is, each single trip of data is segmented based on the same processing mechanism, and the segment data at the same position are combined into a bunch of data.

[0149] Before segmentation, preprocess the single-trip data separately, and establish the position relationship in the horizontal and vertical directions based on the line information, that is, the lane object information. In the specific implementation, the lane object in the vertical direction is made into a line (i.e. the vertical relationship), and the lane object (solid line, dotted line, etc.) in the single-trip data is mounted on the line. The absolute position relationship between the semantic object (identification semantic object, such as arrow, zebra crossing, etc.) in a segment data and the lane object is known, and the semantic objects in the multi-source data can be aligned based on each other. Fig.19 As shown in the figure, the vertical thin line is the produced line. In the same frame of data, the absolute relationship between the above-mentioned lane object and the semantic object is known, so the position relationship of the semantic object in a single trip data can be calibrated. After the calibration is completed, due to the large amount of data and the different starting and ending points between each single trip route, it is not possible to use one trip route as a whole for sampling and alignment. At this time, the single trip data needs to be segmented. In the specific implementation, the single trip data is divided into several segments according to the positions of preset signs such as intersections, zebra crossings, stop lines, etc.

[0150] After the multi-source data is segmented, it is grouped, that is, the multi-source data establishes a group relationship, and multiple single-trip data representing the same section of road are divided into one group. A topological relationship is established between adjacent groups, and each group can be written with an ID based on the longitudinal extension characteristics of the line. Fig. 20 As shown in the figure, the segments at the same position will form a group data. The endpoints of the group data are crosses. The cross represents the position point of the end of the bunch. The relationship of the cross can be used to determine which bunch it belongs to, and the relationship between adjacent bunches can also be determined.

[0151] When matching perception data in multi-source data, refer to Fig.21As shown in the figure, the conventional means are as follows: first, a road network is established according to the topological relationship of the bunch, and the starting group data bunch on each road network is found; then, the matching relationship of the lines between the segments in the starting group data bunch is transferred to the forward and backward associated segments through the tracking ID of the line. The purpose of this is to transfer the matching relationship of the position with good signal to the position with poor signal, because the semantic object of the position with poor signal may be severely deformed and cannot be matched conventionally. The starting group data bunch defined above refers to the overlapping matching of the segments in all the bunches on a single road network, and then a data group bunch is selected as the starting group data bunch. The segment signals in this group bunch (here refers to the signal information on the trajectory, which can be directly obtained) are all good, the matching relationship is the most complete and the score is the highest. The matching of segments will have a score, and the score here refers to the average score of all matching relationships on the bunch). When the tracking ID of the line cannot be found in the previous and next segments, the transmission is stopped, and then the matching scheme provided in the embodiment of the present invention is used until the segment data in all the group data are matched, that is, the multi-source data (multiple vehicle and multiple perception data) matching is completed.

[0152] In the embodiment of the present invention, the matching method is executed at least once. In order to ensure the accuracy of data matching, the matching can be repeated multiple times, for example, three times or more. Multiple matching cycles can match the segment data that has no matching relationship at the first time again according to the matching relationship between the previous and next segments to achieve the best matching effect. Fig.21 and Fig. 22 As shown, the method may include:

[0153] Step S221: Determine associated group data based on the discreteness of segment endpoint positions in group data constructed from multi-source data, so as to construct a segment data set based on segment data included in the associated group data.

[0154] This step is to sort the crosses obtained by clustering in the group data to determine the degree of discreteness of the endpoint positions in the single-pass data. In the specific implementation, the discreteness of the segment endpoint positions is sorted from small to large, so as to associate the segment endpoint group data with the smallest discreteness in the segment endpoint positions and determine it as the associated group data. The purpose of sorting is to ensure the stability of the matching result output; secondly, to process the bunch with small error first, and then process the bunch with large error, so that the matching results of the previous and next segments can be better used. This step needs to traverse the cross (each cross is mounted with bunch information, such as the bunch starting point, bunch end point, etc.) to obtain the associated bunch. It should be noted that in this embodiment, the associated group data can be one or more, and then the segment data set is constructed based on the associated group data.

[0155] Step S222: Determine the offset of the current segment data to be matched in the segment data set.

[0156] In this step, the offset of the segment data is determined based on the positional relationship between the two segments of data. It can be based on boundary information, special semantic object information, vehicle line semantic object information, or matching relationship information between the front and rear connected segment data. The embodiments of the present invention will be described in detail later.

[0157] Step S223: Move any segment of the current segment data to be matched based on the offset and determine the matching rate of the current segment data to be matched, so as to determine the matching relationship of the segment data where the perception data in the multi-source data is located based on the matching rate.

[0158] This step calculates the matching rate of the matching segment data after moving based on the offset. For example, it determines whether the match is successful in different dimensions such as the distance, area difference (length difference), type, height, shape, etc. of the semantic objects included in the segment data, so as to finally determine the matching relationship of the segment data where the perception data is located in the multi-source data.

[0159] The matching method of the perception data in the above-mentioned multi-source data provided in an embodiment of the present invention is used to solve the problem of inconsistent errors in multi-source data when multi-source crowdsourcing data is used to produce high-precision maps, avoid the problem of excessive clustering caused by the dispersion of absolute positions of objects (perception data) representing the same ground element, and thus avoid the inability to form a single object normally after excessive clustering. This matching method lays the foundation for the subsequent adjustment (alignment) of object positions in multi-source data for the production of high-precision maps.

[0160] In an optional embodiment, when constructing a segment data set based on the segment data included in the associated group data in step S221, it is necessary to determine whether the associated bunch is a forward and reverse bunch (up and down roads) at the same location; if it is a forward and reverse bunch, the segments on the two bunches are taken out for matching to obtain segment set 1; if it is not a forward and reverse bunch, the associated bunches are traversed to perform segment matching within a single bunch to obtain segment set 2. This avoids constructing the same segment data set from data collected in different places, which will result in a very low matching success rate.

[0161] In another optional embodiment, before determining the associated group data, the following operation needs to be performed: based on the vehicle line tracking ID in the single-trip data, the segment data divided by the single-trip data are matched and transferred to determine the matching relationship of the segment data within the group data constructed by multi-source data and the matching relationship of the segment data in adjacent groups of data; if the matching relationship of the segment data cannot be determined based on the vehicle line tracking ID, the associated group data is determined based on the discrete degree of the segment endpoint positions in the group data constructed by multi-source data.

[0162] That is, before the implementation of the embodiment of the present invention, there is a judgment process of whether to perform conventional matching based on the vehicle line tracking ID or to use the unconventional matching in the embodiment of the present invention. If the vehicle line tracking ID signal is good, the vehicle line tracking ID is directly used to match the segment data within and between groups, which is less time-consuming and improves the matching efficiency.

[0163] In another optional embodiment, after constructing the segment data set, it is also necessary to extract matching elements of each segment data in the segment data set; wherein the matching elements include at least one of the following: boundary attribute information, landmark semantic objects, the number of lanes in the segment data, and vehicle line information.

[0164] In this embodiment, after constructing the segment data set, the subsequent steps need to match the segment data, and some matching elements are required for matching, such as extracting internal features such as boundary information, semantic object information, line information, etc. in each segment as matching elements.

[0165] When the embodiment of the present invention is implemented, the features are collected and the accuracy of the features is screened. For example, for lines, shorter line data will be filtered. For semantic objects, semantic objects that are conducive to matching, such as arrows and guide strips, will be collected, while semantic objects with fewer common features will no longer be collected. For boundary information, because the boundary information in the perception data may be misidentified or discontinuous, on the one hand, it is necessary to find the outermost line with boundary information, and connect the shape of the line as the boundary shape according to the tracking ID and timestamp; on the other hand, it is necessary to make a rough assessment of the number of lanes of all segments in the same bunch, and judge how many lanes there are in the current bunch position (take the mode of the number of lane lines of the segments on the bunch), and only retain one side of the boundary of the segment that is less than the number of lanes in the current bunch (there is a situation in the data where there is one less lane but there are boundaries on both sides. Retaining one side can reduce matching conflicts. At the same time, because the boundary information is less credible at this time, it can also be combined with the matching results of the previous and next segments for more accurate matching).

[0166] In another optional embodiment, since the segment data in the embodiment of the present invention are matched in pairs, if a segment data does not extract a matching element with matching significance, the segment data cannot perform the subsequent matching process. Therefore, after extracting the matching element of each segment data in the segment data set, the segment data without matching elements in the segment data set needs to be filtered. The purpose of filtering in this embodiment is to directly skip the subsequent matching steps for the segment data without matching significance, thus saving the subsequent matching operation amount and saving the operation space to improve the matching operation efficiency.

[0167] In another optional embodiment, the specific process of determining the offset of the current segment data to be matched in the segment data set in the above step S222 is as follows: Fig.23 As shown, the following steps may be included:

[0168] Step S230: determining a segment data matching order in the segment data set based on the boundary attribute information in each segment data, so as to perform pairwise matching on the segment data included in the segment data set based on the segment data matching order until all segment data are matched; wherein the boundary attribute information may include: a distance between a lane boundary and a trajectory line, and a lane boundary length.

[0169] This step sorts the segment data matching priority according to the lane boundary length and / or the distance between the lane boundary and the trajectory corresponding to the segment data in the segment data set, and then loops through the segment data for pairwise matching. Since different segments in the segment data set correspond to different lane boundary lengths, and the distance between the starting point, middle point, and end point of each line and the trajectory is calculated when making a line, and recorded in the lane information, this step prioritizes matching of segment data with long lane boundaries and close to the trajectory.

[0170] Step S231, determine whether there is a matching relationship between the adjacent segment data of the current segment data to be matched in the segment data set; if there is no matching relationship, execute step S232; if there is a matching relationship, execute step S232 and / or step S233.

[0171] Step S232: Determine a first offset based on the boundary attribute information in the current segment data to be matched.

[0172] In this step, the first offset may be an offset determined by the right boundaries of the two segments of data, may include an offset determined by the left boundaries, or may be an offset determined by both the left and right boundaries. Fig.24 As shown, the offset is calculated by projecting two lines. The line shape in segment2 is line2_geo, and the line shape in segment1 is line1_geo. First, determine whether the point closest to line2_geo can be projected onto line1_geo; if not, continue to determine whether the first point of line2_geo can be projected onto line1_geo, if so, determine the projection point; if not, continue to determine whether the last point of line2_geo can be projected onto line1_geo, if so, determine the projection point; if not, continue to determine whether the point at 1 / 2 of line2_geo can be projected onto line1_geo, if so, determine the projection point; if not, use the first point of line2_geo to project onto the trajectory line of segment2, connect the first point of line2_geo and its projection point on the trajectory line of segment2 to form line2_location_line, select the point of line1_geo closest to line2_location_line to project onto line2_location_line, and move line1_geo to the projection position. Select the point on line2_geo that is closest to the moved line1_geo, project it to the moved line1_geo, and determine whether it can be projected onto the line. If so, determine the projection point, and then use the projection point and the selected projected point to calculate the first offset. If not, there is no lateral offset, and the offset cannot be calculated at this time, so no matching is performed.

[0173] Step S233: determining a second offset based on the vehicle lane information in the connected segment data.

[0174] It should be noted that in this step, since there is a matching relationship between adjacent segment data, there may be multiple closest matching relationships in the adjacent segment data, such as boundary matching, semantic object matching, etc., so it is necessary to jointly calculate the average offset based on multiple matching relationships.

[0175] It should be noted here that when it is determined that a matching relationship exists in step S231, the first offset can be obtained based on the current segment data's own information, or the second offset can be obtained based on the connected segment data. In the subsequent moving step, the movement can be based on the average of the first offset and the second offset. This embodiment does not specifically limit this.

[0176] It should also be noted that in the embodiment of the present invention, after the pairwise matching is completed, the following steps need to be performed, namely: the segment data that has not been successfully matched in the segment data set is recorded, and after all other segment data in the segment data set are matched, the recorded segment data is re-matched. When performing pairwise segment matching, the order of matching will affect the final matching success rate. To shield this problem, the following operation is performed: the segments that have failed to match will be recorded first, and after all segments in the segment set are matched, re-matching will be performed.

[0177] For example, if there are five segments in the segment set, which are sorted as [1, 2, 3, 4, 5], first take 2 out of the matched set a, a = [2].

[0178] The to-be-matched set b = [1,3,4,5], and the unmatched set c = []; traverse b and match it with the last element of the a set. If successful, put it into a, otherwise put it into c.

[0179] 1 and 2 match successfully, a=[2,1], c=[];

[0180] 3 matches 1 successfully, a=[2,1,3], c=[];

[0181] 4 and 3 do not match successfully, a=[2,1,3], c=[4]

[0182] 5 does not match 3, a=[2,1,3], c=[4,5]

[0183] Then assign the unmatched set c to the to-be-matched set b, clear the unmatched set, b=[4,5], c=[]

[0184] Then match the to-be-matched set b with all the segments of the matched set a until the match is successful. If it is not successful yet, determine whether the length of the matched set a has changed (to determine whether new data has been added to the matched set a). If there is a change, match again. If there is no change, end the matching of the current segment set.

[0185] Case 1: Matching is successful

[0186] 4 does not match 2 successfully, a=[2,1,3], c=[]

[0187] 4 matches 1 successfully, a=[2,1,3,4], c=[]

[0188] 5 does not match 2 successfully, a=[2,1,3,4], c=[]

[0189] 5 does not match 1 successfully, a=[2,1,3,4], c=[]

[0190] 5 does not match 3 successfully, a=[2,1,3,4], c=[]

[0191] 5 matches 4 successfully, a=[2,1,3,4,5], c=[]

[0192] Case 2: 4 is not matched successfully

[0193] 4 does not match 2 successfully, a=[2,1,3], c=[]

[0194] 4 does not match 1 successfully, a=[2,1,3], c=[]

[0195] 4 and 3 do not match successfully, a=[2,1,3], c=[4]

[0196] 5 does not match 2 successfully, a=[2,1,3], c=[4]

[0197] 5 matches 1 successfully, a=[2,1,3,5], c=[4]

[0198] It is found that the length of set a has changed, so the matching continues, and the unmatched set c is assigned to the to-be-matched set b, and the unmatched set is cleared, b=[4], c=[]

[0199] 4 does not match 2 successfully, a=[2,1,3,5], c=[]

[0200] 4 does not match 1 successfully, a=[2,1,3,5], c=[]

[0201] 4 does not match 3 successfully, a=[2,1,3,5], c=[]

[0202] 4 and 5 match successfully, a=[2,1,3,5,4], c=[]

[0203] Case 3: The length of a does not change, and the match ends.

[0204] 4 does not match 2 successfully, a=[2,1,3], c=[]

[0205] 4 does not match 1 successfully, a=[2,1,3], c=[]

[0206] 4 and 3 do not match successfully, a=[2,1,3], c=[4]

[0207] 5 does not match 2 successfully, a=[2,1,3], c=[4]

[0208] 5 does not match 1 successfully, a=[2,1,3], c=[4]

[0209] 5 does not match 3, a=[2,1,3], c=[4,5]

[0210] There is a matching relationship between 2, 1, and 3, but there is no matching relationship between 4 and 5.

[0211] After determining the offset of the current segment data to be matched, the above steps S232 and S233 move any segment data in the current segment data to be matched based on the first offset and / or the second offset. The specific process of the offset of the current segment data to be matched provided in the embodiment of the present invention can accurately calculate the offset between the matching segment data and provide data support for subsequent movement.

[0212] In another optional embodiment, the segment data in different groups of data are processed before determining the offset of the current segment data to be matched in the segment data set. Fig.25 As shown, the following steps may be specifically included:

[0213] Step S251, determine whether the current to-be-matched segment data belongs to different groups of data, and the directions of the group data are opposite; if so, execute step S252; otherwise, execute step S253.

[0214] Step S252: determine whether there is an isolation element in the current segment data to be matched; if so, execute step S253; if not, execute step S254.

[0215] Step S253: The current segment of data to be matched is no longer matched.

[0216] Step S254: rotate any segment of the current segment data to be matched, and determine the offset with the rotated segment data. In specific implementation, the left and right boundaries of any segment data are rotated 180 degrees, or the center point of the segment data is rotated, which is not specifically limited in this embodiment.

[0217] The embodiment of the present invention processes segment data with hard isolation elements and belonging to different bunches, wherein the isolation elements include guardrails, curbs, etc. This is to more conveniently match different single-trip data to improve matching efficiency.

[0218] In another optional embodiment, determining the matching rate of the current segment data to be matched includes: determining the matching rate based on the matching dimensions of the landmark semantic objects included in the current segment data to be matched, and the weights corresponding to the matching dimensions; wherein the matching dimensions include at least one of the following: distance between objects, area difference between objects, length difference between objects, consistency of object types, height difference between objects, and similarity of object shapes.

[0219] In specific implementation, the matching rate can be determined by weighted scoring, with corresponding weights of (0.25, 0.1, 0.3, 0.15, 0.2), and the score threshold for successful matching is 0.6. For semantic object groups whose scores meet the preset threshold conditions, they are recorded as successfully matched element groups. For example, if there are six semantic objects on a segment, then in the most perfect case, six successfully matched element groups can be obtained, and the matching rate is 100%. The above-mentioned first offset and / or second offset movement will respectively obtain the overall matching rate of the semantic object and the score value of each group of semantic objects. The group with the highest matching rate is preferentially selected for output, and when the difference in matching rates meets the preset threshold range, the group with the highest score will be output.

[0220] In another optional embodiment, after determining the matching relationship of the segment data, it may also include: verifying the matching relationship based on the boundary attribute information and / or vehicle lane information in the segment data; wherein the vehicle lane information includes: vehicle lane type and vehicle lane tracking ID.

[0221] This step is to check and verify a set of output matching relationships. The purpose is to check the matching results. For example, if the matching results of segment1 and segment2 are incorrect, it is found that there may be a wrong match, and a matching failure will be returned without recording the matching results of the current two segments. It should be noted that the following methods can use a preset threshold method to determine whether there is a matching error, and the embodiment of the present invention does not specifically limit the size of the threshold.

[0222] Reference Fig.26As shown in , it is determined whether there is a phenomenon of matching lanes, that is, the two segments each have an outermost lane line on different sides that is not matched, such as Fig.26 As shown in the figure, the lane line on the left side of segment1 and the lane line on the right side of segment2 are not matched, which is a case of matching lanes. To check whether a lane is matched, each of the two segments has an extra lane. First, collect the tracking IDs (trace_id) of the leftmost line and the rightmost line in each frame; then, determine whether there is a matching relationship between the collected tracking IDs of the leftmost line and the rightmost line; third, if there is no matching relationship on the left side of segment1 and no matching relationship on the right side of segment2, and there is a matching relationship on the right side of segment1 and there is a matching relationship on the left side of segment2, it is judged as a wrong match; if there is no matching relationship on the right side of segment1 and no matching relationship on the left side of segment2, and there is a matching relationship on the left side of segment1 and there is a matching relationship on the right side of segment2, it is judged as a wrong match.

[0223] Reference Fig. 27 As shown, check whether there are a large number of matches between the solid line and the dotted line. Fig. 27 As can be seen, the solid line in segment 1 matches the dashed line in segment 2, and there may be a matching error. In the specific implementation, check whether the solid lines and dashed lines match a large number of line_type: Lane lines contain object types (1: solid line, 2: dashed line, 3: solid line and dashed line). First, check the matching relationship of Line and record the number of matching groups of solid lines and dashed lines; then, if the ratio of the number of matching groups of solid lines and dashed lines to the total number of matching groups exceeds 0.3, it is judged as a wrong match.

[0224] Reference Fig.28 As shown, it is determined whether there is a matching conflict (different lane lines in a single segment match each other). If a line in segment1 matches two lines in segment2, there is a matching conflict. If the check passes, the matching relationship is recorded and the match is successful; if the check fails, the matching relationship is cleared and the two segments fail to match. In specific implementation, the line matching relationship on a single segment is first collected; then, it is determined whether the lines with matching relationships have the same frame and different tracking IDs. If so, it is determined to be a wrong match.

[0225] Reference Fig.29 As shown, sampling of multi-source data objects may include the following steps:

[0226] Step S291: Based on the multiple segment data included in the group data divided from the multi-source data, the vehicle line objects with matching relationships and the semantic objects with matching relationships included in the segment data are combined in pairs to respectively construct a vehicle line object set and a semantic object set of the group data.

[0227] Step S292: based on the car line object set and the semantic object set, sample the car line objects and the semantic objects respectively, so as to determine the object alignment error based on the sampling results.

[0228] In an embodiment of the present invention, object sampling is performed on multi-source data, and the alignment error between objects is determined by sampling points, so that objects in the multi-source data are aligned based on the alignment error, providing a basis for subsequent clustering of multi-source data. In the specific implementation of the embodiment of the present invention, the group data divided by multi-source data is used as a processing unit, which avoids the inconsistency of the first and last points of the multi-source data and improves the processing rate during data processing. The above method samples the vehicle line objects and semantic objects to determine the longitudinal error and lateral error between the multi-source data, which can effectively correct the absolute errors between the multi-source data caused by sampling accuracy and multiple and multiple acquisitions, laying the foundation for the subsequent production of high-definition maps based on multi-source data.

[0229] In an optional embodiment, the above-mentioned sampling of the car line object based on the car line object set is performed, referring to Figure 30-Figure 32 As shown, the following steps may be included:

[0230] Step S301, traverse each set of two-way matching car line objects in the car line object set, and determine whether the matching car line objects have an intersection; if there is no intersection, execute step S302; if there is an intersection, execute step S303.

[0231] In this step, firstly, the direction vectors of the two matching car line objects are determined based on the trajectory lines corresponding to the car line objects; then, based on the direction vectors of the car line objects and the endpoint positions of the car line objects, it is determined whether the matching car line objects have an intersection. Fig.33 As shown, to determine whether two directional lines intersect in a plane, the concept of vectors is needed. For two lines L1 and L2, assume that their direction vectors are v1 and v2 respectively. If the two lines intersect in a plane, then their intersection point P can be expressed as: P = t1*L1+t2*L2. Among them, t1 and t2 are distance parameters, indicating the distance of point P to L1 and L2. Convert the above formula into vector form and we get: P = t1*v1+t2*v2. If the two lines are parallel or coincident, t1 and t2 have only one value, which means that P has only one position. If the two lines have no intersection, t1 and t2 have no common solution, that is, point P does not exist in the plane. Therefore, we can check whether the two lines have an intersection by solving this equation.

[0232] The solution to the above equation is: 1) Calculate the cross product of L1 and L2. If the cross product is zero, then the two lines are parallel or coincident. 2) If the cross product is not zero, then use the direction of the cross product to determine the signs of the two parameters t1 and t2. If the dot product of v1 and v2 is positive, then both t1 and t2 are positive; if the dot product is negative, then both t1 and t2 are negative. 3) Divide the cross product by the magnitude of v1 and v2 to get a vector w that is perpendicular to L1 and L2. Normalize w to get a new vector w_unit. 4) Multiply the w_unit vector by the L1 and L2 vectors respectively to get the two distance parameters t1 and t2. If both distance parameters have real solutions, then the two lines intersect. Refer to Fig.33 As shown, the above two car line objects are projected along the direction perpendicular to the vector. Simply judging that there is no intersection point can determine that the above two car line objects have no intersection. Fig.33 In the middle left image, the projections of the two endpoints of the short car line object have an intersection, so there is an intersection; in the right image, the projections of the two endpoints of the short car line object have no intersection, so there is no intersection, and this car line object is no longer sampled.

[0233] Step S302: traverse other groups of matching car line objects in the car line object set.

[0234] Step S303, determine whether the matched lane object has a curved part; if no curved part exists, execute step S304; if a curved part exists, execute step S305.

[0235] In the specific implementation of the embodiment of the present invention, an expansion algorithm is used to determine whether a lane has a curved portion, for example, referring to Fig.34 As shown, first, the preset radius range is expanded outward based on the axis line of the car line object to obtain the first area; second, the first and last points of the car line object are connected to obtain the first and last lines of the car line object; third, the preset radius range is expanded outward based on the first and last lines to obtain the second area; fourth, based on the first area and the second area, the intersection area is determined to determine whether there is a curve part in the car line object based on the intersection area and a preset ratio threshold.

[0236] In step S303 of the embodiment of the present invention, the straightness of the car line object is determined based on the threshold value, the car line object is expanded with a specified radius to form a figure, and then the line connecting the first and last points of the car line object is expanded with a specified radius to form a figure, and it is determined whether the intersection area of ​​the two figures exceeds the specified threshold to determine whether the car line object includes a curved part. In a specific example, the car line object itself is used as the axis line, and the radius is one meter, and the outer extension is performed; then, a straight line is connected between the two endpoints of the line segment, and the straight line is expanded one meter to the left and right respectively; if the ratio of the intersection area of ​​the two at the endpoints is greater than 0.95, the car line object is judged to be a straight line; if it is less than or equal to 0.95, it is judged that the car line object has a curved part.

[0237] Step S304: Line sampling is performed on the car line object. In the embodiment of the present invention, when line sampling is performed on the straight line portion of the car line object, a point-to-line projection method is used to form sampling points, which will be described in detail below.

[0238] Step S305: performing curve sampling on the curve part of the lane object.

[0239] In a specific embodiment, in the above step S305, curve sampling is performed on the matched car line object. When sampling a straight line, the sampling points are formed by point-to-point projection. However, using point-to-point projection on the curve part will produce erroneous lateral errors, so it is necessary to use unconventional sampling methods. That is, two matched car line objects are used as input, and the matching interval between the two matched car line objects is found. The two curve parts are respectively fitted with ellipses according to their own matching intervals. The feature points on one line are taken to find the corresponding matching points on the other curve. The center of the ellipse obtained by fitting is used to calculate the vector from the center of the circle to the feature point. Based on this vector, the corresponding matching point is found on the other curve. In this way, the curve sampling is completed. Refer to Fig.35 and Fig.36 As shown, the following steps may be specifically included:

[0240] Step S351: Based on the curve part of the lane object, determine the curve segment corresponding to the matching interval of the curve part. In this step, finding the lane line matching interval is mainly divided into four steps, namely: curve segmentation, constructing a curve feature matrix, matching interval search, and determining the best matching interval.

[0241] Step S352: performing ellipse fitting based on the matching interval to obtain two ellipses respectively.

[0242] Step S353: Taking the characteristic points on the curve segment corresponding to a lane object where the direction changes as sampling points, and constructing a matching vector with the center of the fitted ellipse and the sampling points.

[0243] Step S354: Use the center of the ellipse and the matching vector of the curve segment corresponding to the other car line object to fit the ellipse, and determine the matching point on the curve segment corresponding to the other car line object.

[0244] Step S355: Use the sampling points and the matching points as the alignment sampling points of the lane object.

[0245] In the embodiment of the present invention, when ellipse fitting is performed on two matching curve segments, a feature point on a certain line is used as a reference point, and a vector from the feature point to the center of the ellipse is calculated (in a local coordinate system with the center of the ellipse as the origin, the minor axis as the x-axis, and the major axis as the y-axis). Based on this vector, the position corresponding to the feature point is restored on the center of the ellipse of another curve fitting to obtain a set of aligned sampling points.

[0246] In a specific embodiment, referring to Fig.37 , Fig.38 and Fig.39 As shown, determining the matching interval of the curve portion may specifically include the following steps:

[0247] Step S371: based on the curvature of the feature points included in the car line object, intercept a curve portion in the car line object.

[0248] It should be noted that when generating a car line object, the coordinates of the points where the direction changes are retained as feature points on the line. Connecting multiple feature points constitutes a car line object. For lines that do not change direction, only the first and last points are retained. Therefore, when constructing the feature matrix of the car line object, there is no need to perform sampling to generate points that describe the features of the car line object. Fig.39 As shown in (1) and (2), the curvature of the feature points on the car line object can be used to divide it into a straight line part and a curve part according to the curvature change. The straight line part uses straight line sampling, and the curve part uses curve sampling.

[0249] Step S372: based on the extreme value point of the curvature change of the curve part, the curve part is segmented to obtain curve segments. This step calculates the curvature of each feature point on a curve, and each feature point and the previous feature point are used to obtain the curvature change of the feature point. The curve segmentation is performed at the extreme value point of the curvature change to obtain a number of curve segments. Fig.39 As shown in (3), the curve part is divided into curve segments ①-⑤.

[0250] Step S373: construct a curve segment feature matrix based on the feature points on the curve segment whose directions have changed.

[0251] Assume that there are n characteristic points on a curve segment T, which are There are m characteristic points on another curve segment Q, which are Establish the feature matrix M respectively Tand M Q , constructed using the distance between each point on the line and all points. To establish the feature matrix M T For example:

[0252]

[0253] in

[0254] choose As a starting point, The local characteristic matrix of the curve segment is obtained as the termination point.

[0255]

[0256] New feature matrix By changing M T The rows in cyclically shift up (k-1) rows, and the columns in cyclically shift up (k-1) columns are obtained. Therefore, any feature point is selected as the starting feature point, and the corresponding feature matrix can be obtained by using cyclic shift.

[0257] Step S374: matching is performed based on the curve segment feature matrix, and the matching interval of the curve portion of the lane object is determined using the curvature parameters and distance parameters of the two groups of feature points.

[0258] If there is a complete match between curve segment T and curve segment Q, and the feature point and are located in the matching parts of T and Q respectively, then there must be a submatrix R such that f(r ij )minimum.

[0259]

[0260]

[0261] Or the calculation result S pq If it is greater than the specified threshold δ, it is considered that there is a matching relationship.

[0262]

[0263] This step is to find the best matching interval, refer to Fig.39 As shown in (3), ② and ④ can be combined with Fig.39 When matching the short car line object below it, if there are multiple matching intervals, it is necessary to filter out the best matching interval. The details are as follows:

[0264] Take n points at equal intervals in these sampling intervals, calculate the sum of the squares of the curvature differences between the two groups of points and the distance between the two groups of points, and perform weighted summation of the two values ​​to obtain the best matching interval.

[0265]

[0266] The smaller V is, the closer the curve shape is, so the best matching interval is found, as shown in Fig.39 As shown in , curve segment ② is the best matching interval.

[0267] Step S375: Extend the curve segment corresponding to the matching interval equidistantly to both ends, and perform weighted summation on the matching results using the sum of the squares of the curvature differences of the feature points on the two groups of extended curve segments and the distances between the two groups of feature points.

[0268] Step S376: compare the weighted sum of the extended curve segments with the weighted sum of the curve segments corresponding to the current matching interval, and expand the matching interval based on the comparison result.

[0269] The above steps S375 and S376 extend the curve segment to both ends at equal intervals based on the current best matching interval determined. After the extension, the curve segment needs to be recalculated. Value, if If the value is not greater than the existing value, the starting or ending matching point is extended to the extended boundary point, and the above process is repeated until The match ends when the value is greater than the existing value (the smaller the V value, the better the match), completing the expansion of the matching range.

[0270] In another alternative embodiment, referring to Fig.31 As shown, both the straight line portion of the lane object and the completely straight lane object require straight line sampling. The embodiment of the present invention provides two sampling methods when sampling the (straight line) lane object, as follows:

[0271] <Method 1> (Straight Line) Direct Sampling of Car Line Objects

[0272] Reference Fig.40 As shown, for example, sampling points are arranged at the positions of 1 / 10 of the length of the first and last 1 / 10 of the length of the straight line object, which specifically includes the following steps:

[0273] Step S401: use one of the matched vehicle lane objects as a sampling line and the other as a matching line.

[0274] Step S402: Arrange sampling points at the beginning and end points of the sampling line, and insert sampling points on the sampling line based on a preset distance threshold.

[0275] Step S403: Project the sampling points on the sampling line onto the matching line respectively.

[0276] Step S404, determine whether the angle between the line connecting the sampling point and the projection point and the matching line is within a preset angle range; if so, execute step S405.

[0277] Step S405: Use the sampling points and the corresponding projection points as the alignment sampling points of the matched lane object.

[0278] Reference Figure 41-43 As shown, in the embodiment of the present invention, different segments in the same bunch have a matching relationship. Two mutually matching segments 1 and 2 are confirmed, and the car line object in one segment 1 is selected as the line where the sampling point is located, and the car line object in the other segment 2 is the matching line. The sampling points on each sampling line are projected onto the matching line, and the angle between the line connecting the sampling point and the projection point and the matching line is calculated to be greater than 85° and less than 95°. If yes, it means that the projection point meets the requirements and is retained. Points that do not meet the requirements are discarded. A set of matching sampling points can be generated by the above method.

[0279] <Method 2> (Straight Line) Car Line Object Reference Line Sampling

[0280] This method is based on the reference line for sampling. Fig.44 and Fig.45 As shown, the following steps may be included:

[0281] Step S441: based on the projections of the first and last endpoints of the plurality of lane objects on the reference line included in the group data, intercept a target reference line segment covering all the lane objects on the reference line.

[0282] Step S442: Use the target reference line segment as a sampling line and arrange sampling points.

[0283] Step S443: project the sampling points onto the multiple lane objects respectively.

[0284] Step S444: based on the angle between the line connecting the sampling point and the projection point and the car line object, and the number of projection points corresponding to the same sampling point, select the aligned sampling points of the car line object from the projection points corresponding to the sampling point.

[0285] In the embodiment of the present invention, there is a reference line in each group data (bunch) (which can be directly obtained from the map data and exists throughout the bunch), and there are multiple matching lane objects in each bunch. For example, Fig. 20As can be seen from the left, a set of matching car line objects includes three car line objects. Each endpoint of the car line object is projected onto the reference line, and the two farthest points are obtained. The intercepted target reference line segment is obtained on the reference line, so that the length of the intercepted target reference line segment can completely cover the three car line objects. If the length of the intercepted target reference line segment exceeds the preset distance threshold, sampling is performed according to the reference line length divided by the first preset distance value. If sampling is performed according to the second preset distance value, the interval is metered. Fig. 20 The sampling points on the target reference line segment are projected onto the matching car line object. The retained sampling points need to meet two conditions at the same time: ① The angle between the projection point and the sampling point and the matching car line object is between 95-85 (as shown in the right figure below, there is an angle of 100°, and the corresponding sampling point is discarded); ② There are more than two projection points on the matching car line object (such as Fig. 20 Right one, after projection, the upper right endpoint has only one projection point, so the corresponding sampling point is discarded), and then the sampling points that meet the requirements are formed into a group of sampling points.

[0286] In another optional embodiment, the embodiment of the present invention samples semantic objects based on a semantic object set, referring to Fig.46 and Fig.47 As shown, the following steps may be specifically included:

[0287] Step S471, traverse each group of semantic objects that match each other in the semantic object set, and determine whether the ratio of the similar areas of the matched semantic objects is greater than a first area ratio threshold; if it is less than the first area ratio threshold, execute step S472; if it is greater than the first area ratio threshold, execute step S473.

[0288] Step S472: traverse other groups of semantic objects that match each other in the semantic object set.

[0289] Step S473, determine whether the ratio of the similar areas of the matched semantic objects is greater than a second area ratio threshold; wherein the first area ratio threshold is less than the second area ratio threshold; if it is greater than the second area ratio threshold, execute step S474; if it is less than the second area ratio threshold, execute step S475. In the embodiment of the present invention, the first area ratio threshold may be 0.5, and the second area ratio threshold may be 0.9.

[0290] Step S474: Use the centroid points of the semantic objects as alignment sampling points.

[0291] Step S475, determine whether the outer boundary of the semantic object is a rectangle; if it is a rectangle, execute step S472; if it is not a rectangle, execute step S476. In the embodiment of the present invention, since the rectangular pattern lacks alignment features, that is, when matching, it is impossible to accurately identify which part can be matched successfully, the rectangular semantic object is no longer sampled.

[0292] Step S476: Determine the local semantic object corresponding to the highest similarity region in the matched semantic objects, and use the centroid point of the local semantic object as the alignment sampling point.

[0293] In the embodiment of the present invention, when sampling semantic objects, it is first determined whether two semantic objects are similar, and only if they are similar, are they sampled as a pair of semantic objects. In the specific sampling process, if the similarity area of ​​the two semantic objects is large when they match, the centroid point can be directly used as a pair of matching sampling points; if only a partial match is achieved, the two semantic objects can be partially matched and then sampled locally.

[0294] In another alternative embodiment, referring to Fig.48 As shown, determining the highest similarity region may specifically include the following steps:

[0295] Step S481: Determine the respective graphic bounding boxes based on the matched semantic objects, and determine a target graphic bounding box covering the two graphic bounding boxes based on the length and width of the respective graphic bounding boxes.

[0296] For the two matching semantic objects, determine their respective graphic bounding boxes, which can completely cover the semantic objects. Fig.49 As shown, the two semantic objects, one on the left and one on the right, are matched, and the graphic outer frame one and the graphic outer frame two are determined according to the first semantic object and the second semantic object, respectively. Then, according to the length and width of the graphic outer frame one and the graphic outer frame two, the target outer frame is determined, which can completely cover the graphic outer frame one and the graphic outer frame two. When calculating the matching graphic outer frame, when calculating the similarity between the feature matrices of a group of matching semantic objects, it is necessary to use the feature matrix of the semantic object with a smaller area (the second semantic object) to slide the window on the feature matrix of the semantic object with a larger area (the first semantic object). Then, it is necessary to ensure that the two dimensions (horizontal and vertical) of the feature matrix of the semantic object with a larger area must be greater than the two dimensions of the feature matrix of the semantic object with a smaller area (the second semantic object). Therefore, before constructing the semantic object feature matrix, it is necessary to determine the target outer frame of the matching graphic, and generate the feature matrix according to the target outer frame.

[0297] Step S482: construct a granularity rectangle based on the outer border of the target graphic, and construct a first semantic object feature matrix based on the first semantic object and the granularity rectangle in the matched semantic objects; wherein the matched semantic objects include the first semantic object and the second semantic object, and the area of ​​the first semantic object is greater than the area of ​​the second semantic object.

[0298] In this step, a granular rectangle is generated in the target bounding box of the matching semantic object. For example, the size of the granular rectangle used is 10cm*10cm. The granular rectangle is moved from left to right and from top to bottom in the target bounding rectangle, and the intersection area between the granular rectangle and the semantic object after each movement is calculated (refer to Fig.50 The numbers marked in the medium-granularity rectangles represent the intersection area ratio between each granularity rectangle and the semantic object, and the feature matrix of the semantic object is constructed using the intersection area.

[0299] Step S483: Determine a second semantic object feature matrix of the sliding search window and the second semantic object based on the second semantic object and the granularity rectangle.

[0300] This step determines the sliding search window (small rectangular non-zero grid interval search) based on the semantic object with a smaller area, calculates the non-zero interval size of the semantic object with a smaller area (that is, the value of each granularity rectangle is not zero), and uses this non-zero interval dimension to generate a sliding search window (refer to Fig.51 The right side shown is the determined sliding search window).

[0301] Step S484: slide the sliding search window on the outer border of the target graphic to determine the similarity between the first semantic object feature matrix and the second semantic object feature matrix after each slide.

[0302] This step calculates the similarity of the object matrix. The sliding search window is used to move on the feature matrix of the larger semantic object. The similarity s between the matrix in each sliding search window (e.g., a) and the feature matrix of the smaller semantic object (e.g., b) is calculated. The sliding matching process can be referred to Fig.52 The calculation formula is as follows:

[0303]

[0304] Step S485, determine the highest similarity value, and use the region with the highest similarity value as the highest similarity region. In specific implementation, if the similarity is higher than 0.9, use the centroid of the semantic object in the region with the highest similarity and the centroid of the matching semantic object as a set of matching sampling points.

[0305] Step S486, compare the highest similarity value with the preset highest similarity threshold, if the highest similarity value is greater than the preset highest similarity threshold, the centroid point of the local semantic object corresponding to the highest similarity region is used as the alignment sampling point. This step further screens, so that better alignment sampling points can be screened out.

[0306] The semantic object sampling method provided by the embodiment of the present invention can determine the longitudinal error and the lateral error between the semantic objects included in each segment data in the group data, so as to lay a foundation for the alignment of multi-source data in the subsequent steps.

[0307] In order to facilitate a better understanding of the embodiments of the present invention, some technical terms that may be involved in the embodiments of the present invention are explained here. Fig.53 As shown in the figure, a tile is the basic unit of a high-precision map, that is, there are several tile data in the map in the form of raster data; a tile can include multiple groups of data (bunch), a bunch is a collection of segment data (segment), and the areas passed by these segments are required to be roughly the same; a segment represents the data generated by each vehicle passing through a certain area. For example, a road between two intersections is passed by multiple vehicles, and multiple vehicles generate multiple segment data between the two intersections. These different segment data belong to the same bunch; and after a vehicle passes through a certain area, it will collect the vehicle line, semantic objects, etc. in the area. A segment includes vehicle line objects and semantic objects.

[0308] The method for aligning multi-source data in the embodiment of the present invention refers to Figure 54-Figure 56 The following steps may be included:

[0309] Step S541: construct a graph optimization constraint factor in each group of data in the window list to be optimized based on the aligned sampling points of the semantic objects and the vehicle lane objects included in the multiple segment data in each group of data in the window list to be optimized.

[0310] Step S542: Based on the anchor point data of the group data queue, determine the group data boundary of the sliding window to be optimized, so as to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized.

[0311] It should be noted that, when executing the above-mentioned steps S541 and S542, step S541 may be executed first and then step S542, or step S542 may be executed first and then step S541, or step S541 and step S542 may be executed at the same time, or only one of step S541 and step S542 may be executed, and the embodiment of the present invention does not make any specific limitation on this.

[0312] Step S543: perform graph optimization solution based on the graph optimization constraint factors within the group data and / or the graph optimization constraint factors between the group data to obtain the segment data graph optimization transformation matrix included in the group data, and align the multi-source data based on the segment data graph optimization transformation matrix.

[0313] In the embodiment of the present invention, when performing graph optimization solution, the solution may be performed based only on the graph optimization constraint factors within the group data, or it may be performed based only on the graph optimization constraint factors between the group data. Of course, the optimal solution may also be to use the graph optimization constraint factors within and between groups to solve at the same time, and the embodiment of the present invention does not make any specific limitation on this.

[0314] The above-mentioned multi-source data alignment method provided in the embodiment of the present invention can align the data within the group through the intra-group constraints, and align the component data through the component constraints, thereby eliminating the relative errors between multiple multi-source data and ensuring the accuracy and reliability of map elements during reconstruction.

[0315] In an alternative embodiment, referring to Figure 54-Figure 56 As shown, before constructing the graph optimization constraint factors within the group data and the graph optimization constraint factors between the group data, the following steps need to be performed, namely: step S540, based on whether the group data has been aligned and the signal accuracy of the group data, determine the group data included in the window list to be optimized and the anchor data in the group data queue in the group data queue. In this step, due to the large volume of data, it is impossible to perform the alignment operation at the same time, and the aligned group data can be used as a benchmark for the unaligned operation. Therefore, this step needs to determine which data in a series of data needs to be aligned first, so as to facilitate the execution of the alignment step.

[0316] In a specific embodiment, the group data included in the window list to be optimized and the anchor point data in the group data queue are determined, referring to Fig.57 and Fig.56 As shown, the following steps may be included:

[0317] Step S571, initialize the group data queue, randomly select a group data from the group data queue as the current group data, and initialize the search state of the current group data to the closed state.

[0318] This step is to initialize a bunch queue (i.e., clear the bunch queue), randomly add a current bunch 1 to the bunch queue, and initialize the search state of the current bunch 1 to False (including forward search state and backward search state). It should be noted that the bunch queue mentioned below is the group data queue.

[0319] Step S572, determine whether the current group of data has been aligned. If so, execute step S573; if not, execute step S574.

[0320] This step is to execute the queue and pop up the above-mentioned bunch 1. If the alignment operation has been performed, no further alignment is required, and it can be used as anchor data to align other groups of data in the queue. That is, the segment that has been aligned is set to visited (check whether the search status of this bunch 1 has been activated), that is, set as an anchor point, so that the segment will not be adjusted when it enters optimization again.

[0321] Step S573: Use the current group data as the anchor data of the group data queue, and adjust the search state of the group data to a closed state.

[0322] Step S574, determine the quality of the signal accuracy of the current set of data; if the signal accuracy of the current set of data is poor, execute step S575; if the signal accuracy of the current set of data is good, execute step S573.

[0323] This step is to judge whether the signal of the segment in this bunch 1 is good or bad (excellent or inferior). The judgment is made based on the signal quality of the track points in the segment. When the signals of the track points are all good, the signal of the segment is evaluated as good. If the signals of all the segments in a bunch are good, the signal of the group of data is evaluated as good).

[0324] Step S575: Add the current group data to the list of windows to be optimized.

[0325] In the specific implementation of this step, if the signal is not good, add this bunch 1 to the sliding window to be optimized, and check whether this bunch 1 has activated the search state; if not, activate the search state for it, and adjust the forward and backward search states of this bunch 1 to True. Adjust the forward search state to True, and add the forward connection Bunch 2 to the bunch queue, adjust the backward search state to True, and add the backward connection Bunch 3 to the bunch queue.

[0326] In this step, when the signal is good, check whether the search status of bunch 1 has been activated. If it has been activated, adjust the current search direction to False, that is, the boundary of the current window (list of windows to be optimized) in one direction is found, so no more search is performed in this direction.

[0327] Step S576: traverse the group data queue to determine the group data included in the window list to be optimized and the anchor point data in the group data queue.

[0328] This step is to check whether the bunch queue is empty. If not, return to step S571 and continue to execute the dequeue. If it is empty, determine whether the list of windows to be optimized is empty. If it is empty, end this round of alignment process. If not, continue with the following alignment steps.

[0329] In a specific example, initialize the bunch queue, join bunch 1, pop bunch 1,

[0330] ① The signal of bunch 1 is not good, so this bunch 1 is added to the sliding window to be optimized, and the forward search is performed at the same time, and the forward-connected bunch 2 is added to the bunch queue. The backward search is performed, and the backward-connected bunch 3 is added to the bunch queue. At this time, there are two bunches (Bunch 2 and Bunch 3) in the bunch queue;

[0331] ② The signal of bunch 1 is good and has been activated. Adjust the forward search state and no longer search forward, but only search backward.

[0332] At this time, the bunch queue is not empty, so bunch 2 continues to be popped out, and the above operation is repeated to continuously add bunches with poor signals to the sliding window to be optimized, until the bunch queue is empty and no more bunches are popped out.

[0333] In an optional embodiment, a graph optimization constraint factor is constructed within each group data in the window list to be optimized, so that alignment optimization is performed within each bunch. In a specific implementation, this embodiment can achieve the top, bottom, left, and right alignment within the group data by constructing a line-to-line objective function, which may specifically include: first, based on the alignment sampling points of the semantic objects and the vehicle line objects included in the multiple segment data in the group data, determining the alignment error term of the segment data; then, based on the alignment error term and the position of the alignment sampling point, constructing the graph optimization constraint factor within the group data (the line-to-line GICP objective function is used in this embodiment).

[0334] In the embodiment of the present invention, the optimization constraint construction of the bunch inner graph is mainly divided into two types of constraints, one is based on the point-to-point error between semantic object sampling points, and the other is based on the line-to-line error between lane line sampling points.

[0335] The constraint construction between semantic objects is consistent with the traditional ICP algorithm. The ICP algorithm transforms the points to be registered in the point cloud into the same coordinate system and performs the least squares method on the Euclidean distance of all sample points to obtain the optimal transformation matrix T * , now suppose there are two sets of point clouds A={a i}, B={b i}, for each matching point cloud sample {a i ,b i}, the loss function is as follows:

[0336]

[0337] In the above formula and For point a i and b i The transformation matrix from the origin of the world coordinate system to its own position, based on the maximum a posteriori estimation, has the following objective function, where w i Weights for sampling point registration:

[0338] argmin T {∑ i w i ‖b i -Ta i ‖ 2} (2)

[0339] After adding the plane normal vector η i After that, we get the objective function of the point-to-plane case:

[0340] argmin T {∑ i w i ‖η i (b i -Ta i )‖ 2} (3)

[0341] In the g2o framework, the observed trajectory points of the point cloud sets A and B are vertices (in the embodiment of the present invention, the trajectory points are found by using the timestamps of the vehicle-side fusion frames), and each loss function e i is an edge connecting two vertices.

[0342] In a specific example, the raw data input before alignment processing refers to Figure 6 As shown, now sample these two segments, refer to Fig.59 and Fig.60 As shown in the figure, 32 sets of matching sampling points are obtained. Among them, 30 sets of matching relationships come from lane lines and 2 sets come from semantic objects. The above sampling points are used as alignment sampling points to build the model. There are 32 sets of matching relationships between two observation points. Because the two observation points point to each other, that is, every two matching points point to each other, refer to Fig.61As shown in , there are 64 error edges (alignment error terms), which are different when pointing from the upper point to the lower point and from the lower point to the upper point. Each edge represents the error between this set of matching relationships, because the two matching points are the same point, but the poses of the same point observed at different observation points are inconsistent, which indicates that an error has occurred. Using the above formula (3) as the objective function, the least squares method is used to solve and obtain the optimal transformation matrix T * (displacement and object), adjust the two matching segments separately, and get the result as follows Fig.62 shown.

[0343] Reference Fig.62 As shown in the figure, after adjustment, the two segments are not completely aligned. The inventor found that the reason is that the lane line sampling provides a longitudinal error, but the longitudinal error in the lane line sampling is wrong. Therefore, the longitudinal error provided by the lane line sampling point needs to be ignored. In order to solve the above problem, the plane-to-plane GICP algorithm is used in the embodiment of the present invention, and the line-to-line GICP algorithm is proposed to optimize and improve it. The details are as follows:

[0344] In general SLAM scenarios, it is assumed that the corner points of the image have offset errors on the plane, so the distance from point to point or point to surface becomes the error distance from surface to surface. The objective function of surface-to-surface GICP is as follows: is the covariance of point cloud A and B, and has undergone a normal projection:

[0345]

[0346] Use the maximum likelihood estimate to iteratively calculate T

[0347]

[0348]

[0349] e i =‖b i -Ta i ‖2 (7)

[0350]

[0351]

[0352]

[0353]

[0354] ui and vi represent b i and a iThe normal vector of and By rotating the above covariance matrix, p i Let R denote a rotation that transforms the basis vector e1 to some normal vector.

[0355] In the actual application process of map learning, the embodiment of the present invention densely samples semantic objects, restores them to the effect of point cloud to a certain extent, and converts the matching relationship between semantic objects into the matching relationship between sampling points. Since the lane model has a rich road line topology structure, it is more inclined to judge the errors between lines and points, and between lines, compared with the surface-to-surface projection error. In this implementation, the projection information matrix is ​​redesigned, and the GICP is improved in point-to-line and line-to-line calculations, that is, all vectors in the 3D space are projected to a certain projection line segment m, and m is in a polar coordinate system with the polar axis as the positive direction of the x-axis. Then the normal vector of the polar coordinate plane is,

[0356]

[0357] Let the rotation angle from x to m be θ, then we have the Lie algebra The rotation matrix can be obtained:

[0358]

[0359] Here is a specific example, select a simple projection matrix P i (The normal vector represented by this projection matrix should be the same as the above point-to-plane), projecting the xyz coordinate components onto the xz plane, where ξ is an infinitesimal quantity:

[0360]

[0361] The objective function of point-to-line GICP is as follows, where ∑ is the covariance matrix of the GICP after a point-to-plane projection. The following equation degenerates to point-to-plane when the z-axis is completely ignored:

[0362]

[0363] Furthermore, we can get the line-to-line objective function, P ia With P ib The composition of the projection matrix P refers to the above information i However, since the covariance is calculated, the projection matrix P iTake the inverse. Because the projection matrix itself is a singular matrix and cannot be inverted, the infinitesimal introduced plays a role in this step, which ensures the non-singularity of P:

[0364]

[0365]

[0366]

[0367]

[0368] The sampling point group on the lane line is constructed using the new line-to-line GICP objective function, and the sampling point group on the semantic object is constructed using the GICP objective function. The two objective functions are constructed in the same graph model and solved to obtain a new transformation matrix. The segment is adjusted according to the transformation matrix. After adjustment, refer to Fig.63 As shown in the figure, the two segments in the final group data are aligned.

[0369] In another optional embodiment, the above step S12 determines the group data boundary of the sliding window to be optimized based on the anchor point data of the group data queue, so as to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized, which may specifically include: first, based on the current group data in the sliding window to be optimized, a layer-order traversal is performed to determine the group data boundary of the sliding window to be optimized and the associated group data included in the sliding window to be optimized with the anchor point data; then, the graph optimization constraint factor (GICP edge constraint objective function) between the group data included in the sliding window to be optimized is constructed with the trajectory constraints between the segment data included in each group data in the associated group data.

[0370] Reference Fig.64 As shown, when each Bunch realizes intra-segment alignment separately, there will be unevenness between the bunches. The reason for this phenomenon is that the constraint of trajectory continuity between the previous and next segments of each segment in the bunch is not taken into account when performing intra-segment alignment. In order to solve the above problems, several bunches need to be put together for modeling and optimization in the embodiment of the present invention. It should be noted that not all the bunches in the Tile are associated and solved. The reason why all the Bunches in the Tile are not modeled and optimized together is from the perspective of the time complexity of the algorithm.

[0371] In the specific implementation, the need to optimize several bunches together becomes a key consideration. In real business scenarios, not all bunches need to be aligned, that is, there are bunches with good signals, and there may be no relative error inside, or the relative error is very small. These bunches can be used as anchor data, that is, fixed points, and they can be used as the boundaries of the window to eliminate the relative error within the window. Therefore, the construction rule of the optimized sliding window is to start with a bunch, search forward and backward for each segment in this bunch according to the trajectory, and put the bunch where the previous and next segments are located into the optimized sliding window until the front and back bunches of the window are all anchor bunches (in addition to the bunch with good signal, there is another bunch that can be used as an anchor, which is an optimized bunch). Stop the forward and backward search and complete the construction of the optimized sliding window.

[0372] Reference Fig.65 As shown, the optimized sliding window is constructed as Fig.65 As shown in (a), the dotted line track in the forward search needs to search four bunches to find the anchor point, while the solid line track needs to search two windows to find the anchor point. Therefore, the forward search stops after finding the anchor point window of the solid line track, and the backward search is the same. It should be noted that the layer-order traversal method is used for forward and backward search in this embodiment.

[0373] After building the sliding window, you can build a joint objective function for multiple bunches in the window. Based on the constraint objective function within the bunch, add the trajectory constraints between the previous and next segments of each segment in the bunch to the objective function to build a new joint objective function. The formula is as follows:

[0374]

[0375] Where T=[T0,T1,...,T K ] represents the K Bunches in the sliding window, v K represents the transformation matrix of the Kth bunch, v k represents the observation point of the Kth bunch, Represents the constraint between the previous and next trajectory points.

[0376] The sliding window obtained in the previous example is modeled using a new objective function 20. The graph structure in the model is as follows Fig.66 As shown in the figure, the next bunch can be known based on each segment trajectory point in the bunch. When one set of trajectory points finds a bunch that has been visited or has a good signal, the search downwards stops.

[0377] In an embodiment of the present invention, preset trajectory points of segment data included in each group data in the associated group data are used as edge constraint nodes between the associated group data, so as to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized based on the edge constraint nodes.

[0378] Reference Fig.66 As shown, in the implementation of the present invention, each trajectory point is the first trajectory point (graph node) of each segment in this bunch. The dashed edges in the figure represent the constraints between trajectories, and the straight edges represent the GICP edge constraints constructed by the matched sampling points. Since the semantic objects in the intersections are not matched with the lane lines, there are no GICP edges connecting the observation points at the two intersections. Solving this graph model can obtain the transformation matrix of each segment. After adjustment based on the transformation matrix, the relative error between the segments in each bunch in the window can be eliminated, and the Bunches can be made relatively smooth (refer to Fig.67 effect shown).

[0379] The transformation matrix of each segment calculated in the graph optimization solution step is used to adjust the semantic objects, lane lines, etc. in this segment. After the adjustment, the same semantic objects and lane lines in multi-source data will be located at approximately the same longitude and latitude, thereby eliminating the relative error between the same semantic objects and lane lines.

[0380] In another alternative embodiment, referring to Fig.54 As shown, the method may further include the following steps: Step S544, clustering and aggregating based on the aligned multi-source data to aggregate and obtain map data including semantic objects and lane objects. The alignment in the embodiment of the present invention provides a data basis for the subsequent steps of clustering and aggregation, making it easier to cluster the semantic objects and lane objects collected by the multi-source data, and providing a data source for subsequent aggregation into high-definition map elements.

[0381] Based on the same inventive concept, an embodiment of the present invention further provides a multi-source data processing device, referring to Fig.68 As shown, the device may include:

[0382] The segmentation module 681 is used to perform segmentation processing on each single-pass data included in the multi-source data, so as to divide the single-pass data into a plurality of segment data;

[0383] The grouping module 682 is used to construct group data corresponding to the multi-source data based on the segment data;

[0384] The matching module 683 is used to match the segment data included in the group data to determine the matching relationship of the segment data;

[0385] The sampling module 684 is used to sample the vehicle lane object and the semantic object included in the segment data based on the matching relationship of the segment data to obtain the alignment sampling point of the segment data;

[0386] The alignment module 685 is used to align the segment data based on the alignment sampling points to achieve the multi-source data alignment.

[0387] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned multi-source data processing method is implemented.

[0388] Based on the same inventive concept, a computer device is also provided in an embodiment of the present invention, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the multi-source data processing method as described above when executing the program.

[0389] The principles of the problems solved by the above-mentioned devices, media, related equipment and systems in the embodiments of the present invention are similar to those of the above-mentioned methods, so their implementation can refer to the implementation of the above-mentioned methods, and the repeated parts will not be repeated.

[0390] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0391] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0392] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0393] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0394] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for processing multi-source data, characterized in that: include: Performing segmentation processing on each single-pass data included in the multi-source data to divide the single-pass data into a plurality of segment data; constructing group data corresponding to the multi-source data based on the segment data; Matching the segment data included in the group data to determine a matching relationship between the segment data; Based on the matching relationship of the segment data, sampling the vehicle lane object and the semantic object included in the segment data to obtain an alignment sampling point of the segment data; The segment data are aligned based on the alignment sampling points to achieve the multi-source data alignment.

2. The method according to claim 1, characterized in that The single-trip data is processed in segments, including: Based on the intersection segmentation feature objects respectively identified in the multiple single-trip data, determining the segmentation points and segmentation radius of the multi-source data grouping; The plurality of single-pass data are segmented according to the segmentation points and the segmentation radius, so as to divide the single-pass data into a plurality of segment data.

3. The method according to claim 2, characterized in that Constructing group data corresponding to the multi-source data based on the segment data includes: Based on the segment data divided by the single-pass data, group data corresponding to the multi-source data is generated, and an association relationship between the group data and the segment data is established.

4. The method according to claim 3, characterized in that The step of generating group data corresponding to the multi-source data based on the segment data divided by the single-pass data, and establishing an association relationship between the group data and the segment data includes: According to the direction and position of the segment data, multiple segment data corresponding to the same road section are divided into a group data; An association relationship between the group data and the segment data included therein is established so that the segment data carries corresponding group data attributes.

5. The method according to claim 3, characterized in that: After generating the group data, the method further includes: The projection distance between the groups of data is compared with a preset projection distance threshold, so as to merge at least two of the groups of data into one group of data based on the comparison result.

6. The method according to any one of claims 3 to 5, characterized in that: Also includes: Determine whether the length of the group of data is greater than a preset length threshold; If so, an intra-road segmentation feature object is inserted between adjacent intersection segmentation feature objects of the single-trip data, and the segmentation point and segmentation radius of the multi-source data grouping are determined based on the intra-road segmentation feature object.

7. The method according to any one of claims 3 to 5, characterized in that: Also includes: Based on the association relationship among the group data, segment data and segmentation points in the multi-source data, the semantic objects, lane objects and track lines in the multi-source data are associated to establish a topological structure relationship of the multi-source data.

8. The method according to claim 1, characterized in that Matching the segment data included in the group data includes: Determining associated group data based on the discreteness of segmentation point positions in group data constructed from multi-source data, so as to construct a segment data set based on the segment data included in the associated group data; Determine the offset of the current segment data to be matched in the segment data set; Any segment of the data to be matched is moved based on the offset and a matching rate of the data to be matched is determined, so as to determine a matching relationship of the segment of the data where the perception data in the multi-source data is located based on the matching rate.

9. The method according to claim 8, characterized in that Determining the offset of the current segment data to be matched in the segment data set includes: Determine whether there is a matching relationship between adjacent segment data of the current segment data to be matched in the segment data set; If there is no matching relationship, determining the first offset based on the boundary attribute information in the current segment data to be matched; If there is a matching relationship, a first offset is determined based on the boundary attribute information in the current segment data to be matched, and / or a second offset is determined based on the vehicle lane information in the adjacent segment data; Correspondingly, any segment of the data to be matched is moved based on the first offset and / or the second offset.

10. The method according to claim 9, characterized in that Before determining whether there is a matching relationship between adjacent segment data of the current segment data to be matched in the segment data set, the method further includes: Based on the boundary attribute information in each of the segment data, determining a segment data matching order in the segment data set, so as to perform pairwise matching on the segment data included in the segment data set until all the segment data are matched; The boundary attribute information includes: the distance between the lane boundary and the trajectory line, and the lane boundary length.

11. The method according to claim 10, characterized in that The segment data included in the segment data set are matched in pairs until all the segment data are matched, including: The segment data that has not been successfully matched in the segment data set is recorded, and after all other segment data in the segment data set are matched, the recorded segment data is re-matched.

12. The method according to any one of claims 9 to 11, characterized in that: Before determining the association group data, the method further includes: Based on the vehicle line tracking ID in the single trip data, the segment data divided by the single trip data are matched and transferred to determine the matching relationship of the segment data within the group data constructed by the multi-source data and the matching relationship of the segment data in the adjacent group data; If the matching relationship of the segment data cannot be determined based on the vehicle lane tracking ID, the associated group data is determined based on the discrete degree of the segment point positions in the group data constructed from the multi-source data.

13. The method according to claim 1, characterized in that Based on the matching relationship of the segment data, sampling the vehicle lane object and the semantic object included in the segment data includes: Based on a plurality of segment data included in the group data divided from the multi-source data, the vehicle line objects with matching relationships and the semantic objects with matching relationships included in the segment data are combined in pairs to respectively construct a vehicle line object set and a semantic object set of the group data; Based on the vehicle line object set and the semantic object set, the vehicle line objects and the semantic objects are sampled respectively, so as to determine the alignment sampling points based on the sampling results.

14. The method according to claim 13, characterized in that Sampling the vehicle line object based on the vehicle line object set includes: Traversing each set of two-by-two matching car line objects in the car line object set, and determining whether the matching car line objects have an intersection; If there is no intersection, traverse the other groups of matching car line objects in the car line object set; If there is an intersection, determine whether the matched lane object has a curve part; If there is no curve part, a straight line sampling is performed on the lane object; If there is a curve portion, curve sampling is performed on the curve portion of the lane object.

15. The method according to claim 14, characterized in that The curve sampling of the matched lane object includes: Based on the curve portion of the vehicle line object, determining a curve segment corresponding to a matching interval of the curve portion; Performing ellipse fitting based on the matching interval to respectively obtain two ellipses; Taking the characteristic points where the direction of a curve segment corresponding to a lane object changes as sampling points, and constructing a matching vector with the center of the fitted ellipse and the sampling points; Determine a matching point on the curve segment corresponding to the other car line object by fitting the center of the ellipse and the matching vector to the curve segment corresponding to the other car line object; The sampling point and the matching point are used as alignment sampling points of the lane object.

16. The method according to claim 15, characterized in that The matching intervals for determining the curve portion of the lane object include: Based on the extreme value points of the curvature change of the curve portion, the curve portion is segmented to obtain curve segments; Constructing a curve segment feature matrix based on feature points on the curve segment including feature points with changed directions; Matching is performed based on the curve segment feature matrix, and the matching interval of the curve portion of the lane object is determined using the curvature parameters and distance parameters of the two groups of feature points.

17. The method according to claim 16, characterized in that After determining the matching interval of the curve part of the car line object, it also includes: Extending the curve segment corresponding to the matching interval equidistantly to both ends, and weighted summing the matching results by the sum of the squares of the curvature differences of the feature points on the two groups of extended curve segments and the distances between the two groups of feature points; The weighted summation result of the extended curve segment is compared with the weighted summation result of the curve segment corresponding to the current matching interval, and the matching interval is expanded based on the comparison result.

18. The method according to claim 14, characterized in that The straight line sampling of the lane object comprises: Use one of the matched vehicle line objects as the sampling line and the other as the matching line; Arranging sampling points at the beginning and end points of the sampling line, and inserting sampling points on the sampling line based on a preset distance threshold; The sampling points on the sampling line are projected onto the matching line respectively; and it is determined whether the angle between the connecting line of the sampling point and the projection point and the matching line is within a preset angle range; If so, the sampling point and the corresponding projection point are used as the alignment sampling points of the matched lane object.

19. The method according to claim 14, characterized in that The straight line sampling of the lane object comprises: Based on the projections of the first and last endpoints of the plurality of lane objects on the reference line included in the group data, intercepting a target reference line segment covering all the lane objects on the reference line; Using the target reference line segment as a sampling line and arranging sampling points; Projecting the sampling points onto the plurality of lane objects respectively; Based on the angle between the line connecting the sampling point and the projection point and the car line object, and the number of projection points corresponding to the same sampling point, the aligned sampling points of the car line object are screened out from the projection points corresponding to the sampling point.

20. The method according to claim 13, characterized in that Sampling the semantic object based on the semantic object set includes: Traversing each group of semantic objects that match each other in the semantic object set, and determining whether a ratio of similar areas of the matched semantic objects is greater than a first area ratio threshold; If it is less than the first area ratio threshold, traversing other groups of semantic objects that match each other in the semantic object set; If it is greater than the first area ratio threshold, determining whether the ratio of similar areas of the matched semantic objects is greater than a second area ratio threshold; wherein the first area ratio threshold is less than the second area ratio threshold; If it is greater than the second area ratio threshold, the centroid points of the semantic objects are respectively used as alignment sampling points; If it is less than the second area ratio threshold, determining whether the outer boundary of the semantic object is a rectangle; If it is a rectangle, traverse the other groups of semantic objects that match each other in the semantic object set; If it is not a rectangle, a local semantic object corresponding to the highest similarity region is determined in the matched semantic objects, and the centroid point of the local semantic object is used as the alignment sampling point.

21. The method according to claim 20, characterized in that The highest similarity regions are determined to include: Determine the respective graphic bounding boxes based on the matched semantic objects, and determine a target graphic bounding box covering the two graphic bounding boxes based on the length and width of the respective graphic bounding boxes; Constructing a granularity rectangle based on the outer bounding box of the target graphic, and constructing a first semantic object feature matrix based on a first semantic object in the matched semantic objects and the granularity rectangle; wherein the matched semantic objects include a first semantic object and a second semantic object, and an area of ​​the first semantic object is greater than an area of ​​the second semantic object; Based on the second semantic object and the granularity rectangle, determining a second semantic object feature matrix of a sliding search window and the second semantic object; Sliding the sliding search window on the outer border of the target graphic to determine the similarity between the first semantic object feature matrix and the second semantic object feature matrix after each sliding; The highest similarity value is determined, and the region with the highest similarity value is taken as the highest similarity region.

22. The method according to claim 1, characterized in that The aligning the segment data based on the alignment sampling points includes: Based on the aligned sampling points of the semantic objects and the lane objects included in the plurality of segment data in each group data in the to-be-optimized window list, constructing a graph optimization constraint factor in each group data in the to-be-optimized window list; Based on the anchor point data of the group data queue, determine the group data boundary of the sliding window to be optimized, so as to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized; Graph optimization is solved based on the graph optimization constraint factors within the group data and / or the graph optimization constraint factors between the group data to obtain a segment data graph optimization transformation matrix included in the group data, and multi-source data are aligned based on the segment data graph optimization transformation matrix.

23. The method according to claim 22, characterized in that Also includes: Initializing a group data queue, randomly selecting a group data from the group data queue as current group data, and initializing a search state of the current group data to a closed state; Determine whether the current group of data has been subjected to an alignment operation, and if so, use the current group of data as anchor data of the group of data queue, and adjust the search state of the group of data to a closed state; If the alignment operation has not been performed, determining the signal accuracy of the current set of data; If the signal accuracy of the current group of data is poor, adding the current group of data to the list of windows to be optimized; If the signal accuracy of the current group of data is excellent, the current group of data is used as the anchor point data of the group of data queue, and the search state of the group of data is adjusted to a closed state; The group data queue is traversed to determine the group data included in the to-be-optimized window list and the anchor point data in the group data queue.

24. The method according to claim 22, characterized in that Constructing the graph optimization constraint factors in each group of data in the window list to be optimized includes: Determining an alignment error term of the segment data based on alignment sampling points of semantic objects and lane objects included in a plurality of segment data in the group data; A graph optimization constraint factor within the set of data is constructed based on the alignment error term and the positions of the alignment sampling points.

25. The method according to claim 22, characterized in that The anchor point data based on the group data queue determines the group data boundary of the sliding window to be optimized to construct a graph optimization constraint factor between the group data included in the sliding window to be optimized, including: Based on the current group data in the sliding window to be optimized, perform a layer-order traversal, and determine the group data boundary of the sliding window to be optimized and the associated group data included in the sliding window to be optimized by using the anchor point data; A graph optimization constraint factor between the group data included in the sliding window to be optimized is constructed based on the trajectory constraints between the segment data included in each group data in the associated group data.

26. The method according to any one of claims 1 to 25, characterized in that Also includes: Clustering and aggregation are performed based on the aligned multi-source data to aggregate map data including semantic objects and lane objects.

27. A multi-source data processing device, characterized in that: include: A segmentation module, used for performing segmentation processing on each single-pass data included in the multi-source data, so as to divide the single-pass data into a plurality of segment data; A grouping module, used for constructing group data corresponding to the multi-source data based on the segment data; a matching module, configured to match the segment data included in the group data to determine a matching relationship between the segment data; A sampling module, configured to sample the vehicle lane objects and semantic objects included in the segment data based on the matching relationship of the segment data, so as to obtain aligned sampling points of the segment data; An alignment module is used to align the segment data based on the alignment sampling points to achieve the multi-source data alignment.

28. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the multi-source data processing method according to any one of claims 1 to 26 is implemented.

29. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the multi-source data processing method according to any one of claims 1 to 26 is implemented.

Images