Map object determination method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present disclosure disclose a kind of determination method, apparatus, electronic equipment and storage medium of map object, wherein, method includes: obtaining the first point cloud map corresponding to target space range;Determine at least one local point cloud map based on the first point cloud map and pre-set sampling rule;Based on each local point cloud map, the attribute and ordered point set of object corresponding to each local point cloud map are determined using the object detection model obtained by pre-training, and the ordered point set includes target number of coordinate points in the first coordinate system for describing object;The attribute and ordered point set of object corresponding to each local point cloud map are aggregated, and the map object corresponding to target space range is obtained.This embodiment of the present disclosure can realize the automatic labeling of the map object of target space range, greatly improve the labeling efficiency, reduce the cost of manual labeling.

Description

Technical Field

[0001] This disclosure relates to map annotation technology, and in particular to a method, apparatus, electronic device, and storage medium for determining map objects. Background Technology

[0002] In scenarios such as map annotation and 3D scene reconstruction, it is often necessary to annotate various static elements (or objects) in a vehicle driving scene, such as lane lines, curbs, zebra crossings, drivable areas, stop lines, and landmarks. In related technologies, manual annotation is typically used. However, manual annotation methods are costly and inefficient. Summary of the Invention

[0003] To address the aforementioned technical problems of high cost and low efficiency in manual annotation, embodiments of this disclosure provide a method, apparatus, electronic device, and storage medium for determining map objects, thereby effectively improving annotation efficiency and reducing annotation costs.

[0004] According to one aspect of the present disclosure, a method for determining a map object is provided, comprising: acquiring a first point cloud map corresponding to a target spatial range; determining at least one local point cloud map based on the first point cloud map and a preset sampling rule; determining the attributes and ordered point sets of objects corresponding to each local point cloud map based on each local point cloud map using a pre-trained object detection model, wherein the ordered point sets include a target number of coordinate points in a first coordinate system used to describe the object; and aggregating the attributes and ordered point sets of objects corresponding to each local point cloud map to obtain a map object corresponding to the target spatial range.

[0005] According to another aspect of the present disclosure, a map object determination apparatus is provided, comprising: a first acquisition module, configured to acquire a first point cloud map corresponding to a target spatial range; a first processing module, configured to determine at least one local point cloud map based on the first point cloud map and a preset sampling rule; a second processing module, configured to determine the attributes and ordered point sets of objects corresponding to each of the local point cloud maps using a pre-trained object detection model, wherein the ordered point sets include a target number of coordinate points in a first coordinate system used to describe the objects; and a third processing module, configured to aggregate the attributes and ordered point sets of objects corresponding to each of the local point cloud maps to obtain a map object corresponding to the target spatial range.

[0006] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the map object determination method described in any of the above embodiments of the present disclosure.

[0007] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the map object determination method according to any of the above embodiments of the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided that, when instructions in the computer program product are executed by a processor, performs the map object determination method described in any of the above embodiments of the present disclosure.

[0009] Based on the map object determination method, apparatus, electronic device, and storage medium provided in the above embodiments of this disclosure, by sampling the point cloud map of the target spatial range into at least one local point cloud map, obtaining the attributes and vectorized ordered point sets of each object in each local point cloud map through an object detection model, the automatic annotation of the local point cloud map is realized. Then, the annotation results of each local point cloud map are aggregated to obtain the map object of the target spatial range, thereby realizing the automatic annotation of the map object of the target spatial range, greatly improving the annotation efficiency and reducing the cost of manual annotation.

[0010] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0011] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0012] Figure 1 This is an exemplary application scenario of the map object determination method provided in this disclosure;

[0013] Figure 2 This is a flowchart illustrating a method for determining map objects provided in an exemplary embodiment of this disclosure;

[0014] Figure 3 This is a flowchart illustrating a method for determining map objects provided in another exemplary embodiment of this disclosure;

[0015] Figure 4 This is a schematic diagram of the sampling principle of a local point cloud map provided in an exemplary embodiment of this disclosure;

[0016] Figure 5 This is a flowchart illustrating step 203 provided in another exemplary embodiment of this disclosure;

[0017] Figure 6 This is a schematic diagram of the network structure of an object detection model provided in an exemplary embodiment of this disclosure;

[0018] Figure 7 This is a schematic diagram illustrating the decoding principle of a decoding network provided in an exemplary embodiment of this disclosure;

[0019] Figure 8 This is a flowchart illustrating step 204a provided in an exemplary embodiment of this disclosure;

[0020] Figure 9 This is a flowchart illustrating step 204c provided in an exemplary embodiment of this disclosure;

[0021] Figure 10 This is a flowchart illustrating step 204 provided in another exemplary embodiment of this disclosure;

[0022] Figure 11 This is a flowchart illustrating step 204 provided in yet another exemplary embodiment of this disclosure;

[0023] Figure 12 This is a schematic diagram of the structure of a map object determination device provided in an exemplary embodiment of the present disclosure;

[0024] Figure 13 This is a schematic diagram of the structure of a map object determination device provided in another exemplary embodiment of this disclosure;

[0025] Figure 14 This is a schematic diagram of the structure of the second processing module 503 provided in another exemplary embodiment of this disclosure;

[0026] Figure 15 This is a schematic diagram of the structure of the third processing module 504 provided in another exemplary embodiment of this disclosure;

[0027] Figure 16 This is a schematic diagram of the structure of the third processing module 504 provided in yet another exemplary embodiment of this disclosure;

[0028] Figure 17 This is a structural diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0029] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0030] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0031] This disclosure outlines

[0032] In developing this disclosure, the inventors discovered that in scenarios such as map annotation and 3D scene reconstruction, it is often necessary to annotate various static elements (or objects) of the vehicle driving scene, such as lane lines, curbs, zebra crossings, drivable areas, stop lines, and landmarks. Related technologies typically employ manual annotation. However, manual annotation methods are costly and inefficient.

[0033] Exemplary Overview

[0034] Figure 1 This is an exemplary application scenario of the map object determination method provided in this disclosure.

[0035] In scenarios such as map annotation and 3D reconstruction, the map object determination method disclosed herein (executed within the map object determination device of this disclosure) allows for the determination of at least one local point cloud map after obtaining a first point cloud map corresponding to the target spatial range, based on the first point cloud map and preset sampling rules. Then, based on each local point cloud map, a pre-trained object detection model is used to determine the attributes and ordered point sets of the objects corresponding to each local point cloud map. The ordered point set of an object includes a target number of coordinate points in a first coordinate system used to describe the object. Subsequently, the attributes and ordered point sets of the objects corresponding to each local point cloud map can be aggregated to obtain the map object corresponding to the target spatial range. This achieves automated annotation of the map of the target spatial range, greatly improving annotation efficiency and reducing manual annotation costs. The annotated map of the target spatial range can be used to provide an accurate description of the surrounding static environment information for vehicles traveling within the target spatial range.

[0036] Exemplary methods

[0037] Figure 2 This is a flowchart illustrating a method for determining map objects provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, specifically, for example, in-vehicle computing platforms. Figure 2 As shown, it includes the following steps:

[0038] Step 201: Obtain the first point cloud map corresponding to the target spatial range.

[0039] The target spatial range can be set according to actual needs, such as a city area, an area between cities, an area within a city, etc. The specific settings can be tailored to the specific requirements. The first point cloud map is the point cloud map established within the target spatial range. The first point cloud map can be established using any feasible method, such as establishing the first point cloud map corresponding to the target spatial range based on the Simultaneous Localization and Mapping (SLAM) algorithm; there are no specific limitations.

[0040] In some optional embodiments, when building the first point cloud map, at least one of the following can be stored: the vehicle's driving trajectory and the acquisition time information of the sensor data required for mapping. The driving trajectory may include the vehicle's position or pose corresponding to each time frame as the vehicle travels within the target space. For example, when building the first point cloud map, the vehicle travels along a certain path within the target space. During the journey, the sensors on the vehicle acquire data at a certain frame rate. The onboard computing platform or related devices therein can acquire the sensor data and record relevant information such as the vehicle's pose and timestamp when the data is acquired.

[0041] Step 202: Based on the first point cloud map and the preset sampling rules, determine at least one local point cloud map.

[0042] The preset sampling rules can be set according to actual needs. For example, preset sampling rules can include at least one of sampling rules such as time-based sampling and distance-based sampling. Time-based sampling refers to sampling the first point cloud map according to its mapping time information and a preset time interval to obtain local point cloud maps corresponding to each mapping time in the first point cloud map. The mapping time information can represent the acquisition time of the sensor data used for mapping, or the determination time of the incremental part of the point cloud in incremental mapping. The specific time information can be set according to actual needs. Therefore, the mapping time information is similar to vehicle trajectory information, possessing a certain sequential relationship. Vehicle trajectory information can be represented as a position sequence or pose sequence of the vehicle within the target space, and mapping time information can be represented as a time sequence of the vehicle's travel within the target space. The position sequence or pose sequence corresponds to each part in the first point cloud map, and similarly, the time sequence corresponds to each part in the first point cloud map. Based on this, at least one of time-based sampling and distance-based sampling can be implemented.

[0043] In some alternative embodiments, each local point cloud map is part of a first point cloud map, and adjacent sampled local point cloud maps may have overlapping areas to facilitate subsequent aggregation.

[0044] Step 203: Based on each local point cloud map, use the pre-trained object detection model to determine the attributes and ordered point sets of the objects corresponding to each local point cloud map.

[0045] The ordered point set of an object includes a target number of coordinate points in a first coordinate system used to describe the object. The first coordinate system can be the world coordinate system or a reference coordinate system rigidly connected to the world coordinate system used during mapping; that is, the reference coordinate system and the world coordinate system have a fixed transformation relationship, which can be set according to actual needs. Objects can include map element objects such as lane lines, curbs, arrows, stop lines, lane markings, sidewalks, speed bumps, road surface areas, construction areas, intersection areas, and traffic guidance areas. The number of targets can be set according to actual needs; the number of targets for different types of objects can be the same or different, without specific limitations. Object attributes can include at least one of the following: object type, subtypes under the type, etc. The object types can include the aforementioned types such as lane lines, curbs, arrows, stop lines, lane markings, sidewalks, speed bumps, road surfaces, construction areas, intersection areas, and traffic guidance areas. Subtypes can include more specific attributes of the object within its corresponding type. For example, for the lane line type, subtypes can include line type, color type, and multi-line type. Line type can include solid lines and dashed lines; color type can include white, yellow, blue, orange, and red; and multi-line type can include one-way, double-line, and triple-line. Similarly, for the curb type, subtypes can include flat curbs, roadside curbs, water-filled barriers, and fences. Each type can include at least one subtype. Each object can have at least one subtype. For example, the lane line subtype can include both line and color subtypes simultaneously, without specific limitations. The subtypes included in different types can be set according to actual needs, and this disclosure does not impose any limitations.

[0046] In some alternative embodiments, the object detection model can employ any implementable network structure, such as those based on Transformer and its family of network structures. Specifically, a detection network based on a Deformable Detection Transformer (Deformable DETR) can be configured according to actual needs.

[0047] Step 204: Aggregate the attributes and ordered point sets of the objects corresponding to each local point cloud map to obtain the map object corresponding to the target spatial range.

[0048] Aggregation refers to merging the attributes and ordered point sets of objects corresponding to each local point cloud map into the attributes and ordered point sets of the objects in the overall map. For example, it involves stitching together ordered point sets of different lane lines belonging to the same lane line in different local point cloud maps, merging the attributes of each lane line segment. For the stitched ordered point set, if all attributes are the same, a single attribute can be used to describe it; if the attributes of different segments are different, different attributes can be used to describe them, and so on. Another example is merging the ordered point sets of objects belonging to the same speed bump in different local point cloud maps into the ordered point set of the overall speed bump. For instance, if multiple local point cloud maps each contain a portion of the speed bump, these portions are combined to form the overall speed bump, which is represented by the aggregated ordered point set. For example, it can be represented by the ordered point sets of the four corner points of the aggregated speed bump.

[0049] The map object determination method provided in this embodiment samples the point cloud map of the target spatial range into at least one local point cloud map. It obtains the attributes and vectorized ordered point sets of each object in each local point cloud map through an object detection model, thereby achieving automated annotation of the local point cloud map. Then, it aggregates the annotation results of each local point cloud map to obtain the map object of the target spatial range. This method achieves automated annotation of the map object of the target spatial range, greatly improving annotation efficiency and reducing manual annotation costs. Furthermore, by sampling the point cloud map of the target spatial range into multiple local point cloud maps, it is easy to adapt to the object detection model, which helps to achieve automated map annotation for large-scale scenes.

[0050] Figure 3 This is a flowchart illustrating a method for determining map objects provided in another exemplary embodiment of this disclosure.

[0051] In some optional embodiments, step 201, obtaining the first point cloud map corresponding to the target spatial range, includes:

[0052] Step 2011: Acquire sensor data within the target spatial range.

[0053] The sensor data includes at least one of the following: Inertial Measurement Unit (IMU) data, Global Positioning System (GPS) data, radar data, and image data. Radar data may include at least one of the following: lidar data, millimeter-wave radar data, and ultrasonic radar data.

[0054] Step 2012: Based on sensor data, a first point cloud map corresponding to the target spatial range is established using a preset mapping algorithm.

[0055] The preset mapping algorithm can be set according to actual needs, as long as it can obtain the first point cloud map corresponding to the target spatial range. For example, the preset mapping algorithm can be the Simultaneous Localization and Mapping (SLAM) algorithm.

[0056] This embodiment provides effective global point cloud data for obtaining a map of the target spatial range by establishing a first point cloud map of the target spatial range.

[0057] In some optional embodiments, the first point cloud map includes mapping time information.

[0058] The information on the mapping time can be found in the aforementioned content and will not be repeated here.

[0059] In some optional embodiments, step 202, which determines at least one local point cloud map based on the first point cloud map and preset sampling rules, includes:

[0060] Step 2021: Sample the first point cloud map according to the mapping time information and the preset time interval to obtain local point cloud maps.

[0061] The preset time interval can be set according to actual needs, and this disclosure does not impose any restrictions.

[0062] For example, each mapping time corresponds to a partial point cloud map in the first point cloud map. According to a preset time interval, following the mapping time sequence, the partial point cloud maps corresponding to all mapping times within each time interval are sequentially sampled as a local point cloud map. For instance, if the mapping time information includes a time series from 0 to N, and the preset time interval is n, then m+1 local point cloud maps are sampled from the first point cloud map according to 0 to n-1, n to 2n-1, ..., mn to N.

[0063] This embodiment divides the first point cloud map into multiple smaller local point cloud maps by sampling at time intervals, which facilitates model processing.

[0064] In some optional embodiments, the first point cloud map includes vehicle trajectory information during mapping.

[0065] In some optional embodiments, step 202, which determines at least one local point cloud map based on the first point cloud map and preset sampling rules, includes: sampling the first point cloud map according to vehicle driving trajectory information and preset distance intervals to obtain each local point cloud map.

[0066] For details regarding vehicle trajectory information, please refer to the aforementioned content, which will not be repeated here. Preset distance intervals can be set according to actual needs.

[0067] For example, Figure 4 This is a schematic diagram illustrating the sampling principle of a local point cloud map provided in an exemplary embodiment of this disclosure. In this example, the first point cloud map is sampled according to the vehicle's driving trajectory information and preset distance intervals during the mapping process to obtain various local point cloud maps.

[0068] This embodiment divides the first point cloud map into multiple smaller local point cloud maps by sampling at distance intervals, which facilitates model processing.

[0069] In some alternative embodiments, the attribute includes the type of the object; the object includes objects of at least one major class of linear objects, discrete objects, and region objects, each major class including at least one type.

[0070] Linear objects refer to elements on the map that exhibit linear characteristics, such as lane lines, curbs, and stop lines. Discrete objects refer to elements on the map that exhibit discrete characteristics, such as zebra crossings, speed bumps, no-stopping zones, arrows, diamond-shaped speed bumps, and inverted triangle yield signs. Region objects refer to objects on the map that exhibit area characteristics, such as road surfaces, non-drivable areas, construction zones, general obstacles, intersection areas, and traffic guidance areas. Specific settings can be configured according to actual needs. As can be seen, each major category includes one or more types.

[0071] In some optional embodiments, step 203, based on each local point cloud map and using a pre-trained object detection model, determines the attributes and ordered point sets of the objects corresponding to each local point cloud map, including:

[0072] Step 203a: For any local point cloud map, the pre-trained linear object detection model is used to process the local point cloud map to obtain the type and ordered point set corresponding to each linear object in the local point cloud map.

[0073] The network structure of the linear object detection model is similar to that of the object detection models described above. For example, it may include a feature extraction network, an encoding network, a decoding network, and a detection head network. The network structure for each part can be customized according to specific needs. For instance, the feature extraction network can be based on a convolutional neural network, the encoding and decoding networks can use network structures based on Transformers or their derivatives, and the detection head network can use linear neural networks such as Multilayer Perceptrons (MLPs, or feedforward neural networks). No specific limitations are imposed. The type and ordered point set can be obtained through different head networks; that is, the detection head network can include a head network for regressing the type and a head network for regressing the ordered point set, which can be customized according to specific needs. For linear objects, the ordered point set can include the target number of sampling points on the linear object.

[0074] In some optional embodiments, the training of the linear object detection model can be based on pre-obtained point cloud map samples and corresponding label data. During training, a network loss can be determined based on a preset loss function for iterative updates of the model network parameters until the network loss meets a preset condition or the number of iterations reaches a preset threshold, at which point training ends and a trained linear object detection model is obtained. The preset loss function can be set according to actual needs, and may include a point-to-point loss function and a direction loss function. The point-to-point loss function is used to determine the point-to-point loss based on the error of each point in the predicted ordered point set of linear objects relative to each point in the corresponding ordered point set label. The direction loss function is used to determine the direction loss based on the cosine similarity between the direction vectors of two adjacent points in the predicted ordered point set and the direction vectors of two adjacent points in the corresponding ordered point set label. The point-to-point loss is used to supervise the point-level prediction results of the linear object detection network corresponding to the linear object detection model, enabling the trained linear object detection model to detect the coordinates of objects more accurately. The direction loss is used to supervise the order of points in the predicted ordered point set, enabling the obtained linear object detection model to detect more accurate ordered point sets. Specific settings can be configured according to actual needs.

[0075] For example, the network structure of the linear object detection model can adopt a detection network based on a deformable detection transformer (Deformable DETR). The deformable detection transformer is a detection transformer obtained by improving DETR. It uses a multi-scale variable attention module instead of the attention module in DETR to process features, which solves the problems of high computational complexity and slow convergence of DETR. DETR is the first pure end-to-end object detector, which fully integrates convolutional neural networks (CNN) and transformers. It uses the powerful modeling capabilities of the transformer to achieve object detection, thereby realizing the linear object detection of this disclosure.

[0076] In some optional embodiments, the attributes of a linear object may further include subtypes of the linear object under the corresponding type. Accordingly, the linear object detection model can detect and obtain the type, subtype, and ordered point set corresponding to each object. The type and subtype can be obtained through different head networks or through the same head network, which can be set according to actual needs.

[0077] Step 203b: For any local point cloud map, the pre-trained discrete object detection model is used to process the local point cloud map to obtain the type and ordered point set corresponding to each discrete object in the local point cloud map.

[0078] The discrete object detection model is similar to the linear object detection model mentioned above, except that the shape features of the objects are different. Therefore, the shape formed by the ordered point set is different from that of the linear object. For example, the discrete object can form an ordered point set using the coordinates of the corner points (such as the coordinates of the four corner points). Therefore, the overall network structure of the discrete object detection model can be similar to that of the linear object detection model, and can include a feature extraction network, an encoding network, a decoding network, and a detection head network. It is obtained by training with the corresponding point cloud map samples and the labels corresponding to the discrete objects. The details will not be elaborated further.

[0079] In some optional embodiments, the attributes of discrete objects may further include subtypes of the discrete object under a corresponding type. Accordingly, the discrete object detection model can detect and obtain the type, subtype, and ordered point set corresponding to each object. The type and subtype can be obtained through different head networks or through the same head network, which can be set according to actual needs.

[0080] Step 203c: For any local point cloud map, the pre-trained regional object detection model is used to process the local point cloud map to obtain the type and ordered point set corresponding to each regional object in the local point cloud map.

[0081] The region object detection model is similar to the linear object detection model. Its network structure can include a feature extraction network, an encoding network, a decoding network, and a detection head network. It is trained using corresponding point cloud map samples and labels corresponding to region objects. Details will not be elaborated further.

[0082] In one optional example, the ordered set of points of a region object can be represented as the set of region boundary points of that region object. For example, for a drivable region, it can be represented as the set of region boundary points formed by the boundary points of the drivable region. The shape enclosed by this set of region boundary points can be a polygon.

[0083] In some optional embodiments, the attributes of a region object may further include the subtype of the region object under the corresponding type. Accordingly, the region object detection model can detect and obtain the type, subtype, and ordered point set corresponding to each object. The type and subtype can be obtained through different head networks or through the same head network, which can be set according to actual needs.

[0084] The execution of steps 203a to 203c is not in any particular order.

[0085] This embodiment categorizes map elements into three main types based on their characteristics: linear objects, discrete objects, and regional objects. It then uses three detection models to detect the types of objects and ordered point sets in each of these three categories, thereby refining the granularity of map elements and effectively improving detection accuracy and the precision of the detection results.

[0086] Figure 5 This is a flowchart illustrating step 203 provided in another exemplary embodiment of this disclosure.

[0087] In some optional embodiments, step 203, based on each local point cloud map and using a pre-trained object detection model, determines the attributes and ordered point sets of the objects corresponding to each local point cloud map, including:

[0088] Step 2031: For any local point cloud map, use the feature extraction network in the object detection model to extract features from the local point cloud map to obtain the first feature.

[0089] The feature extraction network can adopt any feasible network structure, such as convolutional neural networks and their series of network structures, which can be set according to actual needs.

[0090] Step 2032: Encode the first feature using the encoding network in the object detection model to obtain the encoding result.

[0091] Step 2033: Use the decoding network in the object detection model to decode the encoding result and obtain the decoding result.

[0092] Step 2034: Use the detection head network in the object detection model to process the decoding results and obtain the attributes and ordered point sets corresponding to each object.

[0093] The object detection model can include the linear object detection model, discrete object detection model, and region object detection model mentioned above. Alternatively, it can be a unified object detection model capable of detecting all three categories of objects simultaneously, depending on the specific requirements.

[0094] In one optional example, Figure 6 This is a schematic diagram of the network structure of an object detection model provided in an exemplary embodiment of this disclosure. The feature extraction network is used to extract features from a local point cloud map to obtain a first feature; the encoding network is used to encode the first feature to obtain an encoding result; the decoding network is used to decode the encoding result to obtain a decoding result; and the detection head network is used to perform detection processing on the decoding result to obtain the attributes and ordered point sets corresponding to each object.

[0095] In one optional example, Figure 7This is a schematic diagram illustrating the decoding principle of a decoding network provided in an exemplary embodiment of this disclosure. The decoding network may include a multilayer decoder and an MLP connected to the decoders. Here, only the decoding of an object by one of the decoders is used as an example for illustration. Here, MLP stands for Multilayer Perceptron, Querying stands for query, Scene Feature Map stands for scene feature map, which is the encoding result obtained by the encoding network, and x and y represent the horizontal and vertical coordinates in the first coordinate system, respectively. For the first decoder (decoder 1), its input query feature 1 is the reference query feature used during model training, and the corresponding ordered point set 1 is the reference ordered point set corresponding to the reference query feature. For other decoders i besides the first decoder, its input query feature i is the decoding result i-1 obtained by the previous decoder i-1, and the ordered point set i is the ordered point set obtained by processing the query feature i through MLP. Decoder i queries features from the nearby area of ​​the corresponding position in the encoding result (Scene Feature Map) based on the ordered point set i, and performs decoding to obtain the decoding result i of decoder i. Decoder i is also connected to MLPi, outputs the decoding result i to MLPi, determines the offset (Δx, Δy) through MLPi, adds it to the ordered point set i to obtain the corresponding ordered point set i+1, and enters the decoding process of the next decoder i+1, and so on. The MLPN connected to the last decoder N is used as the head network for regressing the ordered point set, and the obtained decoding result is passed through MLPN to obtain the ordered point set N+1, which is used as the ordered point set of the detected object. For multiple objects, there may be multiple query features i, which correspond to multiple ordered point sets i, and the details will not be elaborated further.

[0096] In practical applications, for attributes, the detection head network can obtain the probability of belonging to various attributes, and determine the corresponding attribute through the probability threshold. The specific settings can be configured according to actual needs.

[0097] This embodiment effectively detects the attributes and ordered point sets of each object through the feature extraction network, encoding network, decoding network and detection head network of the object detection model, providing accurate and effective local map annotation results for overall map annotation.

[0098] In some alternative embodiments, the object's attributes include the object's type and the object's subtypes under that type, with each type including at least one subtype.

[0099] In some optional embodiments, step 203, based on each local point cloud map and using a pre-trained object detection model, determines the attributes and ordered point sets of the objects corresponding to each local point cloud map, including:

[0100] For any local point cloud map, the object detection model is used to process the local point cloud map to obtain the type of each object in the local point cloud map, the subtype of each object under the corresponding type, and the ordered point set.

[0101] For details on subtypes, please refer to the preceding content; they will not be repeated here. The type, subtype, and ordered point set of each object can be output through different header networks.

[0102] This embodiment can obtain the type of each object in the local point cloud map, the subtype of each object under the corresponding type, and the ordered point set through the object detection model, thereby realizing the detection of more specific attributes of map elements and effectively improving the accuracy of map annotation.

[0103] In some optional embodiments, step 204, which aggregates the attributes and ordered point sets of the objects corresponding to each local point cloud map to obtain the map object corresponding to the target spatial range, includes:

[0104] Step 204a: According to the linear object aggregation rules, aggregate the attributes and ordered point sets of the linear objects corresponding to each local point cloud map to obtain the linear object aggregation result.

[0105] The linear object aggregation rules can be set according to the characteristics of the linear objects. For example, if two linear objects belonging to the same object in adjacent local point cloud maps have the same type and a certain overlap, the aggregation of linear objects in each local point cloud map can be achieved based on this.

[0106] Step 204b: According to the discrete object aggregation rules, aggregate the attributes and ordered point sets of the discrete objects corresponding to each local point cloud map to obtain the discrete object aggregation result.

[0107] Among them, the discrete object aggregation rules can be set according to the characteristics of discrete objects. For example, a discrete object can be represented by the coordinates of the four corner points of the detection box. Then, if two discrete objects belonging to the same object in adjacent local point cloud maps have the same type and the detection boxes have a certain overlap, the aggregation of discrete objects in each local point cloud map can be realized.

[0108] Step 204c: According to the regional object aggregation rules, aggregate the attributes and ordered point sets of the regional objects corresponding to each local point cloud map to obtain the regional object aggregation result.

[0109] The region object aggregation rules can be set according to the characteristics of the region objects. For example, if two region objects belonging to the same object in adjacent local point cloud maps have the same type and the polygons formed by their boundary point sets have overlapping areas, the aggregation of region objects in each local point cloud map can be achieved based on this.

[0110] The execution of steps 204a to 204c is not in any particular order.

[0111] Step 204d: Based on the linear object aggregation results, discrete object aggregation results, and regional object aggregation results, determine the map objects corresponding to the target spatial range.

[0112] The linear object aggregation result includes the attributes and ordered point sets of each aggregated linear object; the discrete object aggregation result includes the attributes and ordered point sets of each aggregated discrete object; and the regional object aggregation result includes the attributes and ordered point sets of each aggregated regional object. By fusing the aggregation results, the attributes and ordered point sets of each object corresponding to the target spatial range can be obtained, thereby obtaining the map object of the target spatial range.

[0113] In some alternative embodiments, the map object corresponding to the obtained target spatial range can be rendered according to its ordered point set and attributes to obtain the labeled map corresponding to the target spatial range.

[0114] This embodiment effectively improves aggregation efficiency by aggregating linear objects, discrete objects, and regional objects in each local point cloud map, and then merges the aggregation results to obtain the map object corresponding to the target spatial range, thus effectively realizing automated map annotation.

[0115] In some alternative embodiments, the ordered set of points of the linear object includes ordered coordinate points on the linear object sampled at a first interval.

[0116] The first interval can be set according to actual needs. The first interval of different linear objects can be the same or different, and this disclosure does not limit it. For example, for any local point cloud map, the lane line of any segment can be represented as multiple ordered coordinate points.

[0117] Figure 8 This is a flowchart illustrating step 204a provided in an exemplary embodiment of this disclosure.

[0118] In some optional embodiments, step 204a, according to the linear object aggregation rules, aggregates the attributes and ordered point sets of the linear objects corresponding to each local point cloud map to obtain the linear object aggregation result, including:

[0119] Step 204a1: For any adjacent first local point cloud map and second local point cloud map in each local point cloud map, take any linear object in the first local point cloud map as the first target linear object.

[0120] The adjacent first and second local point cloud maps can be determined based on at least one of the sampling time point, vehicle position, and vehicle pose corresponding to each local point cloud map. Alternatively, they can be determined by combining the distances between points in each local point cloud map; the specific settings can be configured according to actual needs. For example, for points along the vehicle's travel direction, they can be determined based on at least one of time, position, and pose; for adjacent points along the vehicle's lateral direction, they can be determined based on the distance between vehicle positions or poses, or based on the overlap between local point cloud maps.

[0121] In some alternative embodiments, each linear object in the first local point cloud map can be traversed, and each linear object can be used as the first target linear object for subsequent processing.

[0122] Step 204a2: Based on the attributes and ordered point set corresponding to the first target linear object, determine the second target linear object in the second local point cloud map that matches the first target linear object.

[0123] The second target linear object that matches the first target linear object can be determined based on the matching of the attributes of each linear object with the attributes of the first target linear object, and the intersection-over-union (IOU) ratio of the ordered point sets of each linear object with the ordered point sets of the first target linear object. For example, first, linear objects with the same attributes as the first target linear object are identified from among the linear objects. Then, the intersection-over-union ratio of the ordered point sets of each linear object with the same attributes with the first target linear object is determined. If the intersection-over-union ratio is greater than a threshold, the linear object is determined to be the second target linear object that matches the first target linear object.

[0124] Step 204a3: The attributes and ordered point set of the first target linear object are fused with the attributes and ordered point set of the second target linear object to obtain the first linear fusion result.

[0125] Fusion refers to combining the attributes and ordered point set of the first target linear object with the attributes and ordered point set of the second target linear object in a certain way into the attributes and ordered point set of a larger object.

[0126] In some alternative embodiments, fusion can refer to splicing. During the fusion process, the overlapping portion between the first and second target linear objects can be processed in any feasible manner to ensure that there is a certain interval between adjacent points in the fused ordered point set. For example, the ordered point sets of the two parts can be merged and then curve fitted, and the overall ordered point set can be sampled based on the fitted curve; another example is to directly remove some of the closer points in the overlapping part and retain the other part; yet another example is to find the midpoint between two of the closest points in the overlapping part and replace the original two points; and so on. The goal is simply to obtain the ordered point set corresponding to the fused object.

[0127] In some optional embodiments, the ordered point set of the fused object can be sparsified to reduce the number of points. For example, for linear objects with straight lines, the ordered point set of the fused object can be sampled at preset point intervals to reduce the number of points. The specific settings can be configured according to actual needs. Alternatively, the Douglas-Peucker Algorithm can be used for point sparsification, and the specific settings can be configured according to actual needs.

[0128] Step 204a4: Based on the results of each first linear fusion, determine the linear object aggregation result.

[0129] In some alternative embodiments, the first linear fusion results can be further fused, such as merged, to obtain a linear object aggregation result of the first local point cloud map and the second local point cloud map.

[0130] In some alternative embodiments, all adjacent local point cloud maps can be traversed, and each pair of adjacent local point cloud maps can be used as the first local point cloud map and the second local point cloud map, respectively. The above steps are performed to obtain the linear object aggregation results corresponding to each adjacent local point cloud map. Then, based on the linear object aggregation results corresponding to each adjacent local point cloud map, the overall linear object aggregation result corresponding to all local point cloud maps is determined.

[0131] In some alternative embodiments, adjacent groups can also be processed in parallel to improve processing efficiency.

[0132] In some optional embodiments, the linear object aggregation results corresponding to each adjacent local point cloud map can be merged and deduplicated, and the points can be sparsified to obtain the overall linear object aggregation result.

[0133] This embodiment determines the objects to be aggregated by matching the linear objects in adjacent local point cloud maps, thereby achieving effective aggregation of linear objects and improving the accuracy of aggregation.

[0134] In some alternative embodiments, the ordered point set of discrete objects includes at least one corner point coordinate of the discrete object.

[0135] Exemplarily, the ordered point set of discrete objects may include the ordered coordinate points of the four corner points of the discrete object, which may also be referred to as the detection box of the discrete object.

[0136] In some alternative embodiments, step 204b aggregates the attributes and ordered point sets of the discrete objects corresponding to each local point cloud map according to the discrete object aggregation rule to obtain the discrete object aggregation result, including:

[0137] Based on the attributes and ordered point sets of the discrete objects corresponding to each local point cloud map, the non-maximum suppression algorithm is used to determine the discrete object aggregation result.

[0138] Among them, the non-maximum suppression algorithm (Non-Maximum Suppression, abbreviated as: NMS) is a detection box de-duplication algorithm to remove redundant candidate boxes and obtain the most representative result.

[0139] In some alternative embodiments, during discrete object detection, the probability of the ordered point set (detection box) of the regressed discrete object can also be obtained to assist the execution of the non-maximum suppression algorithm. The specific process is as follows: Based on the attributes and ordered point sets of the discrete objects corresponding to each local point cloud map, discrete object groups belonging to the same target discrete object are determined. For any target discrete object, the ordered point sets of the discrete objects in the discrete object group corresponding to it are sorted according to the corresponding probabilities, such as expressed as A < B < C < D < E < F. First, mark the ordered point set F with the maximum probability as the ordered point set to be retained. Starting from the ordered point set with the maximum probability, it is respectively judged whether the overlap degree (intersection over union of the two detection boxes) of other ordered point sets (A - E) with this maximum ordered point set is greater than the threshold. Remove the other ordered point sets whose overlap degree is greater than the threshold, such as removing B and D. Then select the largest ordered point set E from the remaining other ordered point sets A, C, E, mark it as the ordered point set to be retained, judge the overlap degree between E and A, C, and remove the ordered point sets whose overlap degree is greater than the threshold; and so on until there are no remaining ordered point sets. The four corner point coordinates of the rectangular box formed by all the ordered point sets marked as to be retained are used as the fused ordered point set of this target discrete object.

[0140] This embodiment realizes the aggregation of discrete objects in each local point cloud map through the non-maximum suppression algorithm, can effectively obtain the attributes and ordered point sets of each fused discrete object in the target space range, and realizes the accurate and effective annotation of discrete elements in the overall map.

[0141] In some alternative embodiments, the ordered point set of the regional object includes the sampled point coordinates of the boundary of the regional object.

[0142] Figure 9 This is a flowchart illustrating step 204c provided in an exemplary embodiment of this disclosure.

[0143] In some optional embodiments, step 204c, according to the region object aggregation rules, aggregates the attributes and ordered point sets of the region objects corresponding to each local point cloud map to obtain the region object aggregation result, including:

[0144] Step 204c1: Based on the attributes and ordered point sets of the regional objects corresponding to each local point cloud map, determine the regional object group that belongs to the same target regional object among the regional objects.

[0145] Among them, the group of regional objects belonging to the same target area object can be determined based on the matching of the attributes of regional objects in each local point cloud map and the intersection-union ratio of ordered point sets.

[0146] Step 204c2: For any target region object, determine the sampling point coordinates of the boundary of the target region object based on the ordered point set of each region object in the region object group of the target region object, and use it as the aggregated ordered point set of the target region object.

[0147] The coordinates of the sampling points at the boundary of the target region object can be determined using the Douglas-Peucker Algorithm, based on the ordered point sets of each region object in the corresponding region object group. Specifically, the ordered point sets of each region object can be merged first, and then the Douglas-Peucker Algorithm can be used to sparsify the points to obtain the coordinates of the sampling points at the boundary of the target region object.

[0148] Step 204c3: Determine the aggregation result of the regional objects based on the aggregated ordered point set corresponding to each target regional object.

[0149] This involves merging the aggregated ordered point sets corresponding to each target region object to obtain the region object aggregation result.

[0150] This embodiment aggregates ordered point sets of regional objects belonging to the same target region object group to obtain aggregated ordered point sets of each target region object, thereby obtaining the overall regional object aggregation result of the target spatial range, realizing the automated labeling of regional objects on the target spatial range map.

[0151] Figure 10 This is a flowchart illustrating step 204 provided in another exemplary embodiment of this disclosure.

[0152] In some optional embodiments, step 204, which aggregates the attributes and ordered point sets of the objects corresponding to each local point cloud map to obtain the map object corresponding to the target spatial range, includes:

[0153] Step 2041a: Determine the current local point cloud map.

[0154] This involves iterating through each local point cloud map, taking each local point cloud map as the current local point cloud map, and then performing the following steps on the current local point cloud map.

[0155] Step 2042a: Select any object in the current local point cloud map as the first target object.

[0156] This involves iterating through all objects in the current local point cloud map, taking each object as the first target object, and performing the following steps on the first target object.

[0157] Step 2043a: Based on the attributes and ordered point set corresponding to the first target object, determine the previous aggregated object in the previous object aggregation result that corresponds to the first target object.

[0158] The preceding object aggregation result is the aggregation result obtained first. Different methods can be used to determine the corresponding preceding aggregation object for the first target object of different major categories.

[0159] For example, the first target object is a lane line in the current local point cloud map. The preceding aggregate objects belonging to the lane line can be determined from the preceding aggregate objects. Then, by combining the intersection-union ratio of the ordered point set of the first target object with the ordered point sets of each preceding aggregate object, the preceding aggregate object corresponding to the first target object can be determined.

[0160] Step 2044a: Merge the ordered point set of the first target object with the aggregated ordered point set of the preceding aggregated object to obtain the current aggregated object corresponding to the first target object.

[0161] In this context, the preceding aggregated object is also represented as an attribute and an ordered set of points. Therefore, the fusion of the first target object and the preceding aggregated object can be referred to the fusion or aggregation methods of linear objects, discrete objects, and region objects mentioned above, and will not be repeated here.

[0162] Step 2045a: Determine the aggregation result of the current object based on the current aggregation object corresponding to each object in the current local point cloud map.

[0163] In this process, by aggregating each object in the current local point cloud map with the previously aggregated object, the current object aggregation result of the current aggregation process can be obtained.

[0164] Step 2046a: Take the current object aggregation result as the previous object aggregation result, take the next adjacent local point cloud map of the current local point cloud map as the current local point cloud map, and repeat the step of taking any object in the current local point cloud map as the first target object in step 2042a.

[0165] Step 2047a: In response to the completion of the aggregation of all local point cloud maps, determine the map object corresponding to the target spatial range based on the current object aggregation result.

[0166] After aggregating all local point cloud maps, all aggregated map objects within the target space can be obtained.

[0167] This embodiment effectively aggregates the object attributes and ordered point sets of each local point cloud map through incremental aggregation, thereby achieving automated labeling of the map of the target spatial range.

[0168] Figure 11 This is a flowchart illustrating step 204 provided in another exemplary embodiment of this disclosure.

[0169] In some optional embodiments, step 204, which aggregates the attributes and ordered point sets of the objects corresponding to each local point cloud map to obtain the map object corresponding to the target spatial range, includes:

[0170] Step 2041b: Take any adjacent first local point cloud map and second local point cloud map in each local point cloud map as an adjacent group, and take any object in the first local point cloud map as the second target object.

[0171] Step 2042b: Based on the attributes and ordered point set corresponding to the second target object, determine the third target object that matches the second target object in the second local point cloud map.

[0172] Step 2043b: Merge the attributes and ordered point set of the second target object with the attributes and ordered point set of the third target object to obtain the fusion result corresponding to the adjacent group.

[0173] Step 2044b: Based on the fusion results of each adjacent group, determine the map object corresponding to the target spatial range.

[0174] The specific operations of steps 2041b to 2044b are similar to those of steps 204a1 to 204a4. The difference is that the second target object in this embodiment can be a linear object, a discrete object, or a region object. For the specific operations of different types of objects in each step, please refer to the foregoing content, which will not be repeated here.

[0175] This embodiment aggregates objects in adjacent local point cloud maps to obtain the aggregation results corresponding to adjacent groups, and then merges the aggregation results of each adjacent group to achieve the aggregation of the overall objects in the target spatial range and obtain the map annotation results of the target spatial range.

[0176] This disclosure utilizes a divide-and-conquer approach to segment a large-scale point cloud map into smaller point cloud maps. Based on these smaller maps, a trained object detection model can be used to annotate static elements, effectively improving the precision and accuracy of map annotation through fine-grained annotation. The annotation results from these smaller maps are then aggregated into vectorized maps to obtain a high-precision vectorized map of the large-scale scene. While some semantic segmentation-based map annotation methods exist, their post-processing is complex and cumbersome, and they often cannot handle certain situations, resulting in low annotation accuracy and poor performance. The map object determination method disclosed here achieves point-level annotation, significantly improving annotation precision and accuracy compared to semantic segmentation annotation, thus effectively enhancing the annotation effect.

[0177] The embodiments described above can be implemented individually or in any combination without conflict. The specific implementation can be set according to actual needs, and this disclosure does not limit them.

[0178] Any of the map object determination methods provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any of the map object determination methods provided in this disclosure can be executed by a processor, such as by a processor executing any of the map object determination methods mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0179] Exemplary device

[0180] Figure 12 This is a schematic diagram of a map object determination apparatus provided in an exemplary embodiment of the present disclosure. The apparatus of this embodiment can be used to implement corresponding method embodiments of the present disclosure, such as… Figure 12 The device shown includes: a first acquisition module 501, a first processing module 502, a second processing module 503, and a third processing module 504.

[0181] The first acquisition module 501 is used to acquire the first point cloud map corresponding to the target spatial range.

[0182] The first processing module 502 is used to determine at least one local point cloud map based on the first point cloud map and preset sampling rules.

[0183] The second processing module 503 is used to determine the attributes and ordered point sets of objects corresponding to each local point cloud map based on each local point cloud map and using a pre-trained object detection model. The ordered point set includes a number of target coordinate points in the first coordinate system used to describe the object.

[0184] The third processing module 504 is used to aggregate the attributes and ordered point sets of the objects corresponding to each local point cloud map to obtain the map object corresponding to the target spatial range.

[0185] Figure 13 This is a schematic diagram of the structure of a map object determination device provided in another exemplary embodiment of this disclosure.

[0186] In some optional embodiments, the first acquisition module 501 includes:

[0187] The first acquisition unit 5011 is used to acquire sensor data within the target space range.

[0188] The sensor data includes at least one of inertial measurement unit data, global positioning data, radar data, and image data.

[0189] The mapping unit 5012 is used to establish a first point cloud map corresponding to the target spatial range based on sensor data and using a preset mapping algorithm.

[0190] In some optional embodiments, the first point cloud map includes mapping time information.

[0191] In some optional embodiments, the first processing module 502 includes:

[0192] The first sampling unit 5021 is used to sample the first point cloud map according to the mapping time information and the preset time interval to obtain the local point cloud maps.

[0193] In some optional embodiments, the first point cloud map includes vehicle trajectory information during mapping.

[0194] In some optional embodiments, the first processing module 502 includes:

[0195] The first sampling unit 5021 is used to sample the first point cloud map according to the vehicle driving trajectory information and the preset distance interval to obtain local point cloud maps.

[0196] In some alternative embodiments, the attribute includes the type of the object; the object includes objects of at least one major class of linear objects, discrete objects, and region objects, each major class including at least one type.

[0197] In some optional embodiments, the second processing module 503 includes:

[0198] The first processing unit 503a is used to process any local point cloud map using a pre-trained linear object detection model to obtain the type and ordered point set corresponding to each linear object in the local point cloud map.

[0199] The second processing unit 503b is used to process any local point cloud map using a pre-trained discrete object detection model to obtain the type and ordered point set corresponding to each discrete object in the local point cloud map.

[0200] The third processing unit 503c is used to process any local point cloud map using a pre-trained regional object detection model to obtain the type and ordered point set corresponding to each regional object in the local point cloud map.

[0201] Figure 14 This is a schematic diagram of the structure of the second processing module 503 provided in another exemplary embodiment of this disclosure.

[0202] In some optional embodiments, the second processing module 503 includes:

[0203] The feature extraction unit 5031 is used to extract features from any local point cloud map using the feature extraction network in the object detection model to obtain the first feature.

[0204] The encoding unit 5032 is used to encode the first feature using the encoding network in the object detection model to obtain the encoding result.

[0205] The decoding unit 5033 is used to decode the encoded result using the decoding network in the object detection model to obtain the decoded result.

[0206] The detection unit 5034 is used to process the decoding results using the detection head network in the object detection model to obtain the attributes and ordered point sets corresponding to each object.

[0207] In some alternative embodiments, the object's attributes include the object's type and the object's subtypes under that type, with each type including at least one subtype.

[0208] In some optional embodiments, the second processing module 503 is specifically used for:

[0209] For any local point cloud map, the object detection model is used to process the local point cloud map to obtain the type of each object in the local point cloud map, the subtype of each object under the corresponding type, and the ordered point set.

[0210] In some optional embodiments, the third processing module 504 includes:

[0211] The first aggregation unit 504a is used to aggregate the attributes and ordered point sets of the linear objects corresponding to each local point cloud map according to the linear object aggregation rules, so as to obtain the linear object aggregation result.

[0212] The second aggregation unit 504b is used to aggregate the attributes and ordered point sets of the discrete objects corresponding to each local point cloud map according to the discrete object aggregation rules, so as to obtain the discrete object aggregation result.

[0213] The third aggregation unit 504c is used to aggregate the attributes and ordered point sets of the regional objects corresponding to each local point cloud map according to the regional object aggregation rules, so as to obtain the regional object aggregation result.

[0214] The fourth aggregation unit 504d is used to determine the map object corresponding to the target spatial range based on the aggregation results of linear objects, discrete objects, and regional objects.

[0215] In some alternative embodiments, the ordered set of points of the linear object includes ordered coordinate points on the linear object sampled at a first interval.

[0216] In some optional embodiments, the first aggregation unit 504a is specifically used for:

[0217] For any adjacent first and second local point cloud maps in each local point cloud map, take any linear object in the first local point cloud map as the first target linear object; based on the attributes and ordered point set corresponding to the first target linear object, determine the second target linear object in the second local point cloud map that matches the first target linear object; fuse the attributes and ordered point set of the first target linear object with the attributes and ordered point set of the second target linear object to obtain the first linear fusion result; based on each first linear fusion result, determine the linear object aggregation result.

[0218] In some alternative embodiments, the ordered set of points of the discrete object includes the coordinates of at least one corner point of the discrete object.

[0219] In some optional embodiments, the second aggregation unit 504b is specifically used for:

[0220] Based on the attributes and ordered point sets of the discrete objects corresponding to each local point cloud map, a non-maximum suppression algorithm is used to determine the aggregation result of the discrete objects.

[0221] In some alternative embodiments, the ordered set of points of the region object includes the coordinates of sampled points of the boundary of the region object.

[0222] In some optional embodiments, the third aggregation unit 504c is specifically used for:

[0223] Based on the attributes and ordered point sets of the regional objects corresponding to each local point cloud map, the regional object groups belonging to the same target regional object are determined. For any target regional object, the sampling point coordinates of the boundary of the target regional object are determined based on the ordered point sets of each regional object in the regional object group of the target regional object, which are used as the aggregated ordered point set of the target regional object. Based on the aggregated ordered point sets corresponding to each target regional object, the regional object aggregation result is determined.

[0224] Figure 15 This is a schematic diagram of the structure of the third processing module 504 provided in another exemplary embodiment of this disclosure.

[0225] In some optional embodiments, the third processing module 504 includes:

[0226] The first determining unit 5041a is used to determine the current local point cloud map.

[0227] The second determining unit 5042a is used to take any object in the current local point cloud map as the first target object.

[0228] The third determining unit 5043a is used to determine the previous aggregated object corresponding to the first target object in the previous object aggregation result based on the attributes and ordered point set corresponding to the first target object.

[0229] Among them, the first object aggregation result is the object aggregation result obtained first.

[0230] The fusion unit 5044a is used to fuse the ordered point set of the first target object with the aggregated ordered point set of the preceding aggregated object to obtain the current aggregated object corresponding to the first target object.

[0231] The fourth determining unit 5045a is used to determine the current object aggregation result based on the current aggregation object corresponding to each object in the current local point cloud map.

[0232] The fifth determining unit 5046a is used to take the current object aggregation result as the previous object aggregation result, take the next adjacent local point cloud map of the current local point cloud map as the current local point cloud map, and the second determining unit 5042a repeatedly executes the step of taking any object in the current local point cloud map as the first target object.

[0233] The sixth determining unit 5047a is used to determine the map object corresponding to the target spatial range based on the current object aggregation result in response to the completion of the aggregation of all local point cloud maps.

[0234] Figure 16This is a schematic diagram of the structure of the third processing module 504 provided in another exemplary embodiment of this disclosure.

[0235] In some optional embodiments, the third processing module 504 includes:

[0236] The fourth processing unit 5041b is used to treat any adjacent first local point cloud map and second local point cloud map in each local point cloud map as an adjacent group, and to treat any object in the first local point cloud map as a second target object.

[0237] The fifth processing unit 5042b is used to determine a third target object in the second local point cloud map that matches the second target object based on the attributes and ordered point set corresponding to the second target object.

[0238] The sixth processing unit 5043b is used to fuse the attributes and ordered point set of the second target object with the attributes and ordered point set of the third target object to obtain the fusion result corresponding to the adjacent group.

[0239] The seventh processing unit 5044b is used to determine the map object corresponding to the target spatial range based on the fusion results of each adjacent group.

[0240] The beneficial technical effects corresponding to the exemplary embodiments of this device can be found in the corresponding beneficial technical effects of the exemplary method section above, and will not be repeated here.

[0241] Exemplary electronic devices

[0242] Figure 17 This is a structural diagram of an electronic device provided in an embodiment of the present disclosure, including at least one processor 11 and a memory 12.

[0243] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0244] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute one or more computer program instructions to implement the methods and / or other desired functions of the various embodiments of this disclosure described above.

[0245] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0246] The input device 13 may also include, for example, a keyboard, a mouse, etc.

[0247] The output device 14 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0248] Of course, for the sake of simplicity, Figure 17 Only some of the components of the electronic device 10 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0249] Exemplary computer program products and computer-readable storage media

[0250] In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods in the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0251] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0252] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the methods in the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0253] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0254] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0255] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A method for determining map objects, comprising: Obtain the first point cloud map corresponding to the target spatial range; Based on the first point cloud map and the preset sampling rules, at least one local point cloud map is determined; Based on each of the local point cloud maps, using a pre-trained object detection model, the attributes and ordered point sets of the objects corresponding to each of the local point cloud maps are determined. The ordered point sets include a number of target coordinate points in a first coordinate system used to describe the objects. The attributes and ordered point sets of the objects corresponding to each of the local point cloud maps are aggregated to obtain the map object corresponding to the target spatial range; the aggregation refers to merging the attributes and ordered point sets of the objects corresponding to each of the local point cloud maps into the attributes and ordered point sets of the overall object within the target spatial range.

2. The method according to claim 1, wherein, The attribute includes the type of the object; the object includes objects of at least one major category of linear objects, discrete objects, and region objects, and each major category includes at least one type. The step of determining the attributes and ordered point sets of objects corresponding to each of the local point cloud maps using a pre-trained object detection model includes: For any of the aforementioned local point cloud maps, the local point cloud map is processed using a pre-trained linear object detection model to obtain the type and ordered point set corresponding to each linear object in the local point cloud map. The local point cloud map is processed using a pre-trained discrete object detection model to obtain the type and ordered point set corresponding to each discrete object in the local point cloud map. The local point cloud map is processed using a pre-trained regional object detection model to obtain the type and ordered point set corresponding to each regional object in the local point cloud map.

3. The method according to claim 1, wherein, The step of determining the attributes and ordered point sets of objects corresponding to each of the local point cloud maps using a pre-trained object detection model includes: For any of the local point cloud maps, the feature extraction network in the object detection model is used to extract features from the local point cloud map to obtain a first feature; The first feature is encoded using the encoding network in the object detection model to obtain the encoding result; The encoded result is decoded using the decoding network in the object detection model to obtain the decoded result; The decoding results are processed using the detection head network in the object detection model to obtain the attributes and ordered point sets corresponding to each object.

4. The method according to claim 1, wherein, The attribute includes the type of the object and the subtypes of the object under that type, and each type includes at least one subtype; The step of determining the attributes and ordered point sets of objects corresponding to each of the local point cloud maps using a pre-trained object detection model includes: For any of the local point cloud maps, the object detection model is used to process the local point cloud map to obtain the type corresponding to each object in the local point cloud map, the subtype of each object under the corresponding type, and the ordered point set.

5. The method according to claim 1, wherein, The step of aggregating the attributes and ordered point sets of the objects corresponding to each of the local point cloud maps to obtain the map object corresponding to the target spatial range includes: According to the linear object aggregation rules, the attributes and ordered point sets of the linear objects corresponding to each local point cloud map are aggregated to obtain the linear object aggregation result; According to the discrete object aggregation rules, the attributes and ordered point sets of the discrete objects corresponding to each of the local point cloud maps are aggregated to obtain the discrete object aggregation result; According to the regional object aggregation rules, the attributes and ordered point sets of the regional objects corresponding to each of the local point cloud maps are aggregated to obtain the regional object aggregation result; Based on the linear object aggregation result, the discrete object aggregation result, and the regional object aggregation result, the map object corresponding to the target spatial range is determined.

6. The method according to claim 5, wherein, The ordered point set of the linear object includes ordered coordinate points on the linear object sampled at a first interval; The process of aggregating the attributes and ordered point sets of the linear objects corresponding to each local point cloud map according to the linear object aggregation rules to obtain the linear object aggregation result includes: For any adjacent first local point cloud map and second local point cloud map in each of the local point cloud maps, any linear object in the first local point cloud map is taken as the first target linear object. Based on the attributes and ordered point set corresponding to the first target linear object, a second target linear object that matches the first target linear object is determined in the second local point cloud map; The attributes and ordered point set of the first target linear object are fused with the attributes and ordered point set of the second target linear object to obtain a first linear fusion result; Based on each of the first linear fusion results, the linear object aggregation result is determined.

7. The method according to claim 5, wherein, The ordered set of points of the discrete object includes the coordinates of at least one corner point of the discrete object; The step of aggregating the attributes and ordered point sets of discrete objects corresponding to each local point cloud map according to discrete object aggregation rules to obtain discrete object aggregation results includes: Based on the attributes of the discrete objects corresponding to each of the local point cloud maps and the ordered point sets, a non-maximum suppression algorithm is used to determine the aggregation result of the discrete objects.

8. The method according to claim 5, wherein, The ordered set of points of the region object includes the coordinates of the sampled points of the boundary of the region object; The step of aggregating the attributes and ordered point sets of the regional objects corresponding to each of the local point cloud maps according to the regional object aggregation rules to obtain the regional object aggregation result includes: Based on the attributes and ordered point sets of the regional objects corresponding to each of the local point cloud maps, a group of regional objects belonging to the same target regional object is determined among the regional objects. For any target region object, the sampling point coordinates of the boundary of the target region object are determined based on the ordered point set of each region object in the region object group of the target region object, and are used as the aggregated ordered point set of the target region object; Based on the aggregated ordered point set corresponding to each of the target region objects, the aggregation result of the region objects is determined.

9. The method according to claim 1, wherein, The attributes and ordered point sets of the objects corresponding to each of the local point cloud maps are aggregated to obtain the map object corresponding to the target spatial range, including: Determine the current local point cloud map; Take any object in the current local point cloud map as the first target object; Based on the attributes and ordered point set corresponding to the first target object, determine the preceding aggregated object corresponding to the first target object in the preceding object aggregation result, wherein the preceding object aggregation result is the object aggregation result obtained in advance; The ordered point set of the first target object is merged with the aggregated ordered point set of the preceding aggregated object to obtain the current aggregated object corresponding to the first target object; Based on the current aggregated object corresponding to each of the objects in the current local point cloud map, determine the current object aggregation result; The current object aggregation result is used as the previous object aggregation result, the next adjacent local point cloud map of the current local point cloud map is used as the current local point cloud map, and the step of taking any object in the current local point cloud map as the first target object is repeated. In response to the completion of the aggregation of all the local point cloud maps, the map object corresponding to the target spatial range is determined based on the current object aggregation result.

10. The method according to claim 1, wherein, The step of aggregating the attributes and ordered point sets of the objects corresponding to each of the local point cloud maps to obtain the map object corresponding to the target spatial range includes: Take any adjacent first local point cloud map and second local point cloud map in each of the local point cloud maps as an adjacent group, and take any object in the first local point cloud map as the second target object. Based on the attributes and ordered point set corresponding to the second target object, a third target object matching the second target object is determined in the second local point cloud map; The attributes and ordered point set of the second target object are fused with the attributes and ordered point set of the third target object to obtain the fusion result corresponding to the adjacent group; Based on the fusion results corresponding to each adjacent group, the map object corresponding to the target spatial range is determined.

11. The method according to claim 1, wherein, The first point cloud map includes map creation time information; The step of determining at least one local point cloud map based on the first point cloud map and preset sampling rules includes: According to the mapping time information and the preset time interval, the first point cloud map is sampled to obtain each of the local point cloud maps.

12. The method according to claim 1, wherein, The first point cloud map includes vehicle trajectory information during map creation; The step of determining at least one local point cloud map based on the first point cloud map and preset sampling rules includes: Based on the vehicle's driving trajectory information and preset distance intervals, the first point cloud map is sampled to obtain each of the local point cloud maps.

13. The method according to claim 1, wherein, The acquisition of the first point cloud map corresponding to the target spatial range includes: Acquire sensor data within the target spatial range, wherein the sensor data includes at least one of inertial measurement unit data, global positioning data, radar data, and image data; Based on the sensor data, a preset mapping algorithm is used to establish the first point cloud map corresponding to the target spatial range.

14. A device for determining map objects, comprising: The first acquisition module is used to acquire the first point cloud map corresponding to the target spatial range; The first processing module is used to determine at least one local point cloud map based on the first point cloud map and preset sampling rules. The second processing module is used to determine the attributes and ordered point sets of objects corresponding to each local point cloud map based on each of the local point cloud maps and using a pre-trained object detection model. The ordered point sets include a number of target coordinate points in a first coordinate system used to describe the object. The third processing module is used to aggregate the attributes and ordered point sets of the objects corresponding to each of the local point cloud maps to obtain the map object corresponding to the target spatial range; the aggregation refers to merging the attributes and ordered point sets of the objects corresponding to each of the local point cloud maps into the attributes and ordered point sets of the overall object within the target spatial range.

15. A computer-readable storage medium storing a computer program for performing the method for determining map objects according to any one of claims 1-13.

16. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method for determining map objects according to any one of claims 1-13.

17. A computer program product, when the instructions in the computer program product are executed by a processor, performs the method for determining map objects as described in any one of claims 1-13.

Images