Method and device for constructing map, electronic equipment and computer readable medium

Abstract

The invention relates to a method and device for constructing a map, electronic equipment and a computer readable medium, and relates to the field of artificial intelligence. The method for constructing the map comprises the steps that a heat map is generated based on a plurality of first curves, and the first curves are used for identifying map elements in at least one local area in the map to be constructed; based on the heat map, a second curve is obtained through the AI network, and the second curve is used for identifying map elements in the map to be constructed. Optionally, the method executed by the electronic equipment can be executed by using an artificial intelligence model. According to the method for constructing the map provided by the embodiment of the invention, the input information is fully utilized, so that the method has higher robustness; according to the method, the global map construction problem is converted into the set prediction problem, the heat map serves as input, different use cases can be processed in a unified mode, excessive threshold values are prevented from being used, and better universality is achieved.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to the field of artificial intelligence, and more particularly to methods, apparatuses, electronic devices, and computer-readable media for constructing maps. Background Art

[0002] The construction of a high-definition map (HD map) can be regarded as the prediction of a set of vectorized static map elements (e.g., crosswalks, lane dividers, road boundaries, etc.) in a bird's eye view (BEV).

[0003] HD maps are mainly used in the field of autonomous driving and are generally composed of instance-level vectorized representations of map elements. HD maps provide rich and accurate static environment information of driving scenarios, which is crucial for downstream tasks such as autonomous driving system planning and HD map automatic annotation systems. HD maps include a global high-definition map (Global HD Map) and a local high-definition map (Local HD Map). The local HD map generally refers to a close-range map (e.g., about 60 meters in distance) and is usually created using one frame of data, while the global HD map generally refers to a long-range map (e.g., dozens to hundreds of meters) and is usually created using all the data of a scene (e.g., a sequence of multiple frames).

[0004] Currently, the global map construction method in vectorized map annotation (VMA) is a curve-based map construction method that iteratively fuses local curves in an incremental and serialized manner. That is, all unit vectorized maps are merged into a global vectorized map in an incremental and serialized manner. The global map construction method of VMA has problems of poor robustness and generality.

[0005] Therefore, there is a need for a map construction method with better robustness and generality. Summary of the Invention

[0006] Generally, example embodiments of the present disclosure relate to a technical solution for generating a heat map from input local curves, predicting a global curve through the heat map, and then generating an HD map, so as to improve the robustness and generality of HD map construction.

[0007] In one aspect of the disclosure, a method for constructing a map is provided, including: generating a heat map based on multiple first curves, where the first curves are used to identify map elements in at least one local area of the map to be constructed; obtaining a second curve through an AI network based on the heat map, where the second curve is used to identify map elements in the map to be constructed.

[0008] In some embodiments, the heat map includes rasterized curves, and generating the heat map based on multiple first curves includes: generating the rasterized curves based on the coordinate information of points on the multiple first curves and the confidence of each first curve, where the coordinate information of points on the rasterized curves is determined based on the coordinate information of corresponding points on the multiple first curves, and the intensity information of points on the rasterized curves is determined based on the confidence of the first curve including a point with the same coordinate information as the point.

[0009] In some embodiments, the heat map includes rasterized curves, and generating the heat map based on multiple first curves includes: generating the rasterized curves based on the coordinate information of points on the multiple first curves and the confidence of each first curve, where the coordinate information of points on the rasterized curves is determined based on the coordinate information of corresponding points on the multiple first curves, and the intensity information of points on the rasterized curves is determined based on the confidence of the first curve including a point with the same coordinate information as the point.

[0010] In some embodiments, generating the heat map based on multiple first curves further includes: generating a second region based on the coordinate information of points in a first region corresponding to the multiple first curves, where the coordinate information of points on the second region is determined based on the coordinate information of corresponding points in the first region, and the intensity information of points on the second region is determined based on the number of points in the first region corresponding to the point; obtaining the average intensity of the rasterized curves based on the rasterized curves and the second region.

[0011] In some embodiments, before generating the rasterized curves based on the coordinate information of points on the multiple first curves and the confidence of each first curve, it further includes: setting the intensity information of points within a bounding box corresponding to the multiple first curves to a predetermined value.

[0012] In some embodiments, before the coordinate information of points on the multiple first curves and the confidence of each first curve, it further includes: using a coordinate transformation matrix to transform the point coordinates of the multiple first curves and the bounding box to the same coordinate system.

[0013] In some embodiments, the multiple first curves include: a first type of curve that can form a closed region, and a second type of curve other than the first type of curve. Generating a heat map based on the multiple first curves further includes: internally filling the part of the rasterized curve corresponding to the first type of curve to generate a first-class heat map.

[0014] In some embodiments, the multiple first curves include: a first type of curve that can form a closed region, and a second type of curve other than the first type of curve. Generating a heat map based on the multiple first curves further includes: generating a second-class heat map based on the part of the rasterized curve corresponding to the second type of curve.

[0015] In some embodiments, before generating a second-class heat map based on the part of the rasterized curve corresponding to the second type of curve, it further includes: normalizing the average intensity of each rasterized curve.

[0016] In some embodiments, before normalizing the average intensity of each rasterized curve, it further includes: dilating the part of the rasterized curve corresponding to the second type of curve.

[0017] In some embodiments, obtaining a second curve based on the heat map through an AI network includes: predicting a first type of curve based on the first-class heat map through a first AI network; predicting a second type of curve based on the second-class heat map through a second AI network; obtaining the second curve based on the predicted first type of curve and second type of curve.

[0018] In some embodiments, obtaining a second curve based on the heat map through an AI network includes: selecting at least one second type of curve from the predicted second type of curves based on the second-class heat map and determining it as the second type of curve in the second curve.

[0019] In some embodiments, predicting the first type of curve in the second curve based on the heat map through a first AI sub-network includes: converting the first-class heat map into a binary map using a threshold; determining connected regions from the binary map; extracting the boundaries of each connected region and determining them as the first type of curve; calculating the average confidence within each connected region; normalizing the average confidence of each connected region to obtain the confidence of the first type of curve corresponding to the connected region.

[0020] In some embodiments, normalizing the average confidence of each connected region includes: normalizing the average confidence of each connected region based on the maximum value in the average confidence of the connected region.

[0021] In some embodiments, predicting the second type of curve in the second curve based on the second type of heatmap through a second AI sub-network includes: extracting second type of features from the second type of heatmap; using an AI algorithm to predict the coordinate information and confidence of the second type of curve in the second curve based on the second type of features.

[0022] In some embodiments, selecting at least one second type of curve from the predicted second type of curves as the second type of curve in the second curve based on the second type of heatmap includes: selecting at least one second type of curve from the predicted second type of curves, whose similarity to the second type of heatmap is within a set range, and determining it as the second type of curve in the second curve.

[0023] In some embodiments, selecting at least one second type of curve from the predicted second type of curves, whose similarity to the second type of heatmap is within a set range, as the second type of curve in the second curve includes: for the predicted second type of curves, determining the second type of curves in which the similarity to the second type of heatmap is greater than or equal to a threshold; updating the second type of heatmap, wherein the intensity values of the points corresponding to the determined second type of curves and the points within a set range of this point in the second type of heatmap are set to a preset value; continuing to execute the step of determining the second type of curves in which the similarity to the second type of heatmap is greater than or equal to the threshold for the predicted second type of curves until there are no second type of curves in the predicted second type of curves whose similarity to the second type of heatmap is within the set range, and the determined second type of curves are all the second type of curves in the second curve.

[0024] In some embodiments, the method further includes: normalizing the confidence of all the selected second type of curves to determine the confidence of the second type of curves.

[0025] In another aspect of the disclosure, an electronic device is provided. The electronic device includes: at least one processor; and at least one memory, where the at least one memory stores instructions, and when the instructions are executed by the at least one processor, the above-described method is implemented.

[0026] In yet another aspect of the present disclosure, a non-transitory computer-readable medium is provided, on which instructions are stored, and when the instructions are executed by at least one processing unit, at least one processing unit is configured to execute the above-described method.

[0027] It should be understood that the content described in the summary of the invention is not intended to limit the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Brief Description of the Drawings

[0028] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without exceeding the scope of protection required by the present application.

[0029] Figure 1A Shows a method for constructing a map provided by an embodiment of the present disclosure.

[0030] Figure 1B Shows a method for constructing a map provided by an embodiment of the present disclosure.

[0031] Figure 2 Shows a flowchart of generating a heat map based on local curves in the method for constructing a map provided by an embodiment of the present disclosure.

[0032] Figure 3 Shows an example of the input local curve and the output heat map of the method for constructing a map provided by an embodiment of the present disclosure.

[0033] Figure 4 Shows a flowchart of generating a global curve based on the heat map in the method for constructing a map provided by an embodiment of the present disclosure.

[0034] Figure 5 Shows a flowchart of predicting a polygon area in the global curve in the method for constructing a map provided by an embodiment of the present disclosure.

[0035] Figure 6 Shows a flowchart of predicting a broken line in the global curve in the method for constructing a map provided by an embodiment of the present disclosure.

[0036] Figure 7A Shows a device for constructing a map provided by an embodiment of the present disclosure.

[0037] Figure 7B Shows a device for constructing a map provided by an embodiment of the present disclosure.

[0038] Figure 8 Shows the output image of the device for constructing a map provided by an embodiment of the present disclosure and the heat map generation unit.

[0039] Figure 9 Shows the device for constructing a map provided by an embodiment of the present disclosure and its internal units.

[0040] Figure 10A simplified block diagram of an electronic device suitable for implementing an embodiment of the present disclosure is shown.

[0041] Figure 11 A schematic diagram of a computer-readable medium suitable for implementing an embodiment of the present disclosure is shown.

[0042] In all the drawings, the same or similar reference numerals denote the same or similar elements. Detailed implementation manners

[0043] The following description with reference to the accompanying drawings is provided to facilitate a comprehensive understanding of various embodiments of the present disclosure defined by the claims and their equivalents. This description includes various specific details to facilitate understanding but should only be considered exemplary. Therefore, those of ordinary skill in the art will recognize that various changes and modifications can be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and structures may be omitted for clarity and conciseness.

[0044] The terms and phrases used in the following specification and claims are not limited to their dictionary meanings, but are used solely by the inventors to enable a clear and consistent understanding of the present disclosure. Therefore, it should be apparent to those skilled in the art that the following description of the various embodiments of the present disclosure is provided for illustrative purposes only and not for the purpose of limiting the present disclosure as defined by the appended claims and their equivalents.

[0045] It should be understood that the singular forms "a", "an", and "the" may also include plural referents unless the context clearly indicates otherwise. Thus, for example, a reference to "a component surface" includes a reference to one or more such surfaces. When we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may mean that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include a wireless connection or wireless coupling.

[0046] The term "comprising" or "may comprise" refers to the presence of the corresponding disclosed functions, operations, or components that can be used in various embodiments of the present disclosure, rather than limiting the presence of one or more additional functions, operations, or features. In addition, the term "comprising" or "having" may be interpreted to mean that certain characteristics, numbers, steps, operations, components, components, or combinations thereof are included, but should not be interpreted to exclude the possibility of the presence of one or more other characteristics, numbers, steps, operations, components, components, or combinations thereof.

[0047] The term "or" as used in various embodiments of the present disclosure includes any of the recited terms and all combinations thereof. For example, "A or B" may include A, may include B, or may include both A and B. When describing multiple (two or more) items, if the relationship between the multiple items is not explicitly defined, the multiple items may refer to one, more, or all of the multiple items. For example, for the description of "parameter A includes A1, A2, A3", it may be implemented such that parameter A includes A1 or A2 or A3, or it may also be implemented such that parameter A includes at least two of the three items A1, A2, and A3.

[0048] Unless otherwise defined, all terms (including technical or scientific terms) used in the present disclosure have the same meaning as understood by those skilled in the art to which the present disclosure pertains. Commonly used terms defined in a dictionary are interpreted to have a meaning consistent with the context in the relevant technical field, and should not be interpreted idealistically or overly formally unless explicitly defined as such in the present disclosure.

[0049] At least some of the functions in the devices or electronic devices provided in the embodiments of the present disclosure can be implemented by an AI model. For example, at least one of the multiple modules of the device or electronic device can be implemented by an AI model. The functions associated with AI can be executed by a non-volatile memory, a volatile memory, and a processor.

[0050] The processor may include one or more processors. At this time, the one or more processors may be general-purpose processors, such as a central processing unit (CPU), an application processor (AP), etc., or may be a pure graphics processing unit, such as a graphics processing unit (GPU), a vision processing unit (VPU), and / or an AI-specific processor, such as a neural processing unit (NPU).

[0051] The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence (AI) models stored in the non-volatile memory and the volatile memory. The predefined operation rules or artificial intelligence models are provided through training or learning.

[0052] Here, providing through learning means obtaining predefined operation rules or an AI model with desired characteristics by applying a learning algorithm to multiple learning data. The learning can be performed in the device or electronic device itself that executes the AI according to the embodiments, and / or can be implemented by a separate server / system.

[0053] An AI model may include multiple neural network layers. Each layer has multiple weight values, and each layer performs neural network calculations through the calculation between the input data of the layer (such as the calculation result of the previous layer and / or the input data of the AI model) and the multiple weight values of the current layer. Examples of neural networks include, but are not limited to, convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), generative adversarial networks (GANs), and deep Q networks.

[0054] A learning algorithm is a method of using multiple learning data to train a predetermined target device (e.g., a robot) to enable, allow, or control the target device to make a determination or prediction. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0055] The method provided by the present disclosure may relate to one or more fields in the field of image technology.

[0056] According to the present disclosure, in a method for constructing a map executed in an electronic device, output data may be obtained by using image data as input data of an artificial intelligence model. The artificial intelligence model may be obtained through training. Here, "obtained through training" means obtaining a predefined operation rule or artificial intelligence model configured to perform an expected feature (or purpose) by training a basic artificial intelligence model with multiple training data using a training algorithm. The method of the present disclosure may relate to the field of visual understanding of artificial intelligence technology, which is a technology for recognizing and processing things like human vision, and includes, for example, object recognition, object tracking, image retrieval, human recognition, scene recognition, 3D reconstruction / positioning, or image enhancement.

[0057] The following will refer to FIGS. 1 to Figure 11 Describe a method, apparatus, electronic device, and computer-readable medium for constructing a map according to an embodiment of the present disclosure. It should be noted that the following embodiments may refer to, draw on, or combine with each other. For the same terms, similar features, and similar implementation steps in different embodiments, they will not be described repeatedly.

[0058] Figure 1A Illustrates a method for constructing a map provided by an embodiment of the present disclosure.

[0059] The method for constructing a map according to an embodiment of the present disclosure can predict the output by using all the information of the local curve without iteration, and thus has higher robustness and generality.

[0060] As Figure 1AAs shown, the method 10 for constructing a map according to an embodiment of the present disclosure includes steps S101 to S102.

[0061] In step S101, a heat map is generated based on multiple local curves (also referred to as "first curves"), where the local curves are used to identify map elements in at least one local area of the map to be constructed.

[0062] In some embodiments of the present disclosure, the multiple local curves may be identification lines of map elements obtained based on images (which may also be referred to as local images) collected by a camera (such as environmental images captured by a camera on a vehicle during the vehicle's travel), or may be identification lines of map elements obtained based on point cloud data detected by a radar (such as point cloud data of the surrounding environment detected by a radar on a vehicle during the vehicle's travel); in addition, the multiple local curves may be obtained based on real-time collected data or historical collected data, which is not limited in the present disclosure. Among them, the identification lines of map elements may include, for example, identification lines of map elements such as crosswalks, lane dividers, road boundaries, etc.

[0063] In some embodiments of the present disclosure, each local curve may correspond to one frame of local image or multiple frames of local images. The multiple local curves may also include noise that appears during the detection and image processing process, and the confidence levels of the multiple local curves may be the same or different.

[0064] In some embodiments of the present disclosure, the heat map generated based on the multiple local curves may include coordinate information of points on the multiple local curves and confidence level information of the multiple local curves. For example, all the information obtained by detection is used as input to generate the heat map, and the heat map is, for example, a single-channel image. Those skilled in the art can easily understand that multiple local curves corresponding to multiple frames of local images after extraction can be used as input to generate the heat map, or multiple frames of local images obtained by detection can be used as input to generate the heat map. In this embodiment, the generated global curve can be used as the map of the area of the map to be constructed, for example, for the autonomous driving of an intelligent vehicle.

[0065] In step S102, based on the heat map, a global curve (also referred to as "second curve") is obtained through an AI network, where the second curve is used to identify map elements in the map to be constructed.

[0066] In some embodiments of the present disclosure, a global curve is generated based on the heat map including all the information obtained by detection, and the global curve is predicted, for example, by methods such as machine learning and image detection. Those skilled in the art can easily understand that methods for constructing a global curve through other algorithms based on the heat map are also within the protection scope of the present disclosure. By using the heat map as input for predicting the global curve, the present disclosure can transform the global map construction problem into a set prediction problem.

[0067] In some embodiments of the present disclosure, as Figure 1B shown, the method 10 for constructing a map further includes step S103. In step S103, based on the global curve, a map is generated, and the map at least shows the identification lines of the map elements within the area of the map to be constructed. Based on the global curve generated in step S102, a high-precision map of the area of the map to be constructed is generated. The area of the map to be constructed includes, for example, a relatively long distance range (such as dozens to hundreds of meters), and the high-precision map at least shows the identification lines of the map elements within the area of the map to be constructed. It is easy for those skilled in the art to understand that the high-precision map may also include other information required for autonomous driving or image annotation. In this embodiment, in addition to including the global curve, the map may also include images of map elements within the area of the map to be constructed. For example, photos of buildings, sidewalks, etc.

[0068] In the map construction method, when the point coordinates and confidence information of the local map are used separately, incremental processing will cause error accumulation, resulting in its lack of robustness. For example: directly discarding the curves with low confidence, resulting in the inability to fully utilize the information; iteratively merging the curves, making it impossible to correct the incorrect merges in the early stage of iteration, and ultimately resulting in incorrect merge results. In addition, due to the design of the algorithm depending on specific use cases, it lacks generality. Specifically, the algorithm requires a lot of patches to handle different boundary cases; in addition, many operations in the algorithm require setting thresholds, such as sorting, clustering, matching, merging, etc.

[0069] The map construction method provided by the above embodiments of the present disclosure converts multiple local curves into a heat map, making full use of the input information. The heat map includes the coordinates and confidence information of the global curve to be extracted, and can simultaneously and fully utilize the coordinates and confidence information of multiple local curves, having higher robustness. The map construction method provided by the above embodiments of the present disclosure converts the global map construction problem into a set prediction problem, uses the heat map as the input, and makes predictions based on image detection, can handle different use cases in a unified manner, and avoids using too many thresholds, having better generality.

[0070] In some embodiments of the present disclosure, the heat map includes a rasterized curve, and the above step S101 further includes:

[0071] Based on the coordinate information of the points on multiple local curves and the confidence of each local curve, a rasterized curve is generated. Among them, the coordinate information of the points on the rasterized curve is determined based on the coordinate information of the corresponding points on multiple local curves, and the intensity information of the points on the rasterized curve is determined based on the confidence of the local curve including the point with the same coordinate information.

[0072] For example, the coordinates of points on the rasterized curve can be determined as the coordinates of points at that position in multiple local curves; the intensity information of points on the rasterized curve can be the superposition of the confidence levels of local curves of points with the same coordinate information as that point, etc.

[0073] By gridifying multiple local curves, gridified information containing intensity values is obtained. This gridified information can be directly used as a heatmap or can generate a heatmap after further processing, and this heatmap contains all the input information.

[0074] In some embodiments of the present disclosure, the above step S101 further includes:

[0075] Based on the coordinates of points in the detection regions (also referred to as the first regions) corresponding to multiple local curves, a second region is generated. Among them, the coordinates of points on this second region are determined based on the coordinates of points at the corresponding positions in the first region, and the intensity information of points on this second region is determined based on the number of points in the first region corresponding to that point; based on the rasterized curve and the second region, the average intensity of the rasterized curve is obtained.

[0076] Among them, for the multiple local curves obtained by detection, there may be a large amount of overlap in their respective detection regions. To more accurately utilize the confidence information, the detection regions are coordinate-transformed, and the intensity values of the rasterized curves obtained in the above embodiments are processed according to the overlapping coverage times of the detection regions.

[0077] In some embodiments of the present disclosure, before based on the coordinates of points on multiple local curves and the confidence level of each local curve, the method further includes:

[0078] Using a coordinate transformation matrix, the point coordinates of multiple local curves and the first regions are transformed to the same coordinate system.

[0079] Among them, the multiple frames of local images obtained by detection usually have independent coordinate systems. Through coordinate transformation, the coordinates of multiple local curves extracted from multiple frames of local images are transformed to the same coordinate system, for example, transformed to the coordinate system of the 0th frame of local image, which is convenient for subsequent heatmap generation processing. Similarly, the detection regions corresponding to multiple frames of local images also have independent coordinate systems, and through coordinate transformation, the coordinates of the detection regions corresponding to multiple frames of local images are also transformed to the same coordinate system.

[0080] In some embodiments of the present disclosure, as Figure 2 shown, step S101 of method 10 for constructing a map may further include steps S201 to S206. Among them:

[0081] In step S201 (geometric transformation), using a coordinate transformation matrix, the coordinates of multiple local curves and the points on the bounding boxes of the multiple local curves (for example, the boundaries of the detection regions of the local curves) are transformed to the same coordinate system. For example, as Figure 2 shown, LV are the coordinates of the points on the multiple local curves, B are the coordinates of the points on the bounding boxes of the multiple local curves, and the coordinate transformation matrices are R and T, where R and T respectively represent, for example, rotational geometric transformation and translational geometric transformation. The coordinate transformation matrices R and T can be used to transform LV and B respectively to the same coordinate system (for example, the coordinate system of the 0th frame), obtaining and When LV and B are transformed to the coordinate system of the 0th frame, the coordinates of the points on the local curves and the bounding boxes of the local curves corresponding to each frame (the tth frame) can be transformed from the coordinate system of this frame to the coordinate system of, for example, the 0th frame through the coordinate transformation matrices R t→0 and T t→0 .

[0082] In step S202 (curve rasterization), based on the coordinates of the points on the multiple local curves after coordinate transformation and the confidence of each local curve, a rasterized curve is generated. The coordinates of the points on the rasterized curve are the corresponding points on the multiple local curves. For example, the points on the generated rasterized curve include the points on all local curves. The intensity of the points on the rasterized curve is determined based on the superposition of the confidences of the multiple local curves at this point. For example, as Figure 2 shown, LC is the confidence of the multiple local curves. By inputting and LC, the multiple local curves can be rasterized and superimposed. The intensity of rasterization depends on the confidence of the local curves, and finally a single-channel image MC ∈ R H*W can be obtained, where H and W are the height and width of the rasterized curve.

[0083] In some embodiments of the present disclosure, the method for rasterizing a curve includes: obtaining a grayscale image based on the point coordinates and confidence of a group of curves, such as a rasterized curve Optionally, different strategies are used to rasterize polylines and polygons. For polylines, for example, edge rasterization can be performed through the following three steps:

[0084] 1. Initialization: Initialize the image with 0

[0085] 2. Discretization: Calculate the pixel positions of all local curves in MC;

[0086] 3. Rendering: Render the edges of all local curves with the confidence as the intensity in MC. The pixel values passed by multiple polylines are equal to the sum of the values of these polylines at this pixel.

[0087] For a polygon, its interior region can be rasterized, for example, through the following three steps:

[0088] 1. Initialization: Initialize the image with 0

[0089] 2. Discretization: Calculate the pixel positions of all local curves in MC;

[0090] 3. Rendering: Fill the region enclosed by each local curve in MC with the confidence as the intensity. The value of a pixel covered by multiple polygons is equal to the sum of the values of these polygons at that pixel.

[0091] In step S203 (bounding box superposition), the first region can be obtained through the bounding boxes of multiple curves. For example, set the intensity information of the points within the bounding boxes corresponding to the multiple curves after coordinate transformation to a predetermined value (e.g., 1) to fill the bounding boxes, thereby generating the first region. Based on the filled bounding boxes, generate the second region, where the coordinates of the points on the second region are the coordinates of the points at the corresponding positions of the first region, and the intensity of the second region is determined based on the number of points in the first region corresponding to that point. For example, in the case where the intensity information of the points inside the bounding box is set to 1 as described above, the number can be the sum of the intensity information of the first region that coincides with that point. For example, the input is Fill and superimpose all the bounding boxes, and finally obtain a single-channel image MB ∈ R H*W , where H and W are the height and width of the rasterized bounding box (e.g., taking the same size as the rasterized curve). The pixel value of this image (e.g., the rasterized intensity) indicates how many frames of bounding boxes cover this position.

[0092] In some embodiments of the present disclosure, the method of rasterizing the bounding box includes: obtaining a grayscale image (e.g., the second region) based on the transformed bounding box variables of all frames Optionally, the steps of rasterizing the bounding box include:

[0093] 1. Initialization: Initialize the second region with 0

[0094] 2. Discretization: Calculate the transformed bounding box variables of all frames at the pixel positions in MB;

[0095] 3. Rendering: Render the transformed bounding box with an intensity of 1 (e.g., set the intensity or pixel value of the points in the bounding box to 1) to obtain the second region The value of a pixel covered by multiple bounding boxes is equal to the sum of the number of these first regions on that pixel.

[0096] In step S204 (moving average), based on the rasterized curve and the second region, the average intensity of the rasterized curve is obtained. Optionally, the average intensity includes the average of the superposed confidence levels based on the number of occurrences in the first region (e.g., the corresponding intensity or pixel value in the second region). When there are points in multiple frames of local images (corresponding to multiple local curves) at the same coordinate (e.g., points that appear in multiple local curves), the pixel value at this coordinate, i.e., the superposition of the confidence levels of the local curves corresponding to the points that appear at this coordinate, should be restricted by the number of detections. If multiple frames of detection boxes (e.g., the first region) appear at this coordinate, the superposition of the confidence levels should be averaged according to the number of times the detection boxes appear. Input MC and MB, and output the average heatmap. In some embodiments of the present disclosure, the averaging process is only performed within the region filled by the bounding box.

[0097] In some embodiments of the present disclosure, performing a moving average on the rasterized curve includes: inputting two grayscale images and outputting a grayscale image where:

[0098]

[0099] where h ∈ {0, 1, …, H - 1}, w ∈ {0, 1, …, W - 1}.

[0100] In step S206 (heatmap normalization), the average intensity is normalized to obtain a heatmap. Output the heatmap M ∈ R after the normalization process H*W .

[0101] In some embodiments of the present disclosure, the multiple first curves may include: a first type of curve (also referred to as a "polygonal region") that can form a closed region, and a second type of curve (also referred to as a "broken line") other than the first type of curve.

[0102] The above method for constructing a map may further include: internally filling the part of the rasterized curve corresponding to the first type of curve to generate a first category heatmap (also referred to as a "polygonal category heatmap"). For example, the filling value inside the polygonal region can be determined based on the confidence level of the corresponding local curve.

[0103] In some embodiments of the present disclosure, the method 10 for constructing a map may further include: generating a second-class heat map (also referred to as a "polyline-class heat map") based on a portion of the rasterized curve corresponding to the second type of curve. For example, for the polyline class, the boundaries thereof may be rendered based on the confidence of the corresponding local curves (e.g., setting the pixel values of the points on the polyline). If there are overlaps among the polylines in multiple local curves, the pixel value at the overlap is the sum of the pixel values of the corresponding points of the corresponding polylines. Optionally, only the heat map of the polyline class is normalized. The normalization process may include calculating the mean and standard deviation of all heat maps only on valid pixels (non-zero pixels), and normalizing each heat map on all pixels by the mean and standard deviation.

[0104] As Figure 3 shown on the left, the input for heat map generation comes from the local curves of all frames, including lane dividers, crosswalks, and road boundaries. As Figure 3 shown on the right, the generated heat maps respectively include: a lane divider heat map, a crosswalk heat map, and a road boundary heat map, where the interior of the crosswalk heat map (e.g., a polygon area) has been filled.

[0105] In some embodiments of the present disclosure, continuing to refer to Figure 2 , before normalizing the average intensity, the method 10 for constructing a map further includes step S205.

[0106] In step S205 (dilation), the polylines in the polyline-class heat map are dilated. For example, the polylines in the polyline-type heat map are thickened. A Gaussian kernel can be used to dilate the average heat map AM in the above embodiments, and the dilated heat map EM ∈ R H*W .

[0107] In some embodiments of the present disclosure, dilating the polylines in the polyline-class heat map includes: inputting an image and a Gaussian kernel outputting the dilated image where:

[0108] E = ε(I, K)

[0109]

[0110] where h ∈ {p,..., H - p - 1}, w ∈ {p,..., W - p - 1}.

[0111] In some embodiments of the present disclosure, step S102 of method 10 for constructing a map further includes: predicting a first type of curve based on a first category heat map through a first AI network; predicting a second type of curve based on a second category heat map through a second AI network; obtaining a second curve based on the predicted first type of curve and second type of curve. For example, the second curve can be obtained by combining the first type of curve and the second type of curve based on the coordinates of the first type of curve and the second type of curve.

[0112] In some embodiments of the present disclosure, as Figure 4 shown, step S102 of method 10 for constructing a map further includes steps S401 to S404.

[0113] In step S401, a polygon region in the global curve is predicted based on the polygon category heat map. As Figure 4 shown, the polygon category heat map is input, and the polygon region in the global curve is output through polygon detection.

[0114] In step S402, a broken line in the global curve is predicted based on the broken line category heat map. As Figure 4 shown, the broken line category heat map is input, and the predicted broken line in the global curve is output through broken line detection.

[0115] Optionally, in step S403, at least one broken line is selected from the predicted broken lines based on the broken line category heat map to be determined as the second type of curve in the second curve. That is, post-processing is performed on the predicted broken lines to obtain a more accurate result.

[0116] In step S404, a global curve is generated based on the polygon region and the selected broken lines. For example, the polygon region and the broken lines can be combined based on the coordinates of the polygon region and the broken lines to obtain the global curve.

[0117] In some embodiments of the present disclosure, as Figure 5 shown, step S401 of method 10 for constructing a map further includes: steps S501 to S505.

[0118] In step S501 (binarization), the polygon category heat map is converted into a binary image using a threshold. For example, 0.4 is used as the threshold to binarize the heat map.

[0119] In step S502 (connected component extraction), connected regions are determined from the binary image. For example, a connected component extraction algorithm is used to extract connected regions from the binarized heat map.

[0120] In step S503 (boundary extraction), the boundaries of each connected region are extracted and determined as polygonal regions. The contour of each extracted connected region is the boundary coordinates of at least one polygonal region.

[0121] In step S504 (confidence calculation), the average confidence within each connected region is calculated. The average value of the pixels inside the connected domain is used as the confidence of the polygonal region in the global curve.

[0122] In step S505 (confidence normalization), the average confidence of each connected region is normalized. Optionally, the maximum value of the confidence is set to 1, and other confidences can be adjusted according to their ratio to the maximum value. For example, in some embodiments of the present disclosure, normalizing the confidence of each connected region includes: normalizing the average confidence of each connected region based on the maximum value in the average confidence of the connected region.

[0123] In some embodiments of the present disclosure, as Figure 6 shown, step S402 of method 10 for constructing a map further includes: steps S601 to S602.

[0124] In step S601, polyline features are extracted from the polyline category heatmap. For example, ResNet50+FPN is used to extract features from the polyline category heatmap.

[0125] In step S602, the coordinates and confidence of the polyline are predicted based on the extracted features using the MapTR algorithm. In some embodiments, the MapTR decoder is used to optimize the preset query Q for predicting the polyline. For example, the MapTR decoder may include multiple decoder layers that iteratively update the query Q. Here, the initial value of the query Q is part of the detection parameters, obtained through training, and represents the general attributes of the global curve. Inside the decoder, some operations (attention mechanism, linear operation, Softmax, etc.) are performed between the initial query Q and the features obtained by feature extraction to continuously optimize the query Q, and finally the optimized query Q is output. The regression head in the MapTR algorithm is used to predict the coordinates of the global curve from the query Q. The single-class classification head in the MapTR algorithm is used to predict the confidence of the global curve from the query Q.

[0126] In some embodiments of the present disclosure, the training of the MapTR model includes obtaining a training dataset, augmenting the training phase data, and optimizing the training loss.

[0127] When obtaining the training dataset, first use a MapTR model to perform inference on the NuScenes training dataset to obtain local curves, and then use a heatmap generator to convert the local curves into heatmaps, which are the data required for training. Operate on each category separately, and finally there are a total of 1400 training samples (for example, 1400 = 2 * 700, where 2 is the number of categories: lane dividers, road boundaries; 700 is the number of scenes in the NuScenes training set).

[0128] When augmenting the data in the training phase, data augmentation techniques can be used to alleviate the problem of insufficient training data volume. For example, the actual data augmentations used include: random translation, random rotation, and random flipping.

[0129] When optimizing the training loss, follow the MapTR model training loss function, which consists of three parts, including the classification loss point-to-point loss and edge direction loss Combining these loss terms, the overall objective function can be expressed as:

[0130]

[0131] where λ 1 、λ 2 and λ 3 are hyperparameters used to balance these terms.

[0132] During the process of training based on the training data, continuously adjust the parameters of the MapTR model according to the change of the objective function to minimize the objective function, and finally obtain the trained MapTR model.

[0133] In some embodiments of the present disclosure, as Figure 6 shown, step S403 of method 10 for constructing a map may further include: selecting at least one second-type curve from the predicted second-type curves whose similarity to the second-type heatmap is within a set range, and determining it as the second-type curve in the second curve. For example, it can be implemented through steps S1 - S6 described below.

[0134] In some embodiments of the present disclosure, as Figure 6As shown, step S403 of method 10 for constructing a map may further include: for the predicted second type of curve, determining the second type of curve in the second type of curve whose similarity to the second type of heat map is greater than or equal to a threshold; updating the second type of heat map, wherein the intensity values of the points corresponding to the determined second type of curve in the second type of heat map and the points within a set range of this point are set to a preset value; continuing to execute the step of, for the predicted second type of curve, determining the second type of curve in the second type of curve whose similarity to the second type of heat map is greater than or equal to a threshold, until there is no second type of curve in the predicted second type of curve whose similarity to the second type of heat map is within the set range, where the determined second type of curves are all the second type of curves in the second curve.

[0135] In some embodiments of the present disclosure, step S403 of method 10 for constructing a map further includes: steps S1 to S2.

[0136] In step S1, for the predicted second type of curve, determining the second type of curve in the second type of curve whose similarity to the second type of heat map is greater than or equal to a threshold. For example, for the broken lines in the predicted global curve, searching for the first broken line in the broken lines with the strongest consistency (e.g., similarity greater than or equal to a threshold) with the broken line category heat map.

[0137] In step S2, updating the second type of heat map, wherein the intensity values of the points corresponding to the determined second type of curve in the second type of heat map and the points within a set range of this point are set to a preset value. For example, updating the broken line category heat map such that the intensity values of the points corresponding to the first broken line in the broken line category heat map and the points within a set range of this point are set to 0. In some embodiments, S1 and S2 can be repeatedly executed until there is no second type of curve in the predicted second type of curve whose similarity to the second type of heat map is within the set range. Optionally, the confidence levels of all the selected second type of curves can be normalized to determine the confidence level of the second type of curve.

[0138] In some embodiments, in addition to steps S1 and S2, step S403 of method 10 for constructing a map may further include S3 - S4:

[0139] In step S3, adding the coordinates and confidence level of the first broken line to the broken line list.

[0140] In step S4, deleting the first broken line from the broken lines in the predicted global curve.

[0141] In step S5, returning to step S1 and continuing to execute S1 - S5.

[0142] In step S6, when the first polyline cannot be searched in step S1, the confidence levels of all polylines in the polyline list are normalized, the polylines in the list are determined as the polylines in the global curve, and the normalized confidence levels are determined as the confidence levels of the polylines in the global curve.

[0143] In some embodiments of the present disclosure, the confidence level of the first polyline is obtained by calculating the similarity between the first polyline and the polyline category heatmap. For example, the similarity is the average value of the polyline type heatmap within the region surrounded by the bounding box.

[0144] In the case where the polyline category heatmap is M and the curve coordinates of the polyline category heatmap are GV'. In some embodiments of the present disclosure, deleting the wrong curves in the polyline category heatmap includes:

[0145] The first step is to initialize G as an empty list;

[0146] The second step is to find the curve with the strongest consistency with M from GV'; If not found, jump to the seventh step;

[0147] The third step is to update M, and set the pixels near in M to 0. The range of the pixels set to 0 here can be determined based on the same algorithm as the "dilation" algorithm in step S205 above, that is, based on the range covered by the dilated curve, set the pixels therein to 0;

[0148] The fourth step is to add to G, where represents the confidence level, which is determined by the similarity between and M;

[0149] The fifth step is to update GV', and delete from GV';

[0150] The sixth step is to jump to the second step to find more retained curves;

[0151] The seventh step is to normalize the confidence levels of all curves in G, determine the polylines in the list as the polylines in the global curve, and determine the normalized confidence levels as the confidence levels of the polylines in the global curve.

[0152] As Figure 7A shown, in some embodiments of the present disclosure, a device 100 for constructing a map is provided, including a heatmap generation unit 110 and a global curve generation unit 120. The device 100 can execute the method 10 for constructing a map described above.

[0153] The heat map generation unit 110 is configured to generate a heat map based on multiple first curves, where the first curves are used to identify map elements in at least one local area of the map to be constructed.

[0154] The global curve generation unit 120 is configured to obtain a second curve through an AI network based on the heat map, where the second curve is used to identify map elements in the map to be constructed.

[0155] Generally, the input data of the device 100 for constructing a map includes:

[0156] The coordinates of the local curves of each frame of a scene and confidence where N is the number of curves, P is the number of points on each curve, and 2 represents the two-dimensional coordinates x and y;

[0157] The two-dimensional rigid body coordinate transformation R and T from each frame to the 0th frame;

[0158] The coordinate bounding box of the local curves of each frame: B = [xmin, xmax, ymin, ymax].

[0159] The output data of the device 100 for constructing a map includes:

[0160] The coordinates of the global curves of a scene and confidence

[0161] The above embodiments use the extracted local curve data (including local curve coordinates, confidence, etc.) as the input data of the device 100 for constructing a map. It is easy for those skilled in the art to understand that multiple frames of local images obtained by detection can also be directly used as the input of the device 100, and heat map data is generated after processing. These embodiments are also within the protection scope of the present disclosure.

[0162] In some embodiments of the present disclosure, the local curve input data of the device 100 for constructing a map includes:

[0163] Local curves: which are the predictions of the local map construction model for local map curves for all frames, where T is the frame number, and i ∈ {0, 1, 2} represents map element classification (0 represents lane dividers, 1 represents crosswalks, 2 represents road boundaries), where, represents the local curve of the i-th class in the t-th frame; are the coordinates of the points on the local curve, where is the number of the local curve of the i-th class in the t-th frame, P is the number of points on each local curve, and 2 represents two-dimensional point coordinates.

[0164] For example, local map curves are obtained using a local map generation model (MapTR) in the NuScenes val set.

[0165] In the case of selecting the 0th frame as the common coordinate system, the input data for coordinate transformation includes:

[0166] R t→0 : A 2D rotation matrix from coordinate system t to coordinate system 0.

[0167] T t→0 : A 2D translation vector from coordinate system t to coordinate system 0.

[0168] The input data of the bounding box (Bbox) includes:

[0169] B t = [xmin, xmax, ymin, ymax] = [-15, 15, -30, 30], which is the bounding box of the local curve, in meters.

[0170] The output data of the device 100 for constructing the map includes:

[0171] The global curve G = {(GV i , GC i )|i = 0, 1, 2}, where are the coordinates of the points on the predicted global curve, where N i is the number of curves of the i-th class, and P i,n is the number of points on the n-th curve of the i-th class; is the confidence score of the predicted global map element.

[0172] In some embodiments of the present disclosure, as Figure 7B shown, the device 100 for constructing the map further includes a map generation unit 130. The map generation unit 130 is configured to generate a high-precision map of the predetermined area based on the global curve, and the map at least shows the map element identification lines in the predetermined area. Those skilled in the art can easily understand that the high-precision map may also include other information required for autonomous driving or image annotation.

[0173] In some embodiments of the present disclosure, the local curve in the above embodiments corresponds to at least some of the map element identification lines in at least one local image in the predetermined area.

[0174] In some embodiments of the present disclosure, the heat map generation unit 110 of the device 100 for constructing a map is further configured to: generate a rasterized curve based on the coordinate information of the points on multiple first curves and the confidence level of each first curve, wherein the coordinate information of the points on the rasterized curve is determined based on the coordinate information of the points at the corresponding positions on the multiple first curves, and the intensity information of the points on the rasterized curve is determined based on the confidence level of the first curve including the points with the same coordinate information as the point.

[0175] In some embodiments of the present disclosure, the heat map generation unit 110 of the device 100 for constructing a map is further configured to: generate a second region based on the coordinate information of the points in the first region corresponding to the multiple first curves, wherein the coordinate information of the points on the second region is determined based on the coordinate information of the points at the corresponding positions in the first region, and the intensity information of the points on the second region is determined based on the number of points in the first region corresponding to the point; obtain the average intensity of the rasterized curve based on the rasterized curve and the second region.

[0176] In some embodiments of the present disclosure, the heat map generation unit 110 of the device 100 for constructing a map is further configured to: before based on the coordinates of the points on the multiple first curves and the confidence level of each first curve, use a coordinate transformation matrix to transform the coordinates of the points on the multiple first curves and the first region to the same coordinate system.

[0177] In some embodiments of the present disclosure, the heat map generation unit 110 of the device 100 for constructing a map is further configured to: before generating a rasterized curve based on the coordinates of the points on the multiple first curves and the confidence level of each first curve, set the intensity information of the points within the bounding box corresponding to the multiple first curves to a predetermined value.

[0178] In some embodiments of the present disclosure, the multiple first curves include: a first type of curve that can form a closed region, and a second type of curve other than the first type of curve. For example, in some embodiments of the present disclosure, as Figure 8 shown, the multiple local curves include a polygon region and a polyline. The heat map generation unit 110 of the device 100 for constructing a map is further configured to: perform internal filling on the part of the rasterized curve corresponding to the first type of curve to generate a first category heat map; and generate a second category heat map based on the part of the rasterized curve corresponding to the second type of curve.

[0179] In some embodiments of the present disclosure, the heat map generation unit 110 of the device 100 for constructing a map is further configured to: before generating a second category heat map based on the part of the rasterized curve corresponding to the second type of curve, perform normalization processing on the average intensity of each rasterized curve.

[0180] In some embodiments of the present disclosure, the heat map generation unit 110 of the apparatus 100 for constructing a map is further configured to: before normalizing the average intensity of each rasterized curve, perform dilation processing on the portion of the rasterized curve corresponding to the second type of curve.

[0181] In some embodiments of the present disclosure, the input data of the heat map generation unit 110 includes:

[0182] The coordinates LV and confidence LC of the local curve;

[0183] The coordinate transformation matrix R and vector T;

[0184] The bounding box coordinates B.

[0185] The output of the heat map generation unit 110 includes:

[0186] The heat map where H and W are the width and height of the heat map.

[0187] In some embodiments of the present disclosure, as Figure 9 shown, the global curve generation unit 120 of the apparatus 100 for constructing a map further includes: a polygon detection subunit 121, a polyline detection subunit 122, a post-processing subunit 123, and a global curve generation subunit 124.

[0188] The polygon detection subunit 121 is configured to predict the first type of curve based on the first category heat map through the first AI network.

[0189] The polyline detection subunit 122 is configured to predict the second type of curve based on the second category heat map through the second AI network.

[0190] The post-processing subunit 123 is configured to select at least one second type of curve from the predicted second type of curves based on the second category heat map and determine it as the second type of curve in the second curve.

[0191] The global curve generation subunit 124 is configured to generate a global curve based on the output result of the post-processing unit. For example, based on the predicted first type of curve and the second type of curve, obtain the second curve.

[0192] In some embodiments of the present disclosure, the polygon detection subunit 121 is further configured to:

[0193] Convert the first category heat map into a binary map using a threshold; determine the connected regions from the binary map; extract the boundaries of each connected region and determine them as the first type of curve; calculate the average confidence within each connected region; normalize the average confidence of each connected region to obtain the confidence of the first type of curve corresponding to the connected region.

[0194] The polygon detection subunit 121 is configured to implement the function of predicting the polygon regions in the global curve from the heatmap of the polygon category.

[0195] The input data of the polygon detection subunit 121 includes: the heatmap M of the polygon category.

[0196] The output data of the polygon detection subunit 121 includes: the global curve coordinates GV and the confidence level GC of the polygon category.

[0197] In some embodiments of the present disclosure, the polygon detection subunit 121 is configured to normalize the confidence level of at least one connected region through the following steps:

[0198] Normalize the average confidence level of each connected region based on the maximum value in the average confidence levels of the connected regions.

[0199] In some embodiments of the present disclosure, the broken line detection subunit 122 is further configured to: extract the second category features from the second category heatmap; use an AI algorithm to predict the coordinate information and the confidence level of the second type of curve in the second curve based on the second category features. For example, extract the broken line features from the heatmap of the broken line category; and use the MapTR algorithm to predict the coordinates and the confidence level of the broken line.

[0200] The broken line detection subunit 122 is configured to implement the function of predicting the broken lines in the global curve from the heatmap of the broken line category.

[0201] The input data of the broken line detection subunit 122 includes: the heatmap M of the broken line category.

[0202] The output data of the broken line detection subunit 122 includes: the global curve coordinates of the broken line category and the confidence level where N′ is the number of global curves of the broken line category predicted by the broken line detection unit 122.

[0203] In some embodiments of the present disclosure, the architecture of the broken line detection unit 122 (broken line detector) includes:

[0204] Feature extractor: Extract features from the heatmap using ResNet50+FPN.

[0205] Decoder: Using the MapTR algorithm, it predicts the coordinates and confidence of the polyline based on the extracted features. In some embodiments, the MapTR decoder is used to optimize the preset query Q for predicting the polyline. For example, the MapTR decoder may include multiple decoder layers that iteratively update the query Q. Here, the initial value of the query Q is part of the detection parameters, obtained through training, and represents the general attributes of the global curve. Inside the decoder, some operations (attention mechanism, linear operation, Softmax, etc.) are performed between the initial query Q and the features obtained by feature extraction to continuously optimize the query Q, and finally the optimized query Q is output.

[0206] Regression head: Using the regression head in the MapTR algorithm to predict the coordinates of the global curve from the query Q.

[0207] Classification head: Using the single-class classification head in the MapTR algorithm to predict the confidence of the global curve from the query Q.

[0208] In some embodiments of the present disclosure, the training of the MapTR model includes obtaining a training dataset, augmenting the training phase data, and optimizing the training loss.

[0209] When obtaining the training dataset, first, a MapTR model is used to perform inference on the NuScenes training dataset to obtain local curves, and then a heatmap generator is used to convert the local curves into heatmaps, which are the data required for training. Operations are performed for each category separately, and finally, there are a total of 1400 training samples (1400 = 2 * 700, where 2 is the number of categories: lane dividers, road boundaries; 700 is the number of scenes in the NuScenes training set).

[0210] When augmenting the training phase data, data augmentation techniques can be used to alleviate the problem of insufficient training data volume. For example, the actual data augmentation used includes: random translation, random rotation, and random flipping.

[0211] When optimizing the training loss, the MapTR model training loss function is followed. This function consists of three parts, including the classification loss point-to-point loss and edge direction loss Combining these loss terms, the overall objective function can be expressed as:

[0212]

[0213] where λ 1 、λ 2 and λ 3 are hyperparameters used to balance these terms.

[0214] During the training process based on the training data, the parameters of the MapTR model are continuously adjusted based on the change of the objective function to minimize the objective function, and finally the trained MapTR model is obtained.

[0215] In some embodiments of the present disclosure, the post-processing subunit 123 is further configured to: for the predicted second type of curve, determine the second type of curve in the second type of curve whose similarity to the second type of heat map is greater than or equal to the threshold; update the second type of heat map, wherein the intensity values of the points corresponding to the determined second type of curve in the second type of heat map and the points within a set range of the point are set to a preset value; continue to execute the step of determining the second type of curve in the predicted second type of curve whose similarity to the second type of heat map is greater than or equal to the threshold until there is no second type of curve in the predicted second type of curve whose similarity to the second type of heat map is within the set range, where the determined second type of curves are all of the second type in the second curves.

[0216] The post-processing subunit 123 is further configured to:

[0217] The post-processing subunit 123 is further configured to: normalize the confidence levels of all the selected second type of curves to determine the confidence levels of the second type of curves.

[0218] In some embodiments, the post-processing subunit 123 may be configured to:

[0219] S1: For the broken lines in the predicted global curve, search for the first broken line in the broken lines that has the strongest consistency with the broken line category heat map.

[0220] S2: Update the broken line category heat map so that the intensity values near the first broken line are 0.

[0221] S3: Add the coordinates and confidence levels of the first broken line to the broken line list.

[0222] S4: Delete the first broken line from the broken lines in the predicted global curve.

[0223] S5: Return to step S1 and continue to execute S1 - S5.

[0224] S6: When the first broken line cannot be searched in step S1, normalize the confidence levels of all the broken lines in the broken line list.

[0225] In some embodiments of the present disclosure, the confidence level of the first broken line is obtained by calculating the similarity between the first broken line and the broken line category heat map.

[0226] The post-processing subunit 123 is configured to implement the function of selecting a more accurate broken line category global curve.

[0227] The input data of the post - processing subunit 123 includes:

[0228] The heat map M of the polyline category;

[0229] The coordinates GV′ of the global curve output by the polyline detector.

[0230] The output data of the post - processing subunit 123 includes: the final global curve G of the polyline category.

[0231] The map construction method or the device for constructing a map provided by one or more of the above - mentioned embodiments of the present disclosure converts multiple local curves into a heat map, making full use of the input information. The heat map includes the coordinates and confidence information of the global curve to be extracted, and can simultaneously and fully utilize the coordinates and confidence information of multiple local curves, having higher robustness. In addition, the map construction method or the device for constructing a map provided by one or more of the above - mentioned embodiments of the present disclosure converts the global map construction problem into a set prediction problem, uses the heat map as the input, and makes predictions based on image detection, can handle different use cases in a unified manner, and avoids using too many thresholds, having better generality.

[0232] An electronic device is further provided in an embodiment of the present disclosure. The electronic device includes a processor. Optionally, it may further include a transceiver and / or a memory coupled to the processor. The processor is configured to execute the steps of the method provided in any optional embodiment of the present disclosure.

[0233] Figure 10 The structural schematic diagram of an electronic device applicable to the embodiment of the present disclosure is shown in Figure 10 as shown, Figure 10 As shown, the electronic device 1100 includes: a processor 1101 and a memory 1103. Among them, the processor 1101 and the memory 1103 are connected, such as connected through a bus 1102. Optionally, the electronic device 1100 may further include a transceiver 1104. The transceiver 1104 can be used for data interaction between the electronic device and other electronic devices, such as data sending and / or data receiving, etc. It should be noted that in practical applications, the transceiver 1104 is not limited to one, and the structure of the electronic device 1100 does not constitute a limitation to the embodiment of the present disclosure. Optionally, the electronic device may be a first network node, a second network node, or a third network node.

[0234] The processor 1101 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules, and circuits described in connection with the present disclosure. The processor 1101 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0235] The bus 1102 may include a path for transmitting information between the above components. The bus 1102 may be a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 1102 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0236] The memory 1103 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory), or other type of dynamic storage device that can store information and instructions. It may also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, which is not limited herein.

[0237] The memory 1103 is used to store a computer program for implementing the embodiments of the present disclosure and is controlled by the processor 1101 for execution. The processor 1101 is used to execute the computer program stored in the memory 1103 to implement the steps shown in the foregoing method embodiments.

[0238] Figure 11 An example of a computer-readable medium 1200 in the form of a CD or DVD is shown. A program is stored on the computer-readable medium.

[0239] In general, the various embodiments of the present disclosure may be implemented in hardware or special-purpose circuits, software, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software, which may be executed by a controller, a microprocessor, or other computing devices. Although the various aspects of the embodiments of the present disclosure are shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented as, by way of non-limiting example, hardware, software, firmware, special-purpose circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0240] The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which are executed in a device on a target real or virtual processor to perform the method 10 as described above with reference to FIGS. 1 to Figure 6 . In general, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functions of program modules may be combined or divided as needed. The machine-executable instructions for program modules may be executed locally or within a distributed device. In a distributed device, program modules may be located in local and remote storage media.

[0241] The computer program code for implementing the method of the present disclosure may be written in one or more programming languages. These computer program codes may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program code is executed by the computer or other programmable data processing devices, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code may be executed entirely on the computer, partially on the computer, as a stand-alone software package, partially on the computer and partially on a remote computer, or entirely on a remote computer or server.

[0242] In the context of the present disclosure, the computer program code or related data can be carried by any suitable carrier so that the device, apparatus or processor can perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals can include electrical, optical, radio, acoustic or other forms of propagated signals, such as carrier waves, infrared signals, etc.

[0243] A computer-readable medium can be any tangible medium that contains or stores a program for or related to an instruction execution system, apparatus or device. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, apparatus or devices, or any suitable combination thereof. More detailed examples of computer-readable storage media include electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), optical storage devices, magnetic storage devices, or any suitable combination thereof. As used herein, the term "non-transitory" or "non-transient" is a limitation on the medium itself (i.e., tangible, rather than a signal), rather than a limitation on the persistence of data storage (e.g., RAM versus ROM).

[0244] Furthermore, although the operations of the methods of the present disclosure are described in a particular order in the drawings, this does not require or imply that the operations must be performed in that particular order, or that all of the illustrated operations must be performed to achieve the desired result. On the contrary, the steps depicted in the flowcharts can be changed in the order of execution. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution. It should also be noted that the features and functions of two or more devices according to the present disclosure can be embodied in one device. Conversely, the features and functions of one device described above can be further divided and embodied by multiple devices.

[0245] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the description and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and do not necessarily have to be used to describe a particular order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order other than that shown or described in words.

[0246] It should be understood that although the flowcharts of the embodiments of the present disclosure indicate various operation steps by arrows, the execution order of these steps is not limited to the order indicated by the arrows. Unless there is a clear description in this article, in some implementation scenarios of the embodiments of the present disclosure, the implementation steps in each flowchart can be executed in other orders according to requirements. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage among these sub-steps or stages can also be executed at different times respectively. In the scenario where the execution times are different, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and the embodiments of the present disclosure do not limit this.

[0247] The above text and drawings are provided only as examples to assist the reader in understanding the present disclosure. They are not intended and should not be construed as limiting the scope of the present disclosure in any way. Although certain embodiments and examples have been provided, it will be apparent to those skilled in the art based on the content disclosed herein that, without departing from the scope of the present disclosure, the shown embodiments and examples can be changed, and other similar implementation means based on the technical idea of the present disclosure can be adopted, which also fall within the protection scope of the embodiments of the present disclosure.

Claims

1. A method for constructing a map, wherein, the method includes: generating a heat map based on multiple first curves, wherein the first curves are used to identify map elements in at least one local area of the map to be constructed; obtaining a second curve through an AI network based on the heat map, wherein the second curve is used to identify map elements in the map to be constructed.

2. The method according to claim 1, wherein, the heat map includes rasterized curves, and generating the heat map based on multiple first curves includes: generating the rasterized curves based on the coordinate information of points on the multiple first curves and the confidence of each first curve, wherein the coordinate information of points on the rasterized curves is determined based on the coordinate information of corresponding points on the multiple first curves, and the intensity information of points on the rasterized curves is determined based on the confidence of the first curve including points with the same coordinate information as the point.

3. The method according to claim 2, wherein, generating the heat map based on multiple first curves further includes: generating a second area based on the coordinate information of points in a first area corresponding to the multiple first curves, wherein the coordinate information of points on the second area is determined based on the coordinate information of corresponding points in the first area, and the intensity information of points on the second area is determined based on the number of points in the corresponding first area of the point; obtaining the average intensity of the rasterized curves based on the rasterized curves and the second area.

4. The method according to claim 2, wherein, before generating the rasterized curves based on the coordinate information of points on the multiple first curves and the confidence of each first curve, further includes: setting the intensity information of points within the bounding box corresponding to the multiple first curves to a predetermined value.

5. The method according to claim 4, wherein, before based on the coordinate information of points on the multiple first curves and the confidence of each first curve, further includes: using a coordinate transformation matrix to transform the point coordinates of the multiple first curves and the bounding box to the same coordinate system.

6. The method according to any one of claims 2-5, wherein, the multiple first curves include: a first type of curve that can form a closed area, and a second type of curve other than the first type of curve, and generating the heat map based on multiple first curves further includes: performing internal filling on the part of the rasterized curves corresponding to the first type of curve to generate a first category heat map.

7. The method according to claims 2-4, wherein, the multiple first curves include: a first type of curve that can form a closed area, and a second type of curve other than the first type of curve, and generating the heat map based on multiple first curves further includes: generating a second category heat map based on the part of the rasterized curves corresponding to the second type of curve.

8. The method according to claim 7, wherein, before generating a second category heat map based on the part of the rasterized curves corresponding to the second type of curve, further includes: performing normalization processing on the average intensity of each rasterized curve.

9. The method according to claim 8, wherein, before normalizing the average intensity of each of the rasterized curves, further comprising: performing a dilation process on a portion of the rasterized curve corresponding to the second type of curve.

10. The method according to claim 6, wherein, obtaining the second curve based on the heat map through an AI network includes: predicting a first type of curve based on the first type of heat map through a first AI network; predicting a second type of curve based on the second type of heat map through a second AI network; obtaining the second curve based on the predicted first type of curve and second type of curve.

11. The method according to claim 10, wherein, obtaining the second curve based on the heat map through an AI network includes: selecting at least one second type of curve from the predicted second type of curves based on the second type of heat map, and determining it as the second type of curve in the second curve.

12. The method according to claim 10, wherein, predicting the first type of curve in the second curve based on the heat map through a first AI sub-network includes: converting the first type of heat map into a binary map using a threshold; determining connected regions from the binary map; extracting the boundaries of each of the connected regions and determining them as the first type of curve; calculating the average confidence within each of the connected regions; normalizing the average confidence of each of the connected regions to obtain the confidence of the first type of curve corresponding to the connected region.

13. The method according to claim 12, wherein, normalizing the average confidence of each of the connected regions includes: normalizing the average confidence of each of the connected regions based on the maximum value among the average confidences of the connected regions.

14. The method according to claim 10, wherein, predicting the second type of curve in the second curve based on the second type of heat map through a second AI sub-network includes: extracting second type of features from the second type of heat map; using an AI algorithm to predict the coordinate information and confidence of the second type of curve in the second curve based on the second type of features.

15. The method according to claim 11, wherein, selecting at least one second type of curve from the predicted second type of curves based on the second type of heat map as the second type of curve in the second curve includes: selecting at least one second type of curve from the predicted second type of curves, the similarity of which with the second type of heat map is within a set range, and determining it as the second type of curve in the second curve.

16. The method according to claim 15, wherein, selecting at least one second type of curve from the predicted second type of curves, the similarity of which with the second type of heat map is within a set range, as the second type of curve in the second curve includes: for the predicted second type of curves, determining the second type of curves in the second type of curves, the similarity of which with the second type of heat map is greater than or equal to a threshold; Update the second type of heat map, where the intensity values of the points corresponding to the determined second type of curve and the points within a set range of this point in the second type of heat map are set to a preset value; Continue to execute the step of determining, for the predicted second type of curve, the second type of curve in the second type of curve whose similarity to the second type of heat map is greater than or equal to a threshold, until there is no second type of curve in the predicted second type of curve whose similarity to the second type of heat map is within a set range, where the determined second type of curves are all the second type of curves in the second curve.

17. The method according to claim 16, wherein, the method further comprises: Normalize the confidence levels of all the selected second type of curves to determine the confidence level of the second type of curve.

18. An electronic device, comprising: at least one processor; and at least one memory, the at least one memory stores instructions that, when executed by the at least one processor, implement the method according to any one of claims 1 - 17.

19. A computer-readable medium having instructions stored thereon that, when executed by at least one processing unit, cause the at least one processing unit to be configured to execute the method according to any one of claims 1 - 17.

Images