Training of map update models, lane-level map update methods, and navigation methods.

Abstract

This disclosure provides a training method and apparatus for a map update model, as well as a method for updating and navigating lane-level maps, relating to the field of artificial intelligence technology, specifically to map navigation, autonomous driving, and intelligent transportation. The specific implementation scheme is as follows: historical lane elements corresponding to road top-view samples are input into a priori encoding network to generate priori encoded features; the priori encoded features and road top-view samples are input into an update network to generate lane update information; a loss function is constructed based on the lane update information and the target lane elements corresponding to the road top-view samples; and the parameters of the map update model are adjusted based on the loss function. This approach improves the accuracy and reliability of the trained map update model.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to technologies such as map navigation, autonomous driving, and intelligent transportation, and particularly to a method for training a map update model, updating lane-level maps, and a navigation method. Background Technology

[0002] High-freshness lane-level maps are an indispensable infrastructure and key enabling technology for ensuring the safety of autonomous driving systems and the user experience.

[0003] Map data updates mainly include 1) identifying changes in location; and 2) modifying the changed instances in the current map database. Summary of the Invention

[0004] This disclosure provides a method for training a map update model, updating a lane-level map, a navigation method, an apparatus, a device, and a storage medium.

[0005] In a first aspect, embodiments of this disclosure provide a training method for a map update model, the method comprising: inputting historical lane elements corresponding to road top view samples into a prior coding network to generate prior coding features; inputting the prior coding features and road top view samples into an update network to generate lane update information; constructing a loss function based on the lane update information and target lane elements corresponding to road top view samples; and adjusting the parameters of the map update model based on the loss function.

[0006] Secondly, embodiments of this disclosure provide a method for updating a lane-level map, the method comprising: obtaining a road top view and historical lane elements corresponding to the road top view; inputting the road top view and historical lane elements into a map update model to generate lane update information, wherein the map update model is a map update model obtained by the method described in any implementation of the first aspect above; and updating the lane-level map based on the lane update information.

[0007] Thirdly, embodiments of this disclosure provide a navigation method, which includes: performing route navigation based on a target lane-level map, wherein the target lane-level map is an updated lane-level map obtained by the method described in any implementation of the second aspect above.

[0008] Fourthly, embodiments of this disclosure provide a training apparatus for a map update model. The apparatus includes: a first generation module, a second generation module, a function construction module, and a parameter adjustment module. The first generation module is configured to input historical lane elements corresponding to road top-view samples into a priori encoding network to generate priori encoded features. The second generation module is configured to input the priori encoded features and road top-view samples into an update network to generate lane update information. The function construction module is configured to construct a loss function based on the lane update information and target lane elements corresponding to the road top-view samples. The parameter adjustment module is configured to adjust the parameters of the map update model based on the loss function.

[0009] Fifthly, embodiments of this disclosure provide a lane-level map updating apparatus, the apparatus comprising: a data acquisition module, a lane updating module, and a map updating module, wherein the data acquisition module is configured to acquire a road top view and historical lane elements corresponding to the road top view; the lane updating module is configured to input the road top view and historical lane elements into a map updating model to generate lane updating information, wherein the map updating model is a map updating model obtained by the apparatus described in any implementation of the fourth aspect above; and the map updating module is configured to update the lane-level map based on the lane updating information.

[0010] In a sixth aspect, embodiments of this disclosure provide a navigation device, the device comprising: a map navigation module configured to: perform route navigation based on a target lane-level map, wherein the target lane-level map is an updated lane-level map obtained by the device as described in any implementation of the fifth aspect above.

[0011] In a seventh aspect, embodiments of this disclosure provide an electronic device including one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the methods described in any of the implementations of the first, second, or third aspects.

[0012] Eighthly, embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, which, when executed by a processor, performs the method as described in any implementation of the first or second aspect.

[0013] Ninthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the first, second, or third aspects.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] Figure 1 This is an exemplary system architecture diagram to which this disclosure can be applied;

[0016] Figure 2 This is a flowchart of an embodiment of a map update model training method according to the present disclosure;

[0017] Figure 3 This is a flowchart of yet another embodiment of the training method for the map update model according to the present disclosure;

[0018] Figure 4 This is a flowchart of one embodiment of the lane-level map updating method according to the present disclosure;

[0019] Figure 5 This is a schematic diagram of an application scenario of the lane-level map update method disclosed herein;

[0020] Figure 6 This is a schematic diagram of one embodiment of a training apparatus for a map update model according to the present disclosure;

[0021] Figure 7 This is a schematic diagram of one embodiment of a lane-level map updating device according to the present disclosure;

[0022] Figure 8 This is a schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present disclosure. Detailed Implementation

[0023] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] Figure 1 An exemplary system architecture 100 is shown, which is an embodiment of the training method for the map update model of this disclosure.

[0026] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0027] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc.

[0028] Terminal devices 101, 102, and 103 can be either hardware or software. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules, or as a single software program or software module. No specific limitations are made here.

[0029] Server 105 can be a server that provides various services, such as inputting historical lane elements corresponding to road top view samples into a priori coding network to generate priori coding features; inputting priori coding features and road top view samples into an update network to generate lane update information; constructing a loss function based on lane update information and target lane elements corresponding to road top view samples; and adjusting the parameters of the map update model based on the loss function.

[0030] It should be noted that server 105 can be either hardware or software. When server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When server 105 is software, it can be implemented as multiple software programs or software modules (e.g., used to provide training services for map update models), or as a single software program or software module. No specific limitations are made here.

[0031] It should be noted that the map update model training method provided in the embodiments of this disclosure can be executed by server 105, by terminal devices 101, 102, and 103, or by server 105 and terminal devices 101, 102, and 103 in cooperation with each other. Accordingly, the various parts (e.g., various units, sub-units, modules, and sub-modules) included in the map update model training device can all be set in server 105, all of them can be set in terminal devices 101, 102, and 103, or they can be set in server 105 and terminal devices 101, 102, and 103 respectively.

[0032] It should be understood that Figure 1The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0033] Figure 2 A flowchart 200 illustrates an embodiment of a training method for a map update model. The map update model may include a prior encoding network and an update network. The training method for this map update model may specifically include the following steps:

[0034] Step 201: Input the historical lane elements corresponding to the road top view sample into the prior coding network to generate prior coding features.

[0035] In this embodiment, the execution entity (e.g., Figure 1 The server 105 or terminal device 101, 102, 103 can obtain the training sample set locally or from a remote server that stores the training sample set.

[0036] The training sample set may include road top view samples and corresponding historical lane elements and target lane elements. Historical lane elements are lane elements determined before the road top view samples are generated, and target lane elements are lane elements determined after the road top view samples are generated.

[0037] Here, lane elements are used to indicate directional elements marked on the road, such as lane lines and lane markings. Lane elements can have different shapes and colors. The road top-view sample can be acquired by an image acquisition device installed on the vehicle.

[0038] Specifically, vehicles can use onboard sensing devices, such as onboard radar, onboard cameras, and onboard cameras, to collect road data around the vehicle and perform coarse-grained change detection on the road data. If a change is detected, images of the changed road sections are collected according to a preset collection strategy to generate overhead road map samples.

[0039] The acquisition strategy can be set according to the number of lanes. For example, if the number of lanes in a road segment that has changed is less than or equal to a preset value, a single-trip image acquisition is performed on that road segment; if the number of lanes in a road segment that has changed is greater than the preset value, the image acquisition is performed on that road segment multiple times.

[0040] Here, the preset value can be 2, 3, etc., and this application does not limit it.

[0041] The historical lane elements corresponding to the road top view sample can be obtained by mapping the area corresponding to the road top view sample to the historical lane-level map to obtain the historical mapping area, and determining the lane elements in the historical mapping area as historical lane elements.

[0042] The target lane element corresponding to the road top view sample can be obtained by mapping the area corresponding to the road top view sample to the latest lane-level map to obtain the latest mapped area, and determining the lane element in the latest mapped area as the target lane element.

[0043] Furthermore, the implementing entity can input the historical lane elements corresponding to the road top view sample into the prior coding network to generate prior coding features.

[0044] Among them, the prior coding network can use a multi-head self-attention mechanism to encode the geometric features, semantic features and spatial relationship features of lane elements.

[0045] Here, semantic features can include the category of lane elements (such as structured roads, unstructured roads), instance number, etc.; geometric features can include the geometric shape data and orientation of lane elements; spatial relationship features can include the topological spatial relationship, sequential spatial relationship, metric spatial relationship, etc. of lane elements.

[0046] Step 202: Input the prior encoded features and road top view samples into the update network to generate lane update information.

[0047] In this embodiment, after obtaining the prior coding features, the executing entity can input the prior coding features and road top view samples into the update network to generate lane update information.

[0048] The lane update information may include the updated lane element, or it may include both the updated lane element and the type of change of the updated lane element. This application does not limit this.

[0049] Step 203: Construct a loss function based on the lane update information and the target lane elements corresponding to the road top view samples.

[0050] In this embodiment, lane update information can be used as the predicted output of the map update model, and the target lane element corresponding to the road top view sample can be used as the expected output of the map update model. Lane update information can include the updated lane element. The execution subject can construct a loss function based on the updated lane element and the target lane element using mean squared error, cosine similarity, etc.

[0051] Step 204: Adjust the parameters of the map update model based on the loss function.

[0052] In this embodiment, after obtaining the loss function, the executing entity can adjust the parameters of the prior coding network and the update network included in the map update model according to the loss function to obtain the parameter-adjusted map update model.

[0053] The map update model training method provided in this disclosure involves inputting historical lane elements corresponding to road top view samples into a prior coding network to generate prior coding features; inputting the prior coding features and road top view samples into an update network to generate lane update information; constructing a loss function based on the lane update information and target lane elements corresponding to road top view samples; and adjusting the parameters of the map update model based on the loss function. Specifically, by using prior coding features containing geometric, semantic, and spatial relationship features of lane elements as a reference to generate lane update information, the accuracy of the generated lane update information is improved, thereby enhancing the accuracy of the trained map update model.

[0054] In some alternative approaches, both historical lane elements and target lane elements can include vectorized point sets and element style information.

[0055] In this implementation, the training sample set may include road top view samples and corresponding historical lane elements and target lane elements. Both historical lane elements and target lane elements may include vectorized point sets and element style information.

[0056] The element style information may include the element shape and element color. The vectorized point set is composed of vector graphic elements (e.g., points, lines, circles, etc.) obtained by converting the pixels that make up the lane element. The vector graphic elements may include coordinate information and direction information.

[0057] Specifically, the road top-view sample provides the latest observation results of the current road scene within the corresponding region R, and the lane elements (i.e., historical lane elements) of the historical lane-level map corresponding to the road top-view sample within region R are vectorized from the point set of elements. and element style information Composition. The lane elements (i.e., target lane elements) of the latest lane-level map corresponding to the road top-view sample within region R are composed of a set of vectorized points. and element style information constitute.

[0058] Where, N ori_his This indicates the number of historical lane elements corresponding to a single road top-view sample. N represents the style of a single historical lane element. ori This represents the number of target lane elements corresponding to a single road top view sample, where c is the total number of styles.

[0059] This implementation improves the accuracy of predicted lane elements by setting lane elements to include vectorized point sets and element style information, fully considering the coordinates, colors, shapes, and other information included in lane elements.

[0060] Figure 3 A flowchart 300 illustrates an embodiment of a training method for a map update model. This training method for the map update model may specifically include the following steps:

[0061] Step 301: Input the historical lane elements corresponding to the road top view sample into the prior coding network to generate prior coding features.

[0062] In this embodiment, the implementation details and technical effects of step 301 can be found in the description of step 201, and will not be repeated here.

[0063] Step 302: Input the prior encoded features and road top view samples into the update network to generate lane update information.

[0064] In this embodiment, the implementation details and technical effects of step 302 can be found in the description of step 202, and will not be repeated here.

[0065] Step 303: Construct the first sub-loss function based on the updated lane elements and the target lane elements.

[0066] In this embodiment, the lane update information may include the updated lane elements and the change type of the updated lane elements. The executing entity can construct a first sub-loss function based on the updated lane elements and the target lane elements using mean square error, cosine similarity, etc.

[0067] Step 304: Based on the change type of the updated lane elements and the change type of the target lane elements, construct the second sub-loss function.

[0068] In this embodiment, the executing entity can construct a second sub-loss function using mean squared error, cosine similarity, etc., based on the change type of the updated lane elements and the change type of the target lane elements.

[0069] The change type of the target lane element can be obtained by matching the historical lane elements and the target lane element.

[0070] Specifically, the type of change of the target lane element The labeling process can include two stages: in the first stage, historical lane elements and target lane elements are matched to obtain the preliminary change type of the target lane elements; in the second stage, the final change type of the target lane elements is determined through manual adjustment.

[0071] Where, N ori This indicates the number of lane elements in the image. Indicates the type of change for a single lane element.

[0072] Step 305: Determine the loss function based on the first sub-loss function and the second sub-loss function.

[0073] In this embodiment, the executing entity can directly determine the loss function based on the first sub-loss function and the second sub-loss function, or it can determine the loss function based on the first sub-loss function, the first weight corresponding to the first sub-loss function, the second sub-loss function, and the second weight corresponding to the second sub-loss function.

[0074] The first weight corresponding to the first sub-loss function can be positively correlated with the preset first confidence threshold of the updated lane element, that is, the larger the first confidence threshold, the larger the first weight; the second weight corresponding to the second sub-loss function can be positively correlated with the preset second confidence threshold of the change type of the updated lane element, that is, the larger the second confidence threshold, the larger the second weight.

[0075] Specifically, the loss function L can be expressed by the following formula:

[0076] L=αL l +βL c

[0077] Among them, L l Let L be the first sub-loss function, α be the first weight, and L be the first sub-loss function. c β is the second sub-loss function, and β is the second weight.

[0078] Step 306: Adjust the parameters of the map update model based on the loss function.

[0079] In this embodiment, the implementation details and technical effects of step 306 can be found in the description of step 204, and will not be repeated here.

[0080] The above embodiments of this disclosure construct a first sub-loss function based on the updated lane elements and the target lane elements; construct a second sub-loss function based on the change type of the updated lane elements and the change type of the target lane elements; determine the loss function according to the first and second sub-loss functions, and further consider the change type of lane elements, so that the trained map update model can generate lane elements and the change type of lane elements at the same time, thereby further improving the accuracy and reliability of the trained map update model.

[0081] In some optional ways, the types of changes to lane elements can include: element style change, element style no change, element addition, and element deletion.

[0082] In this implementation, the change type of the updated lane element or the change type of the target lane element can include element style change (e.g., road signs change from yellow to white, lane lines change from solid lines to dashed lines, etc.), no change in element style, element addition (e.g., adding new lane lines, adding new road signs, etc.), and element deletion (e.g., deleting existing lane lines, deleting existing road signs, etc.).

[0083] Specifically, if The type of change for a single lane element is indicated by 0 (no change in style), 1 (style change), 2 (addition of element), and 3 (deletion of element).

[0084] This implementation method enables prediction of multiple change types by setting the change type, which can include: element style change, element style no change, element addition, and element deletion.

[0085] In some alternative approaches, prior encoded features and road top view samples are input into an update network to generate lane update information, including: inputting prior encoded features and road top view samples into a first update sub-network to generate updated lane elements; and inputting the updated lane elements and prior encoded features into a second update sub-network to generate the change type of the updated lane elements.

[0086] In this implementation, the update network may include a first update sub-network and a second update sub-network. After obtaining the prior coding features, the executing entity can input the prior coding features and the road top view sample into the first update sub-network to generate the updated lane elements; and input the updated lane elements and the prior coding features into the second update sub-network to generate the change type of the updated lane elements.

[0087] Here, the first update sub-network may include an encoding network, a fusion network, and a decoding network. The executing entity may first input the road top view sample into the encoding network to generate image features, then input the image features and prior encoded features into the fusion network to generate fused features, and finally input the fused features into the decoding network to generate updated lane elements.

[0088] Among them, the fusion network can use a cross-attention mechanism to fuse image features and prior encoded features to obtain fused features.

[0089] Here, the second update sub-network can be an association network, which is used to associate the updated lane elements with the prior encoded features to generate an association matrix. The elements in the association matrix are used to indicate the change type of the updated lane elements.

[0090] This implementation method generates updated lane elements by inputting prior encoded features and road top view samples into the first update sub-network; and generates updated lane elements and prior encoded features into the second update sub-network by inputting updated lane elements and their variation types. In other words, it uses prior encoded features to generate updated lane elements and their variation types, thereby improving the accuracy and reliability of the generated updated lane elements and their variation types.

[0091] In some alternative approaches, the updated lane elements and prior encoded features are input into a second update sub-network to generate the change type of the updated lane elements, including: inputting the updated lane elements and prior encoded features into a first association network to generate an initial association matrix; and inputting the initial association matrix into a second association network to generate a target association matrix.

[0092] In this implementation, the second update sub-network may include a first association network and a second association network. The executing entity may first input the updated lane elements and prior encoded features into the first association network to generate an initial association matrix; then input the initial association matrix into the second association network to generate a target association matrix.

[0093] Here, the second association network can use a preset matching strategy to process the initial association matrix, so that each updated lane element matches at most one historical lane element, and each historical lane element matches at most one target lane element, thus obtaining a matched association matrix. Furthermore, the second association network can directly generate the target association matrix based on the matched association matrix.

[0094] In this context, one element of the target correlation matrix is ​​used to indicate the type of change of a lane element.

[0095] Here, the matrix elements included in the target association matrix can be used to indicate the change type of lane elements. The specific change type can include element association, element addition, and element deletion. Element association means that there is a corresponding historical lane element among all historical lane elements. Element association can further include two cases: element style change and element style no change.

[0096] Specifically, the type of change in lane elements indicated by a matrix element of the target correlation matrix can include element association, element addition, and element deletion. The matrix element M of the target correlation matrix M... ij This is used to indicate the relationship between the updated lane element i and the historical lane element j, i.e., whether there exists a historical lane element j corresponding to the updated lane element i. If M ijIf M = 1, then the change type of lane element i relative to the historical lane element j is element-related, meaning there exists a historical lane element j corresponding to the updated lane element i. If M ij If the value is 0, then the updated lane element i and the historical lane element j are unrelated; that is, there is no historical lane element j corresponding to the updated lane element i. In this case, the change type of the updated lane element i relative to the historical lane element j can be considered as an element addition, and the change type of the historical lane element j relative to the updated lane element i can be considered as an element deletion.

[0097] Furthermore, based on the target correlation matrix M, the types of changes in lane elements can be divided into four categories. The first category: no change in element style, M ij =1, and the style of lane element i after the update is consistent with the style of the historical lane element j; Second type: Element style change, M ij =1, and the style of lane element i after the update is inconsistent with the style of the historical lane element j; the third type: element addition, for the updated lane element i, M i =0, indicating that among all values ​​of j, there is no historical lane element corresponding to the i-th updated lane element; Fourth type, element deletion, for historical lane element j, M j =0 indicates that among all values ​​of i, there is no updated lane element corresponding to the j-th historical lane element.

[0098] This implementation method generates an initial association matrix by inputting the updated lane elements and prior encoded features into a first association network; then, it inputs the initial association matrix into a second association network to generate a target association matrix. In other words, the updated lane elements and prior encoded features are first associated with the same feature space, and then the two are matched to generate the target association matrix, which effectively improves the reliability of generating the target association matrix.

[0099] In some alternative approaches, the updated lane elements and prior encoded features are input into a first association network to generate an initial association matrix, including: inputting the updated lane elements and prior encoded features into a multilayer perceptron network to generate a first feature and a second feature, respectively; inputting the first feature and the second feature into an outer product network to generate a feature-level association matrix; and inputting the feature-level association matrix into a classification network to generate an initial association matrix.

[0100] In this implementation, the first association network may include a multilayer perceptron network, an outer product network, and a classification network. The executing entity may first input the updated lane elements and prior encoded features into the MLP (Multilayer Perceptron) network to generate a first feature and a second feature, respectively; input the first feature and the second feature into the outer product network to generate a feature-level association matrix; and input the feature-level association matrix into the classification network to generate an initial association matrix.

[0101] Among them, the MLP network can uniformly update the feature space of the new lane elements and the historical lane elements, and the classification network can construct the initial association matrix at the confidence level.

[0102] Here, the outer product network is used to perform outer product calculation on the first feature and the second feature to generate the feature-level correlation matrix, i.e., the feature-level correlation matrix.

[0103] Specifically, the initial correlation matrix This can be expressed by the following formula:

[0104]

[0105] Among them, P e 'Indicates that the lane element P will be updated. e The first feature obtained from the input MLP, E m ′ indicates that the prior encoded feature E m The second feature obtained by inputting into the MLP. This indicates that the first feature P e 'and second feature E m The feature-level correlation matrix is ​​obtained by inputting the outer product network and performing outer product calculations. Let A represent the feature-level correlation matrix. The input classification network generates an initial correlation matrix, where M and N represent the number of lane elements after the update and the number of historical lane elements, respectively. F represents the classification operation. This represents the outer product of vectors.

[0106] This implementation improves the reliability and accuracy of the determined initial correlation matrix by inputting the updated lane elements and prior encoded features into a multilayer perceptron network, respectively, to generate a first feature and a second feature; inputting the first feature and the second feature into an outer product network to generate a feature-level correlation matrix; and inputting the feature-level correlation matrix into a classification network to generate an initial correlation matrix.

[0107] In some alternative approaches, the initial association matrix is ​​input into the second association network to generate the target association matrix, including: inputting the initial association matrix into a matching network to generate a matched association matrix; and inputting the matched association matrix into a filtering network to generate the target association matrix.

[0108] In this implementation, the second association network may include a matching network and a filtering network. The executing entity may first input the initial association matrix into the matching network to generate a matched association matrix; then input the matched association matrix into the filtering network to generate the target association matrix.

[0109] The matching network can be used to perform matching processing on the initial association matrix using a preset matching strategy. The preset matching strategy is used to ensure that each updated lane element is matched with at most one historical lane element, and each historical lane element is matched with at most one target lane element.

[0110] Here, the preset matching strategy may include the Hungarian matching strategy, namely the Hungarian matching algorithm, which is an algorithm used to solve the maximum matching problem in a bipartite graph and is often used to handle problems such as task allocation and resource allocation.

[0111] The filtering network can be used to filter elements in the matched association matrix based on the confidence of the updated lane element and / or the matching degree between the updated lane element and the target lane element (e.g., removing elements in the matched association matrix whose confidence of the corresponding updated lane element is less than a preset confidence threshold and / or whose matching degree between the updated lane element and the target lane element is less than a preset matching degree threshold).

[0112] This implementation improves the reliability of the generated target association matrix by inputting the initial association matrix into the matching network to generate a matched association matrix, and then inputting the matched association matrix into the filtering network to generate the target association matrix.

[0113] In some alternative approaches, the first sub-loss function may include a regression loss function, a cosine similarity loss function, and a classification loss function.

[0114] In this implementation, the executing entity can determine the first sub-loss function based on the regression loss function, the cosine similarity loss function, the classification loss function, and the weights corresponding to each loss function.

[0115] Among them, the regression loss function is mainly used for regression tasks in supervised learning, and its purpose is to predict continuous value outputs; the cosine similarity loss function is often used to evaluate whether the directions of two vectors are similar; and the classification loss function is often used for classification problems in supervised learning.

[0116] For the first sub-loss function, the regression loss function is used to determine whether the coordinates of the updated lane element and the target lane element are close, the cosine similarity loss function is used to determine whether the directions of the vectors of the updated lane element and the target lane element are close, and the classification loss function is used to determine whether the categories of the updated lane element and the target lane element are close.

[0117] Among them, the third weight corresponding to the regression loss function can be positively correlated with the preset third confidence threshold of the coordinates of the updated lane element; the fourth weight corresponding to the cosine similarity loss function can be positively correlated with the preset fourth confidence threshold of the coordinates of the updated lane element; and the fifth weight corresponding to the classification loss function can be positively correlated with the preset fifth confidence threshold of the category of the updated lane element.

[0118] This implementation improves the accuracy of the determined first sub-loss function by setting the loss function to include regression loss function, cosine similarity loss function and classification loss function.

[0119] In some alternative approaches, the second sub-loss function is a classification loss function constructed based on the change type of the updated lane element and the change type of the target lane element.

[0120] In this implementation, the change type of the updated lane element can be represented by the target correlation matrix. The elements of the target correlation matrix can represent the change type of an updated lane element relative to a historical lane element. A classification loss function is constructed based on the target correlation matrix and the change type of the target lane element, that is, the change between the updated lane element and the historical lane element is regarded as a classification task.

[0121] Specifically, the second sub-loss function L c This can be expressed by the following formula:

[0122]

[0123] Among them, M ij This represents an element in the target correlation matrix, i.e., a matrix element. Indicates with M ij The corresponding truth label, i.e., the change type of the target lane element, L f This represents the loss function used for classification.

[0124] This implementation improves the reliability of the determined second sub-loss function by setting the second sub-loss function to a classification loss function constructed based on the change type of the updated lane element and the change type of the target lane element.

[0125] Figure 4 A flow 400 illustrating an embodiment of a lane-level map update method is provided. This lane-level map update method may specifically include the following steps:

[0126] Step 401: Obtain the road top view and the corresponding historical lane elements.

[0127] In this embodiment, the executing entity may first obtain the road top view locally or on a remote server that stores the road top view.

[0128] Furthermore, the executing entity can map the area corresponding to the road top view to the lane-level map to obtain the mapped area, and determine the lane elements in the mapped area as the historical lane elements corresponding to the road top view.

[0129] Step 402: Input the road top view and historical lane elements into the map update model to generate lane update information.

[0130] In this embodiment, the executing entity can input the road top view and historical lane elements into the map update model to generate lane update information.

[0131] The map update model can be as follows: Figure 2 , Figure 3 The map update model obtained by the method described in the corresponding embodiment will not be repeated here.

[0132] Step 403: Update the lane-level map based on the lane update information.

[0133] In this embodiment, if the lane update information includes updated lane elements, the executing entity can update the lane-level map based on the updated lane elements; if the lane update information includes updated lane elements and the change type of the updated lane elements, the executing entity can update the lane-level map based on the updated lane elements and the change type of the updated lane elements to obtain the updated lane-level map.

[0134] It should be noted that, for the updated lane-level map, the change type of the lane element in the updated lane-level map can be displayed as a character or as the color of the updated lane element (e.g., blue for a changed element style, purple for an element style unchanged, red for an added element, and orange for a deleted element). This application does not limit this.

[0135] The lane-level map update method provided in the above embodiments of this disclosure improves the efficiency of generating lane-level maps and the accuracy and reliability of the generated lane-level maps by obtaining a road top view and the corresponding historical lane elements; inputting the road top view and historical lane elements into a map update model to generate lane update information; and updating the lane-level map based on the lane update information.

[0136] In some alternative approaches, the method further includes performing at least one of the following based on the updated lane-level map: determining road lane-level traffic information; determining lane-level candidate planning trajectories; and determining vehicle lane-level control strategies.

[0137] In this implementation, the executing entity can determine the road lane-level traffic information based on the updated lane-level map.

[0138] Among them, road lane-level traffic information can include lane usage frequency, traffic density, lane usage status, etc. Road lane-level traffic information can help optimize lane allocation and traffic light control to alleviate traffic congestion and other problems.

[0139] Furthermore, the implementing entity can determine lane-level candidate planning trajectories based on the updated lane-level map to avoid lanes with frequent lane changes or accidents, thereby enhancing the reliability and safety of autonomous driving.

[0140] Furthermore, the implementing entity can perform high-precision driving behavior modeling based on the updated lane-level map to determine lane-level control strategies, enabling the autonomous driving system to better learn human driver behavior patterns, such as lane changing, overtaking, and evasion.

[0141] exist Figure 4 Based on the corresponding implementation, the updated lane-level map can also be applied to route navigation scenarios to achieve lane-level navigation.

[0142] See also Figure 5 , Figure 5 This is a schematic diagram illustrating an application scenario of the lane-level map update method according to this embodiment.

[0143] The executing entity can acquire a road top-view image 501 via the image acquisition device of the autonomous vehicle, map the area corresponding to the road top-view image 501 onto a lane-level map to obtain a mapped area, and determine the lane elements in the mapped area as the historical lane elements 502 corresponding to the road top-view image. Further, the executing entity can input the road top-view image 501 and the corresponding historical lane elements 502 into a map update model 503 to generate lane update information. The map update model 503 can include a priori encoding network and an update network. The update network can include a first update sub-network and a second update sub-network. The first update sub-network can include an encoding network, a fusion network, and a decoding network. The priori encoding network encodes the historical lane elements to generate priori encoded features. The encoding network encodes the road top-view image to generate image features. The fusion network fuses the image features and the priori encoded features to generate fused features. The decoding network decodes the fused features to generate updated lane elements. The second update sub-network generates the change type of the updated lane elements based on the updated lane elements and the priori encoded features.

[0144] Furthermore, the executing entity can update the lane-level map by updating the lane elements and the change type of the updated lane elements, thus obtaining the updated lane-level map.

[0145] Further reference Figure 6 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a training device for a map update model, which is similar to... Figure 2 The method embodiments shown correspond to those described.

[0146] like Figure 6 As shown, the training device 600 for the map update model in this embodiment includes: a first generation module 601, a second generation module 602, a function construction module 603, and a parameter adjustment module 604.

[0147] The first generation module 601 can be configured to input the historical lane elements corresponding to the road top view sample into the prior coding network to generate prior coding features.

[0148] The second generation module 602 can be configured to input prior encoded features and road top view samples into the update network to generate lane update information.

[0149] The function building module 603 is configured to construct a loss function based on the lane update information and the target lane elements corresponding to the road top view samples.

[0150] The parameter adjustment module 604 is configured to adjust the parameters of the map update model based on the loss function.

[0151] In some optional embodiments of this example, the function construction module is further configured to: construct a first sub-loss function based on the updated lane element and the target lane element; construct a second sub-loss function based on the change type of the updated lane element and the change type of the target lane element; and determine the loss function based on the first sub-loss function and the second sub-loss function.

[0152] In some optional ways of this embodiment, the change type includes at least one of the following: element style change, no change in element style, element addition, and element deletion.

[0153] In some optional embodiments of this example, the update network includes a first update sub-network and a second update sub-network, and the second generation module further includes: a first update unit configured to input prior coding features and the road top view sample into the first update sub-network to generate updated lane elements; and a second update unit configured to input the updated lane elements and prior coding features into the second update sub-network to generate the change type of the updated lane elements.

[0154] In some optional embodiments of this example, the second update unit is further configured to: input the updated lane elements and prior encoded features into the first association network to generate an initial association matrix; and input the initial association matrix into the second association network to generate a target association matrix.

[0155] In some optional embodiments of this example, the updated lane elements and prior encoded features are input into the first association network to generate an initial association matrix, including: inputting the updated lane elements and prior encoded features into the multilayer perceptron network respectively to generate a first feature and a second feature; inputting the first feature and the second feature into an outer product network to generate a feature-level association matrix; and inputting the feature-level association matrix into a classification network to generate an initial association matrix.

[0156] In some optional ways of this embodiment, inputting the initial association matrix into the second association network to generate the target association matrix includes: inputting the initial association matrix into the matching network to generate the matched association matrix; and inputting the matched association matrix into the filtering network to generate the target association matrix.

[0157] In some optional embodiments of this example, the first sub-loss function includes: a regression loss function, a cosine similarity loss function, and a classification loss function.

[0158] In some alternative embodiments of this example, the second sub-loss function is a classification loss function constructed based on the change type of the updated lane element and the change type of the target lane element.

[0159] In some optional embodiments of this example, both the historical lane element and the target lane element include a set of vectorized points and element style information.

[0160] Further reference Figure 7 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a lane-level map updating device, which is similar to... Figure 4 The method embodiments shown correspond to those described.

[0161] like Figure 7 As shown, the lane-level map updating device 700 of this embodiment includes: a data acquisition module 701, a lane updating module 702, and a map updating module 703.

[0162] The data acquisition module 701 can be configured to acquire the road top view and the historical lane elements corresponding to the road top view.

[0163] The lane update module 702 can be configured to input a road top view and historical lane elements into the map update model to generate lane update information.

[0164] The map update module 703 is configured to update the lane-level map based on lane update information.

[0165] In some optional embodiments of this invention, the device further includes a strategy execution module, wherein the strategy execution model is configured to perform at least one of the following based on the updated lane-level map: determining road lane-level traffic information; determining lane-level candidate planned trajectories; and determining vehicle lane-level control strategies.

[0166] This disclosure also provides a navigation device including a map navigation module configured to perform route navigation based on a target lane-level map. The target lane-level map is... Figure 7 The updated lane-level map obtained by the lane-level map updating device shown.

[0167] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0168] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0169] like Figure 8 The diagram shown is a block diagram of an electronic device for a map update model training method according to an embodiment of the present disclosure.

[0170] 800 is a block diagram of an electronic device for a training method of a map update model according to embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0171] like Figure 8As shown, the electronic device includes one or more processors 801, a memory 802, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 Take the 801 processor as an example.

[0172] The memory 802 is the non-transitory computer-readable storage medium provided in this disclosure. The memory stores instructions executable by at least one processor to cause the at least one processor to perform the training method for the map update model provided in this disclosure. The non-transitory computer-readable storage medium of this disclosure stores computer instructions for causing a computer to perform the training method for the map update model provided in this disclosure.

[0173] Memory 802, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the map update model training method in the embodiments of this disclosure (e.g., appendix). Figure 6 The first generation module 601, the second generation module 602, the function construction module 603, and the parameter adjustment module 604 are shown. The processor 801 executes various functional applications and data processing of the server by running non-transient software programs, instructions, and modules stored in the memory 802, thereby implementing the training method of the map update model in the above method embodiment.

[0174] Memory 802 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the use of the face-tracking electronic device. Furthermore, memory 802 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 802 may optionally include memory remotely located relative to processor 801, and this remote memory can be connected to the lane-line detection electronic device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0175] The electronic device for training the map update model may further include an input device 803 and an output device 804. The processor 801, memory 802, input device 803, and output device 804 can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0176] Input device 803 can receive input digital or character information, as well as key signal inputs related to user settings and function control of the electronic device for lane detection, such as touch screens, keypads, mice, trackpads, touchpads, pointers, one or more mouse buttons, trackballs, joysticks, etc. Output device 804 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The display device may include, but is not limited to, liquid crystal displays (LCDs), light-emitting diode (LED) displays, and plasma displays. In some embodiments, the display device may be a touch screen.

[0177] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0178] These computational programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0179] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0180] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0181] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0182] The technical solution according to the embodiments of this disclosure improves the accuracy and reliability of the trained map update model.

[0183] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0184] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for training a map update model, the map update model comprising a prior encoding network and an update network, the update network comprising a first update subnetwork and a second update subnetwork, and the method comprising: The historical lane elements corresponding to the road top view sample are input into the prior coding network to generate prior coding features. The prior coding network is used to encode the geometric features, semantic features and spatial relationship features of the lane elements. The prior encoded features and the road top view sample are input into the first update sub-network to generate updated lane elements; The updated lane elements and the prior encoded features are input into the second update sub-network to generate the change type of the updated lane elements; Based on the lane update information and the target lane elements corresponding to the road top view sample, a loss function is constructed. The lane update information includes the updated lane elements and the change type of the updated lane elements. The parameters of the map update model are adjusted based on the loss function.

2. The method according to claim 1, wherein, The loss function is constructed based on the lane update information and the target lane elements corresponding to the road top view samples, including: Based on the updated lane elements and the target lane elements, a first sub-loss function is constructed; Based on the change type of the updated lane element and the change type of the target lane element, a second sub-loss function is constructed; The loss function is determined based on the first sub-loss function and the second sub-loss function.

3. The method according to claim 2, wherein, The change type includes at least one of the following: element style change, element style no change, element addition, and element deletion.

4. The method according to claim 1, wherein, The second update sub-network includes a first association network and a second association network, and the step of inputting the updated lane element and the prior encoded feature into the second update sub-network to generate the change type of the updated lane element includes: The updated lane elements and the prior encoded features are input into the first association network to generate an initial association matrix; The initial correlation matrix is ​​input into the second correlation network to generate a target correlation matrix, wherein an element of the target correlation matrix is ​​used to indicate the change type of a lane element.

5. The method according to claim 4, wherein, The first association network includes: a multilayer perceptron network, an outer product network, and a classification network. The step of inputting the updated lane elements and the prior encoded features into the first association network to generate an initial association matrix includes: The updated lane elements and the prior encoded features are respectively input into the multilayer perceptron network to generate the first feature and the second feature; The first feature and the second feature are input into the outer product network to generate a feature-level correlation matrix; The feature-level correlation matrix is ​​input into the classification network to generate an initial correlation matrix.

6. The method according to claim 4, wherein, The second association network includes a matching network and a filtering network, and the step of inputting the initial association matrix into the second association network to generate the target association matrix includes: The initial association matrix is ​​input into the matching network to generate a matched association matrix. The matching network is used to perform matching processing on the initial association matrix using a preset matching strategy. The matched association matrix is ​​input into a filtering network to generate a target association matrix. The filtering network is used to filter the elements in the matched association matrix based on the confidence of the updated lane elements and / or the matching degree between the updated lane elements and the target lane elements.

7. The method according to claim 2, wherein, The first sub-loss function includes: regression loss function, cosine similarity loss function, and classification loss function.

8. The method according to claim 2, wherein, The second sub-loss function is a classification loss function constructed based on the change type of the updated lane element and the change type of the target lane element.

9. The method according to any one of claims 1-8, wherein, Both the historical lane elements and the target lane elements include vectorized point sets and element style information.

10. A method for updating a lane-level map, comprising: Obtain the road top view and the corresponding historical lane elements; The road top view and the historical lane elements are input into the map update model to generate lane update information, wherein the map update model is a map update model obtained by using any one of the methods described in claims 1-9; The lane-level map is updated based on the lane update information.

11. The method according to claim 10, further comprising: Based on the updated lane-level map, perform at least one of the following: Determine road lane-level traffic information; Determine lane-level candidate planning trajectories; Determine the vehicle lane-level control strategy.

12. A navigation method, comprising: Route navigation is performed based on a target lane-level map, wherein the target lane-level map is an updated lane-level map obtained by using the method described in claim 10 or 11.

13. A training apparatus for a map update model, the map update model comprising a prior encoding network and an update network, the update network comprising a first update subnetwork and a second update subnetwork, and the apparatus comprising: The first generation module is configured to input the historical lane elements corresponding to the road top view sample into the prior coding network to generate prior coding features. The prior coding network is used to encode the geometric features, semantic features and spatial relationship features of the lane elements. The second generation module includes a first update unit and a second update unit. The first update unit is configured to input the prior encoded features and the road top view sample into the first update sub-network to generate updated lane elements. The second update unit is configured to input the updated lane element and the prior encoded feature into the second update sub-network to generate the change type of the updated lane element; The function construction module is configured to construct a loss function based on the lane update information and the target lane element corresponding to the road top view sample. The lane update information includes the updated lane element and the change type of the updated lane element. The parameter adjustment module is configured to adjust the parameters of the map update model based on the loss function.

14. The apparatus according to claim 13, wherein, The lane update information includes the updated lane elements and the change type of the updated lane elements, and the function construction module is further configured to: Based on the updated lane elements and the target lane elements, a first sub-loss function is constructed; Based on the change type of the updated lane element and the change type of the target lane element, a second sub-loss function is constructed; The loss function is determined based on the first sub-loss function and the second sub-loss function.

15. The apparatus according to claim 14, wherein, The change type includes at least one of the following: element style change, element style no change, element addition, and element deletion.

16. The apparatus according to claim 13, wherein, The second update subnetwork includes a first association network and a second association network, and the second update unit is further configured to: The updated lane elements and the prior encoded features are input into the first association network to generate an initial association matrix; The initial correlation matrix is ​​input into the second correlation network to generate a target correlation matrix, wherein an element of the target correlation matrix is ​​used to indicate the change type of a lane element.

17. The apparatus according to claim 16, wherein, The first association network includes: a multilayer perceptron network, an outer product network, and a classification network. The step of inputting the updated lane elements and the prior encoded features into the first association network to generate an initial association matrix includes: The updated lane elements and the prior encoded features are respectively input into the multilayer perceptron network to generate the first feature and the second feature; The first feature and the second feature are input into the outer product network to generate a feature-level correlation matrix; The feature-level correlation matrix is ​​input into the classification network to generate an initial correlation matrix.

18. The apparatus according to claim 16, wherein, The second association network includes a matching network and a filtering network, and the step of inputting the initial association matrix into the second association network to generate the target association matrix includes: The initial association matrix is ​​input into the matching network to generate a matched association matrix. The matching network is used to perform matching processing on the initial association matrix using a preset matching strategy. The matched association matrix is ​​input into a filtering network to generate a target association matrix. The filtering network is used to filter the elements in the matched association matrix based on the confidence of the updated lane elements and / or the matching degree between the updated lane elements and the target lane elements.

19. The apparatus according to claim 14, wherein, The first sub-loss function includes: regression loss function, cosine similarity loss function, and classification loss function.

20. The apparatus according to claim 14, wherein, The second sub-loss function is a classification loss function constructed based on the change type of the updated lane element and the change type of the target lane element.

21. The apparatus according to any one of claims 13-20, wherein, The lane elements include a set of vectorized points and element style information.

22. A lane-level map updating device, comprising: The data acquisition module is configured to acquire a road top view and the historical lane elements corresponding to the road top view; The lane update module is configured to input the road top view and the historical lane elements into a map update model to generate lane update information, wherein the map update model is a map update model obtained using the apparatus as described in any one of claims 13-21. The map update module is configured to update the lane-level map based on the lane update information.

23. The apparatus of claim 22, further comprising: The policy execution module is configured to: Based on the updated lane-level map, perform at least one of the following: Determine road lane-level traffic information; Determine lane-level candidate planning trajectories; Determine the vehicle lane-level control strategy.

24. A navigation device, the device comprising: Map navigation module, the map navigation module is configured to: Route navigation is performed based on a target lane-level map, which is an updated lane-level map obtained using the apparatus described in claim 22 or 23.

25. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.

26. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-12.

27. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-12.

Images