High-precision map generation method and device, equipment and storage medium

Abstract

The invention discloses a high-precision map generation method and device, equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring a to-be-processed look-around image; inputting the look-around image into a preset map element detection model to perform feature extraction processing on the look-around image to obtain image features; performing map element detection processing on the image features to obtain map element query features; updating the map element query feature to obtain a target map element query feature; and performing classification regression processing on the target map element query features to obtain an output result of the preset map element detection model, and generating the high-precision map according to the output result. According to the scheme, the reliability and efficiency of high-precision map generation can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, specifically to technical fields such as intelligent transportation and autonomous driving, and in particular to a method, device, equipment and storage medium for generating a high-precision map. Background Art

[0002] In autonomous driving systems, high-definition maps (HD Maps) usually encapsulate precise vectorized details of map elements, such as crosswalks, lane dividers, road boundaries, etc. The accuracy of HD Maps directly affects the safety of autonomous driving.

[0003] At present, the technical solutions in related technologies generally define map construction as a semantic segmentation task in the Bird's Eye View (BEV) space to generate a rasterized high-precision map. Alternatively, it is based on the projection of the BEV space, and a high-precision map is obtained by using the constructed end-to-end map element detection model. However, the above-mentioned related technology solutions face limitations because a large amount of post-processing is required to obtain vectorized information. Moreover, for the solution based on BEV space projection, there is a conversion from image space to BEV space, which will result in a relatively high time consumption and high pressure on data processing. Summary of the invention

[0004] This application provides a method, device, equipment and storage medium for generating a high-precision map. The technical solution is as follows:

[0005] In a first aspect, a method for generating a high-precision map is provided, the method comprising:

[0006] Obtain the surround view image to be processed;

[0007] Inputting the surround view image into a preset map element detection model to perform feature extraction processing on the surround view image to obtain image features;

[0008] Performing map element detection processing on the image features to obtain map element query features;

[0009] performing update processing on the map element query feature to obtain a target map element query feature;

[0010] Classification and regression processing is performed on the target map element query features to obtain the output result of the preset map element detection model, so as to generate the high-precision map according to the output result.

[0011] In a possible implementation, the preset map element detection model includes an image semantic segmentation network and a depth prediction network, and the performing map element detection processing on the image features to obtain map element query features includes:

[0012] Obtaining a map element semantic feature according to the image semantic segmentation network and the image feature, so as to determine a first map element feature according to the map element semantic feature;

[0013] performing map element image feature extraction processing on the image feature according to the map element semantic feature to obtain a second map element feature;

[0014] According to the depth prediction network and the map element semantic features, performing map element position feature extraction processing on the image features to obtain map element position embedding features;

[0015] The first map element feature, the second map element feature, and the map element position embedding feature are concatenated to obtain a map element query feature.

[0016] In a possible implementation, performing map element position feature extraction processing on the image feature according to the depth prediction network and the map element semantic feature to obtain a map element position embedding feature includes:

[0017] According to the depth prediction network and the image feature, obtaining the depth image feature;

[0018] According to the semantic features of the map elements, the depth image features are sampled to obtain the depth image features of the map elements;

[0019] Perform position embedding processing on the map element depth image feature to obtain a map element position embedding feature.

[0020] In a possible implementation, determining the first map element feature according to the map element semantic feature includes:

[0021] Performing dimensionality-upgrading processing on the semantic features of the map elements;

[0022] According to the result of the dimension-raising process, a first map element feature is obtained.

[0023] In a possible implementation, performing map element image feature extraction processing on the image feature according to the map element semantic feature to obtain a second map element feature includes:

[0024] According to the semantic features of the map elements, sampling processing is performed on the image features;

[0025] Obtaining map element image features according to the results of the sampling process;

[0026] Performing dimensionality-upgrading processing on the map element image features;

[0027] According to the result of the dimension increasing process, a second map element feature is obtained.

[0028] In a possible implementation, the preset map element detection model includes an attention mechanism network, and the updating process of the map element query feature to obtain the target map element query feature includes:

[0029] According to the attention mechanism network, updating the map element query feature;

[0030] Based on the result of the updating process, a target map element query feature is obtained.

[0031] In a possible implementation, the attention mechanism network includes a self-attention layer, a first cross-attention layer, and a second cross-attention layer. The updating process of the map element query feature according to the attention mechanism network includes:

[0032] According to the self-attention layer, the map element query feature is subjected to category fusion processing to obtain a map element category query feature;

[0033] According to the first cross attention layer, performing category interaction processing on the map element category query feature to obtain a map element category interaction query feature;

[0034] Get the map element query feature queue of the previous frame of the current frame;

[0035] According to the second cross attention layer and the map element query feature queue of the previous frame of the current frame, the map element category interactive query feature is subjected to inter-frame interactive processing to obtain the result of the inter-frame interactive processing;

[0036] The result of the inter-frame interaction process is used as the result of the update process.

[0037] In a possible implementation, the preset map element detection model further includes a classification network and a regression network, the output result includes map element category information and location information corresponding to the map element category information, and the classification and regression processing is performed on the target map element query feature to obtain the output result of the preset map element detection model, including:

[0038] Inputting the target map element query feature into the classification network to output the map element category information;

[0039] The target map element query feature is input into the regression network to output the location information corresponding to the map element category information.

[0040] In a second aspect, a model training method is provided, the method comprising:

[0041] Get a sample surround image;

[0042] Inputting the sample surround image into a map element detection model to be trained to perform feature extraction processing on the sample surround image to obtain image features;

[0043] Performing map element detection processing on the image features to obtain map element query features;

[0044] performing update processing on the map element query feature to obtain a target map element query feature;

[0045] Performing classification and regression processing on the target map element query features to obtain an output result of the map element detection model to be trained;

[0046] The map element detection model to be trained is iteratively trained according to the output result and the sample surround image to obtain a trained map element detection model.

[0047] In a third aspect, a device for generating a high-precision map is provided, the device comprising:

[0048] A first acquisition unit, used to acquire a surround view image to be processed;

[0049] A first extraction unit, used for inputting the surround view image into a preset map element detection model to perform feature extraction processing on the surround view image to obtain image features;

[0050] A first detection unit, configured to perform map element detection processing on the image feature to obtain a map element query feature;

[0051] A first updating unit, configured to update the map element query feature to obtain a target map element query feature;

[0052] The first output unit is used to perform classification and regression processing on the target map element query feature to obtain the output result of the preset map element detection model, so as to generate the high-precision map according to the output result.

[0053] In a fourth aspect, a model training device is provided, the device comprising:

[0054] A second acquisition unit is used to acquire a sample surround image;

[0055] A second extraction unit is used to input the sample surround image into a map element detection model to be trained, so as to perform feature extraction processing on the sample surround image to obtain image features;

[0056] A second detection unit is used to perform map element detection processing on the image feature to obtain a map element query feature;

[0057] A second updating unit, configured to update the map element query feature to obtain a target map element query feature;

[0058] A second output unit, used for performing classification and regression processing on the target map element query feature to obtain an output result of the map element detection model to be trained;

[0059] The first training unit is used to iteratively train the map element detection model to be trained according to the output result and the sample surround image to obtain a trained map element detection model.

[0060] In a fifth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the method of the above-mentioned aspects and any possible implementation manner.

[0061] In a sixth aspect, an electronic device is provided, including:

[0062] at least one processor; and

[0063] a memory communicatively connected to the at least one processor; wherein,

[0064] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any possible implementation manner and the aspects described above.

[0065] In a seventh aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the above-mentioned aspects and any possible implementation method.

[0066] The beneficial effects of the technical solution provided by this application include at least:

[0067] It can be seen from the above technical scheme that, on the one hand, the embodiment of the present application can obtain the surround view image to be processed, and then input the surround view image into a preset map element detection model to perform feature extraction processing on the surround view image to obtain image features, perform map element detection processing on the image features to obtain map element query features, update the map element query features to obtain target map element query features, perform classification and regression processing on the target map element query features to obtain the output result of the preset map element detection model, and generate the high-precision map according to the output result. Since the surround view image can be directly processed by using the preset map element detection model to generate the high-precision map by using the output result of the model, while ensuring the accuracy of the information used to generate the high-precision map, there is no need to convert the image space information into the bird's-eye view space information, which reduces the pressure of data processing and ensures the efficiency and reliability of high-precision map generation.

[0068] As can be seen from the above technical solution, on the other hand, the embodiment of the present application can obtain a sample surround image and then input the sample surround image into a map element detection model to be trained, so as to perform feature extraction processing on the sample surround image to obtain image features, perform map element detection processing on the image features to obtain map element query features, update the map element query features to obtain target map element query features, perform classification and regression processing on the target map element query features to obtain the output result of the map element detection model to be trained, and iteratively train the map element detection model to be trained according to the output result and the sample surround image to obtain a trained map element detection model. Since the output result of the map element detection model to be trained can be obtained based on the map element query features extracted from the sample surround image, and the map element detection model to be trained can be iteratively trained according to the sample surround image and the output result of the model, a map element detection model that generates high-precision map element information can be obtained by directly processing the surround image, thereby improving the learning performance of the map element detection model for the surround image and the reliability of the map element detection model detection.

[0069] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0071] Figure 1 It is a flowchart of a method for generating a high-precision map provided by an embodiment of the present application;

[0072] Figure 2 is a flowchart of a model training method provided by another embodiment of the present application;

[0073] Figure 3 is a flowchart of a method for generating a high-precision map provided in another embodiment of the present application;

[0074] Figure 4 It is a schematic diagram of the architecture of a map element detection model in a method for generating a high-precision map provided in an embodiment of the present application;

[0075] Figure 5 is a structural block diagram of a device for generating a high-precision map provided in another embodiment of the present application;

[0076] Figure 6 It is a structural block diagram of a model training device provided in another embodiment of the present application.

[0077] Figure 7 It is a block diagram of an electronic device used to implement the high-precision map generation method and model training method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0078] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0079] Obviously, the described embodiments are only part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0080] It should be noted that the terminal devices involved in the embodiments of the present application may include but are not limited to mobile phones, personal digital assistants (PDAs), wireless handheld devices, tablet computers and other smart devices; display devices may include but are not limited to personal computers, televisions and other devices with display functions.

[0081] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0082] Please refer to Figure 1 , which shows a schematic diagram of a process of generating a high-precision map provided by an embodiment of the present application. The method for generating a high-precision map may specifically include:

[0083] Step 101: Obtain a surround view image to be processed.

[0084] Step 102: Input the surround view image into a preset map element detection model to perform feature extraction processing on the surround view image to obtain image features.

[0085] Step 103: Perform map element detection processing on the image features to obtain map element query features.

[0086] Step 104: update the map element query feature to obtain the target map element query feature.

[0087] Step 105: Perform classification regression processing on the target map element query features to obtain the output result of the preset map element detection model, so as to generate the high-precision map according to the output result.

[0088] It should be noted that the preset map element detection model can be a model of map elements used to generate high-precision maps.

[0089] It should be noted that the surround view image can be determined by seamlessly stitching images captured by multiple acquisition devices installed around the data acquisition vehicle.

[0090] It should be noted that the map element query feature may be a map element query feature, that is, an instance query.

[0091] It should be noted that part or all of the execution subject of steps 101 to 105 may be an application located in the local terminal, or may also be a functional unit such as a plug-in or software development kit (SDK) set in the application located in the local terminal, or may also be a processing engine located in a network-side server, or may also be a distributed system located on the network side, for example, a processing engine or distributed system in an information processing platform on the network side, etc. This embodiment does not specifically limit this.

[0092] It is understandable that the application may be a local program (nativeApp) installed on the local terminal, or may be a webpage program (webApp) of a browser on the local terminal, which is not limited in this embodiment.

[0093] In this way, the surround image can be directly processed by utilizing the preset map element detection model to obtain the output result for generating a high-precision map. While ensuring the accuracy of the information used to generate the high-precision map, there is no need to convert the image spatial information into the bird's-eye view spatial information, thus reducing the pressure of data processing and ensuring the efficiency and reliability of high-precision map generation.

[0094] Optionally, in a possible implementation of this embodiment, the preset map element detection model may include a backbone network. In step 102, the surround image is input into the preset map element detection model, and the backbone network may be used to perform feature extraction processing on the surround image to obtain image features.

[0095] In this implementation, the image feature may be a high-dimensional feature of an image corresponding to the surround view image to be processed, that is, a high-dimensional image feature.

[0096] It can be understood that high-dimensional features are more structured and complex features composed of low-dimensional features.

[0097] In this way, the features of the input surround view image can be extracted through the backbone network, which can further improve the reliability of the image features.

[0098] Optionally, in a possible implementation of this embodiment, the preset map element detection model may include an image semantic segmentation network and a depth prediction network. In step 103, map element semantic features may be obtained specifically according to the image semantic segmentation network and the image features, so as to determine a first map element feature according to the map element semantic features, and then perform map element image feature extraction processing on the image features according to the map element semantic features to obtain a second map element feature, perform map element position feature extraction processing on the image features according to the depth prediction network and the map element semantic features to obtain a map element position embedding feature, and perform splicing processing on the first map element feature, the second map element feature, and the map element position embedding feature to obtain a map element query feature.

[0099] In this implementation, the map elements may include map elements in a high-precision map. The map elements may include, but are not limited to, road shapes, lane information, traffic signs, traffic lights, etc. For example, lane lines, crosswalk lines, etc.

[0100] In a specific implementation process of this implementation, first, the depth image feature can be obtained according to the depth prediction network and the image feature. Secondly, the depth image feature can be sampled according to the map element semantic feature to obtain the map element depth image feature. Thirdly, the map element depth image feature can be positionally embedded to obtain the map element position embedded feature.

[0101] In one case of the specific implementation process, the map element detection model may further include an embedding feature extraction network. Furthermore, the embedding feature extraction network of the map element detection model may be used to perform position embedding processing on the map element deep image features to obtain map element position embedding features.

[0102] Here, the embedding feature extraction network may include a network based on a multi-layer neural network. The map element position embedding feature may include an embedding feature of each map element position.

[0103] In another specific implementation process of this implementation mode, firstly, the semantic feature of the map element may be subjected to dimensionality-upgrading processing. Secondly, the first map element feature may be obtained according to the result of the dimensionality-upgrading processing.

[0104] In one case of the specific implementation process, the map element semantic features can also be processed by a first multilayer perceptron (MLP) network to increase the dimension of the map element semantic features to obtain the map element semantic features after the increase in dimension. Secondly, the map element semantic features after the increase in dimension can be used as the first map element features.

[0105] In another specific implementation process of the implementation mode, the sampling process of the image feature may include sampling process and dimension upgrading process. First, the image feature may be sampled according to the map element semantic feature. Second, the map element image feature may be obtained according to the result of the sampling process. Third, the map element image feature may be dimension upgraded. Third, the second map element feature may be obtained according to the result of the dimension upgrading process.

[0106] In this implementation, the result of the sampling process may include the image feature having the map element semantics in the image feature. Here, the image feature having the map element semantics in the image feature may be the second map element feature.

[0107] In one case of the specific implementation process, a second multi-layer perceptron network may be used to perform dimensionality upscaling on the map element image features, and then a second map element feature may be obtained based on the result of the dimensionality upscaling.

[0108] In this way, the first map element feature, the second map element feature, and the map element position embedding feature obtained by performing three types of processing on the image features can be spliced ​​together to obtain a more effective map element query feature, so as to improve the reliability of the subsequent processing results of the map element query feature.

[0109] It should be noted that the specific implementation process provided in this implementation method can be combined with the various specific implementation processes provided in the aforementioned implementation methods to implement the method for generating a high-precision map of this embodiment. For a detailed description, please refer to the relevant content in the aforementioned implementation method, which will not be repeated here.

[0110] Optionally, in a possible implementation of this embodiment, the preset map element detection model may include an attention mechanism network. In step 104, the map element query feature may be updated based on the attention mechanism network, and then the target map element query feature may be obtained based on the result of the update process.

[0111] In this implementation, the attention mechanism network may include a self-attention layer, a first cross-attention layer, and a second cross-attention layer.

[0112] Here, the self-attention layer may be an intra-class self-attention mechanism layer, which may be used to perform category fusion processing on map element query features of the same category in different images. The first cross-attention layer may be an intra-frame cross-attention mechanism layer, which may be used to perform intra-frame cross-processing on map element query features in the same data frame. The second cross-attention layer may be an inter-frame cross-attention mechanism layer, which may be used to perform inter-frame cross-processing on map element query features between different data frames.

[0113] In a specific implementation process of this implementation method, first, the map element query feature can be subjected to category fusion processing according to the self-attention layer to obtain a map element category query feature. Secondly, the map element category query feature can be subjected to category interaction processing according to the first cross-attention layer to obtain a map element category interaction query feature. Thirdly, a map element query feature queue of the previous frame of the current frame can be obtained. Thirdly, based on the second cross-attention layer and the map element query feature queue of the previous frame of the current frame, an inter-frame interaction processing can be performed on the map element category interaction query feature to obtain a result of the inter-frame interaction processing. Thirdly, the result of the inter-frame interaction processing is used as the result of the update processing.

[0114] Here, the current frame may refer to a surround view image data frame currently detected in the detection process.

[0115] In this implementation, the attention mechanism network can be a transformer module, and this transformer module can include multiple layers of loops in the network structure.

[0116] In another specific implementation process of this implementation, a map element query feature queue of the current frame can be obtained based on the result of the update process. The map element query feature queue can be a collection of map element query features of multiple categories.

[0117] In this way, by updating the map element query features according to the attention mechanism network, more accurate target map element query features can be obtained, the accuracy of the map elements is improved, and thus the reliability of the generated high-precision map is improved.

[0118] It should be noted that the specific implementation process provided in this implementation method can be combined with the various specific implementation processes provided in the aforementioned implementation methods to implement the method for generating a high-precision map of this embodiment. For a detailed description, please refer to the relevant content in the aforementioned implementation method, which will not be repeated here.

[0119] Optionally, in a possible implementation of this embodiment, the preset map element detection model may further include a classification network and a regression network, and the output result includes map element category information and location information corresponding to the map element category information. In step 105, the target map element query feature may be input into the classification network to output the map element category information, and then the target map element query feature may be input into the regression network to output the location information corresponding to the map element category information.

[0120] In this implementation, the classification network may be a classification head network, and the regression network may be a regression head network.

[0121] Here, the location information corresponding to the map element category information may be a set of location points. For example, if the map element category information is a lane line, the location information corresponding to it may be a set of location points marking the lane line.

[0122] In this way, the target map element query features can be processed by using the classification network and the regression network respectively to obtain the location information corresponding to the map element category information, which can further improve the accuracy of the detected map element information.

[0123] It should be noted that the specific implementation process provided in this implementation method can be combined with the various specific implementation processes provided in the aforementioned implementation methods to implement the method for generating a high-precision map of this embodiment. For a detailed description, please refer to the relevant content in the aforementioned implementation method, which will not be repeated here.

[0124] Figure 2 is a flow chart of a model training method provided by another embodiment of the present application, such as Figure 2 The method can be implemented as follows:

[0125] Step 201: Acquire a sample surround image.

[0126] Step 202: Input the sample surround image into the map element detection model to be trained to perform feature extraction processing on the sample surround image to obtain image features.

[0127] Step 203: Perform map element detection processing on the image features to obtain map element query features.

[0128] Step 204: update the map element query feature to obtain the target map element query feature.

[0129] Step 205: Perform classification and regression processing on the target map element query features to obtain the output result of the map element detection model to be trained.

[0130] Step 206: Iteratively train the map element detection model to be trained according to the output result and the sample surround image to obtain a trained map element detection model.

[0131] It should be noted that part or all of the execution subject of steps 201 to 206 may be an application located in the local terminal, or may also be a functional unit such as a plug-in or software development kit (SDK) set in the application located in the local terminal, or may also be a processing engine located in a network-side server, or may also be a distributed system located on the network side, for example, a processing engine or distributed system in a model training platform on the network side, etc. This embodiment does not specifically limit this.

[0132] It is understandable that the application may be a local program (nativeApp) installed on the local terminal, or may be a webpage program (webApp) of a browser on the local terminal, which is not limited in this embodiment.

[0133] In this way, the output results of the map element detection model to be trained can be obtained based on the map element query features extracted from the sample surround view image, and the map element detection model to be trained can be iteratively trained based on the sample surround view image and the output results of the model. A map element detection model that can generate high-precision map element information by directly processing the surround view image can be obtained, thereby improving the learning performance and reliability of the map element detection model for the surround view image.

[0134] Optionally, in a possible implementation of this embodiment, the map element detection model to be trained may include an image semantic segmentation network, a depth prediction network, a first multi-layer perceptron network, a second multi-layer perceptron network, an attention mechanism network, a classification network and a regression network.

[0135] In this implementation, the attention mechanism network may include a transformer-based multi-loop network. The attention mechanism network may include a self-attention layer, a first cross-attention layer, and a second cross-attention layer.

[0136] It is understandable that the trained map element detection model can be used to execute the method for generating a high-precision map in the aforementioned embodiment. Specifically, the surround image can be input into a preset map element detection model to perform feature extraction processing on the surround image to obtain image features, map element detection processing on the image features to obtain map element query features, update processing on the map element query features to obtain target map element query features, classification and regression processing on the target map element query features to obtain the output result of the preset map element detection model, and generate the high-precision map based on the output result.

[0137] Optionally, in a possible implementation of this embodiment, the preset map element detection model includes an image semantic segmentation network and a depth prediction network. In step 203, map element semantic features can be obtained specifically according to the image semantic segmentation network and the image features, so as to determine a first map element feature according to the map element semantic features, and then map element image feature extraction processing can be performed on the image features according to the map element semantic features to obtain a second map element feature, map element position feature extraction processing can be performed on the image features according to the depth prediction network and the map element semantic features to obtain a map element position embedding feature, and the first map element feature, the second map element feature, and the map element position embedding feature can be spliced ​​to obtain a map element query feature.

[0138] In a specific implementation process of this implementation method, first, the depth image feature can be obtained according to the depth prediction network and the image feature. Secondly, according to the map element semantic feature, the depth image feature is sampled to obtain the map element depth image feature. Thirdly, the map element depth image feature is positionally embedded to obtain the map element position embedding feature.

[0139] In a specific implementation process of this implementation mode, first, the semantic feature of the map element may be subjected to dimensionality upgrading processing, and then the first map element feature may be obtained according to the result of the dimensionality upgrading processing.

[0140] In a specific implementation process of this implementation method, first, the image feature can be sampled according to the semantic feature of the map element. Secondly, the map element image feature is obtained according to the result of the sampling process. Thirdly, the map element image feature is dimensionally upgraded. Thirdly, the second map element feature is obtained according to the result of the dimensionally upgraded process.

[0141] Optionally, in a possible implementation of this embodiment, the preset map element detection model may further include an attention mechanism network. In step 204, the map element query feature may be updated based on the attention mechanism network; and the target map element query feature may be obtained based on the result of the update process.

[0142] In an implementation, the attention mechanism network may include a self-attention layer, a first cross-attention layer, and a second cross-attention layer.

[0143] In a specific implementation process of this implementation method, according to the self-attention layer, the map element query feature is subjected to category fusion processing to obtain a map element category query feature; according to the first cross-attention layer, the map element category query feature is subjected to category interaction processing to obtain a map element category interaction query feature; a map element query feature queue of a previous frame of the current frame is obtained; according to the second cross-attention layer and the map element query feature queue of a previous frame of the current frame, an inter-frame interaction processing is performed on the map element category interaction query feature to obtain a result of the inter-frame interaction processing; and the result of the inter-frame interaction processing is used as the result of the update processing.

[0144] Optionally, in a possible implementation of this embodiment, the preset map element detection model further includes a classification network and a regression network, and the output result includes map element category information and location information corresponding to the map element category information. In step 205, the target map element query feature can be input into the classification network to output the map element category information; and the target map element query feature can be input into the regression network to output the location information corresponding to the map element category information.

[0145] It should be noted that the specific implementation process provided in this implementation method can be combined with the various specific implementation processes provided in the aforementioned implementation methods to implement the method for generating a high-precision map of this embodiment. For a detailed description, please refer to the relevant content in the aforementioned implementation method, which will not be repeated here.

[0146] In order to better understand the method of the embodiment of the present application, the method of the embodiment of the present application is described below with reference to the accompanying drawings and specific application scenarios.

[0147] Figure 3 is a flow chart of a method for generating a high-precision map provided by another embodiment of the present application. Figure 3In this embodiment, in this application scenario, the preset map element detection model may include an image semantic segmentation network, a depth prediction network, a first multi-layer perceptron network, a second multi-layer perceptron network, an attention mechanism network, a classification network, and a regression network. The method for generating a high-precision map may include:

[0148] Step 301: Acquire a sample surround image.

[0149] Step 302: Input the sample surround view image into the map element detection model to be trained to obtain map element category information corresponding to the sample surround view image and location information corresponding to the map element category information.

[0150] Step 303: iteratively train the map element detection model to be trained according to the map element category information, the location information corresponding to the map element category information and the sample surround image to obtain a trained map element detection model.

[0151] Step 304: Acquire the surround view image to be processed.

[0152] Step 305: Input the surround view image into the trained map element detection model, and use the backbone network of the map element detection model to perform feature extraction processing on the surround view image to obtain image features.

[0153] In this embodiment, all surround images may be passed through a backbone network to obtain image features corresponding to the surround images, which may be high-dimensional image features.

[0154] Step 306: Use the image semantic segmentation network of the map element detection model to perform semantic segmentation processing on the image features to obtain map element semantic features.

[0155] In this embodiment, the image semantic segmentation network may be a network based on a two dimensional (2D) image segmentation algorithm.

[0156] Here, the map element semantic features may include but are not limited to semantic features such as road shape, lane information, traffic signs, traffic lights, etc. The map element semantic features may be element semantic features required for map generation.

[0157] Step 307: Utilize the first multi-layer perceptron network of the map element detection model to perform dimensionality-upgrading processing on the semantic features of the map elements to obtain the first map element features.

[0158] Step 308: Sample the image features according to the semantic features of the map elements to obtain the image features of the map elements.

[0159] In this embodiment, the map element semantic features may also be used to filter out the map element image features from the image features.

[0160] Step 309: Utilize the second multi-layer perceptron network of the map element detection model to perform dimensionality upscaling processing on the map element image features to obtain second map element features.

[0161] In this embodiment, the first multilayer perceptron network and the second multilayer perceptron network may be the same or different. The first multilayer perceptron network and the second multilayer perceptron network may both be used to unify the feature channels of the first map element feature and the second map element feature.

[0162] Step 310: Perform depth prediction processing on the image features according to the depth prediction network of the map element detection model to obtain depth image features.

[0163] In this embodiment, the depth prediction network may be a network based on a depth estimation algorithm. The depth prediction network may be used to predict depth information of image features.

[0164] Step 311: Sample the depth image features according to the semantic features of the map elements to obtain the depth image features of the map elements.

[0165] In this embodiment, first, the depth image features are sampled and processed according to the semantic features of the map elements to obtain the map element pixel features. Secondly, based on the camera parameters, the map element pixel features can be projected into the vehicle body coordinate system to obtain the position information of each map element in the vehicle body coordinate system. Thirdly, the position information of each map element in the vehicle body coordinate system is used as the map element depth image feature.

[0166] Here, the depth image feature of the map element may be a three-dimensional (3D) image feature. The map element pixel feature may be a map element semantic image pixel point set. The position information of each map element in the vehicle body coordinate system may be a map element position point set in the vehicle coordinate system, that is, a map element semantic image position point set.

[0167] Step 312: Using the embedding feature extraction network of the map element detection model, perform position embedding processing on the map element deep image features to obtain map element position embedding features.

[0168] In this embodiment, the embedding feature extraction network may be a multi-layer neural network. The map element position embedding feature may include an embedding feature of each map element position.

[0169] Step 313: concatenate the first map element feature, the second map element feature, and the map element position embedding feature to obtain a map element query feature.

[0170] Step 314: update the map element query features according to the attention mechanism network of the map element detection model to obtain the target map element query features.

[0171] In this implementation, the attention mechanism network may include a self-attention layer, a first cross-attention layer, and a second cross-attention layer.

[0172] First, according to the self-attention layer, the map element query features are subjected to category fusion processing to obtain the map element category query features. Secondly, according to the first cross-attention layer, the map element category query features are subjected to category interaction processing to obtain the map element category interaction query features. Thirdly, the map element query feature queue of the previous frame of the current frame is obtained. Thirdly, according to the second cross-attention layer and the map element query feature queue of the previous frame of the current frame, the map element category interaction query features are subjected to inter-frame interaction processing to obtain the result of the inter-frame interaction processing. Finally, the result of the inter-frame interaction processing can be used as the result of the update processing.

[0173] Exemplarily, first, the self-attention layer can utilize the intra-class self-attention mechanism (Self-attention) to perform category fusion processing on the map element query features to obtain the map element category query features.

[0174] Here, since the map element query feature is determined based on the result of 2D segmentation, the category of each map element query feature is known, such as lane lines, crosswalks, etc. The self-attention layer can fuse the same map element categories collected by different acquisition devices. In this way, the map element query features of the same category from different cameras can be fused.

[0175] Secondly, the first cross-attention layer uses the cross-attention mechanism to interact with all map element category query features in the current frame. Specifically, all map element category query features in the same frame, that is, the current frame, can be input into the first cross-attention layer for category interaction processing to obtain map element category interaction query features. In this way, information interaction between each map element query feature can be achieved.

[0176] Again, the first cross-attention layer uses the cross-attention mechanism to interact with all map element category interactive query features in the current frame and the map element query feature queue of the previous frame of the current frame, that is, to achieve cross-attention between frames. Since, after each prediction cycle, a query queue can be obtained to record historical map element query information, that is, the historical map element query feature queue. Each prediction can cross-attention the historical map element query feature queue of the previous frame and the map element category interactive query feature of the current frame. Here, the query queue is initialized randomly.

[0177] In this implementation, the attention mechanism network can be a transformer module, and this transformer module can include multiple layers of loops in the network structure.

[0178] Step 315: Input the target map element query feature into the classification network of the map element detection model to output map element category information, and input the target map element query feature into the regression network of the map element detection model to output location information corresponding to the map element category information.

[0179] In this implementation, the target map element query feature can be passed through a classification network to obtain the category information of the instance and the position information corresponding to the map element category information, and then passed through a regression network head to obtain the position information corresponding to the map element category information in the vehicle coordinate system, that is, a set of position points corresponding to the map element category information.

[0180] In this implementation, the classification network may be a classification head network, and the regression network may be a regression head network.

[0181] Here, the map element category may include but is not limited to road shape, lane line, pedestrian crossing line, street light, traffic signal light, etc. The map element category may be an element category used to generate a high-precision map.

[0182] In addition, the map element query feature queue may be updated based on the target map element query feature, that is, the target real column query, to serve as the information required for predicting future frames.

[0183] In this embodiment, Figure 4 is a schematic diagram of the architecture of a map element detection model in a method for generating a high-precision map provided in another embodiment of the present application, such as Figure 4As shown. First, after the surround image is input into the map element detection model, it can be processed by the backbone network to obtain image features. Secondly, the image features are processed in three branches, one of which is the 2D semantic element segmentation branch, which can use the 2D image segmentation algorithm. The semantic segmentation result of the image feature is obtained, and then the map element semantic features can be screened out, and then the map element semantic features are processed by MLP to obtain the first map element feature. One is the depth prediction branch, which can use the depth prediction algorithm to obtain the image feature with depth information. According to the map element semantic features, the image features with depth information are sampled to obtain the map element pixel point set, that is, the map element semantic image pixel point set, and the map element pixel point set is converted to the coordinate system to obtain the map element position point set in the vehicle coordinate system, that is, the map element deep image feature, and then the map element deep image feature is processed by the embedding feature extraction network to obtain the map element position embedding feature. One is the image feature branch, which can sample the image feature according to the map element semantic features to obtain the map element image feature, and then use MLP to increase the dimension of the map element image feature to obtain the second map element feature. Thirdly, the processing results of the three branches are spliced ​​to obtain the instance query, that is, the map element query feature. Thirdly, the instance query is input into the attention mechanism network, and the map element query feature is first fused by the intra-class self-attention mechanism layer, and then the intra-frame cross-attention mechanism layer is used to perform intra-frame cross-processing on the fusion result, and then the inter-frame cross-attention mechanism layer is used to perform inter-frame interaction processing on the intra-frame cross-processing result and the instance query queue of the previous frame. The instance query queue is updated with the result of the inter-frame interaction processing, and the result of the inter-frame interaction processing is respectively input into the classification network and the regression network to obtain the output result of the model, which may include the map element category information and the location information corresponding to the map element category information.

[0184] Step 316: Generate a high-precision map based on the map element category information and the location information corresponding to the map element category information.

[0185] Based on the technical solution of this embodiment, the surround view image is directly processed by using a preset map element detection model to generate a high-precision map using the output results of the model. This can effectively avoid the current use of dense BEV spatial information to generate high-precision maps and reduce the computational complexity and time consumption of data processing.

[0186] In addition, based on the technical solution of this embodiment, the 2D spatial semantic segmentation information of the image and the 3D spatial information of the image obtained by depth prediction can be combined to identify and predict the category and location information of map elements in the image, thereby improving the accuracy of online map element recognition and prediction.

[0187] In addition, based on the technical solution of this embodiment, 3D online map detection is performed based on 2D segmentation, and the detection results are optimized by combining the temporal information of the query features, so that accurate online high-precision map detection results are obtained without relying on bev space.

[0188] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0189] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0190] Figure 5 FIG. 1 shows a structural block diagram of a device for generating a high-precision map provided by an embodiment of the present application. Figure 5 As shown. The high-precision map generation device 500 of this embodiment may include a first acquisition unit 501, a first extraction unit 502, a first detection unit 503, a first update unit 504 and a first output unit 505. Among them, the first acquisition unit 501 is used to acquire the surround image to be processed; the first extraction unit 502 is used to input the surround image into a preset map element detection model to perform feature extraction processing on the surround image to obtain image features; the first detection unit 503 is used to perform map element detection processing on the image features to obtain map element query features; the first update unit 504 is used to update the map element query features to obtain target map element query features; the first output unit 505 is used to perform classification regression processing on the target map element query features to obtain the output result of the preset map element detection model, so as to generate the high-precision map according to the output result.

[0191] It should be noted that part or all of the high-precision map generation device of this embodiment may be an application located in the local terminal, or may also be a functional unit such as a plug-in or software development kit (SDK) set in the application located in the local terminal, or may also be a processing engine located in a network-side server, or may also be a distributed system located on the network side, for example, a processing engine or distributed system in an information processing platform on the network side, etc. This embodiment does not specifically limit this.

[0192] It is understandable that the application may be a local program (nativeApp) installed on the local terminal, or may be a webpage program (webApp) of a browser on the local terminal, which is not limited in this embodiment.

[0193] Optionally, in a possible implementation of this embodiment, the preset map element detection model includes an image semantic segmentation network and a depth prediction network, and the first detection unit 503 can be used to obtain map element semantic features according to the image semantic segmentation network and the image features, so as to determine a first map element feature according to the map element semantic features; perform map element image feature extraction processing on the image features according to the map element semantic features to obtain a second map element feature; perform map element position feature extraction processing on the image features according to the depth prediction network and the map element semantic features to obtain a map element position embedding feature; and perform splicing processing on the first map element feature, the second map element feature, and the map element position embedding feature to obtain a map element query feature.

[0194] Optionally, in a possible implementation of this embodiment, the first detection unit 503 can be used to obtain the depth image feature based on the depth prediction network and the image feature; sample the depth image feature according to the map element semantic feature to obtain the map element depth image feature; perform position embedding processing on the map element depth image feature to obtain the map element position embedding feature.

[0195] Optionally, in a possible implementation of this embodiment, the first detection unit 503 may be configured to perform dimensionality upscaling processing on the semantic feature of the map element; and obtain the first map element feature according to the result of the dimensionality upscaling processing.

[0196] Optionally, in a possible implementation of this embodiment, the first detection unit 503 can be used to perform sampling processing on the image feature according to the map element semantic feature; obtain map element image feature according to the result of the sampling processing; perform dimensionality upgrading processing on the map element image feature; and obtain second map element feature according to the result of the dimensionality upgrading processing.

[0197] Optionally, in a possible implementation of this embodiment, the preset map element detection model includes an attention mechanism network, and the first updating unit 504 can be used to update the map element query feature according to the attention mechanism network; and obtain the target map element query feature based on the result of the update process.

[0198] Optionally, in a possible implementation of this embodiment, the attention mechanism network includes a self-attention layer, a first cross-attention layer and a second cross-attention layer, and the first updating unit 504 can be used to perform category fusion processing on the map element query feature according to the self-attention layer to obtain a map element category query feature; perform category interaction processing on the map element category query feature according to the first cross-attention layer to obtain a map element category interaction query feature; obtain a map element query feature queue of a previous frame of the current frame; perform inter-frame interaction processing on the map element category interaction query feature according to the second cross-attention layer and the map element query feature queue of the previous frame of the current frame to obtain a result of the inter-frame interaction processing; and use the result of the inter-frame interaction processing as the result of the update processing.

[0199] Optionally, in a possible implementation of this embodiment, the preset map element detection model further includes a classification network and a regression network, the output result includes map element category information and position information corresponding to the map element category information, and the first output unit 505 can be used to input the target map element query feature into the classification network to output the map element category information; and input the target map element query feature into the regression network to output the position information corresponding to the map element category information.

[0200] In this embodiment, the surround image to be processed can be acquired by the first acquisition unit, and then the surround image can be input into the preset map element detection model by the first extraction unit to perform feature extraction processing on the surround image to obtain image features, the first detection unit performs map element detection processing on the image features to obtain map element query features, the first update unit performs update processing on the map element query features to obtain target map element query features, and the first output unit performs classification regression processing on the target map element query features to obtain the output result of the preset map element detection model, so as to generate the high-precision map according to the output result. Since the surround image can be directly processed by using the preset map element detection model to generate the high-precision map by using the output result of the model, while ensuring the accuracy of the information used to generate the high-precision map, there is no need to convert the image space information into the bird's-eye view space information, which reduces the pressure of data processing and ensures the efficiency and reliability of high-precision map generation.

[0201] Figure 6 A structural block diagram of a model training device provided by an embodiment of the present application is shown. Figure 6As shown. The model training device 600 of this embodiment may include a second acquisition unit 601, a second extraction unit 602, a second detection unit 603, a second updating unit 604, a second output unit 605 and a first training unit 606. The second acquisition unit 601 is used to acquire a sample surround image; the second extraction unit 602 is used to input the sample surround image into the map element detection model to be trained, so as to perform feature extraction processing on the sample surround image to obtain image features; the second detection unit 603 is used to perform map element detection processing on the image features to obtain map element query features; the second updating unit 604 is used to perform update processing on the map element query features to obtain target map element query features; the second output unit 605 is used to perform classification and regression processing on the target map element query features to obtain the output result of the map element detection model to be trained; the first training unit 606 is used to iteratively train the map element detection model to be trained according to the output result and the sample surround image to obtain a trained map element detection model.

[0202] It should be noted that part or all of the model training device of this embodiment may be an application located in the local terminal, or may also be a functional unit such as a plug-in or software development kit (SDK) set in the application located in the local terminal, or may also be a processing engine located in a network-side server, or may also be a distributed system located on the network side, for example, a processing engine or distributed system in a model training platform on the network side, etc. This embodiment does not specifically limit this.

[0203] It is understandable that the application may be a local program (nativeApp) installed on the local terminal, or may be a webpage program (webApp) of a browser on the local terminal, which is not limited in this embodiment.

[0204] Optionally, in a possible implementation of this embodiment, the map element detection model to be trained may include an image semantic segmentation network, a depth prediction network, a first multi-layer perceptron network, a second multi-layer perceptron network, an attention mechanism network, a classification network and a regression network.

[0205] In this embodiment, a sample surround image may be acquired by a second acquisition unit, and then the sample surround image may be input into a map element detection model to be trained by a second extraction unit to perform feature extraction processing on the sample surround image to obtain image features, the second detection unit may perform map element detection processing on the image features to obtain map element query features, the second update unit may perform update processing on the map element query features to obtain target map element query features, and the second output unit may perform classification regression processing on the target map element query features to obtain the output result of the map element detection model to be trained, so that the first training The unit can iteratively train the map element detection model to be trained according to the output result and the sample surround view image to obtain a trained map element detection model. Since the output result of the map element detection model to be trained can be obtained based on the map element query feature extracted from the sample surround view image, and the map element detection model to be trained is iteratively trained according to the sample surround view image and the output result of the model, a map element detection model that can generate high-precision map element information by directly processing the surround view image can be obtained, thereby improving the learning performance of the map element detection model for the surround view image and the detection reliability of the map element detection model.

[0206] In the technical solution of this application, the user personal information involved, such as the collection, storage, use, processing, transmission, provision and disclosure of user images and attribute data, etc., complies with the provisions of relevant laws and regulations and does not violate public order and good morals.

[0207] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0208] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0209] like Figure 7As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0210] Multiple components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0211] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as the method for generating a high-precision map and the method for model training. For example, in some embodiments, the method for generating a high-precision map and the method for model training may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the method for generating a high-precision map and the method for model training described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute the high-precision map generation method and the model training method in any other appropriate manner (for example, by means of firmware).

[0212] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0213] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0214] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0215] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0216] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0217] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0218] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0219] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for generating a high-precision map, characterized in that: The method comprises: Obtain the surround view image to be processed; Inputting the surround view image into a preset map element detection model to perform feature extraction processing on the surround view image to obtain image features; Performing map element detection processing on the image features to obtain map element query features; performing update processing on the map element query feature to obtain a target map element query feature; Classification and regression processing is performed on the target map element query features to obtain the output result of the preset map element detection model, so as to generate the high-precision map according to the output result.

2. The method according to claim 1, characterized in that The preset map element detection model includes an image semantic segmentation network and a depth prediction network, and the map element detection processing of the image features to obtain map element query features includes: Obtaining a map element semantic feature according to the image semantic segmentation network and the image feature, so as to determine a first map element feature according to the map element semantic feature; performing map element image feature extraction processing on the image feature according to the map element semantic feature to obtain a second map element feature; According to the depth prediction network and the map element semantic features, performing map element position feature extraction processing on the image features to obtain map element position embedding features; The first map element feature, the second map element feature, and the map element position embedding feature are concatenated to obtain a map element query feature.

3. The method according to claim 2, characterized in that The step of performing map element position feature extraction processing on the image feature according to the depth prediction network and the map element semantic feature to obtain a map element position embedding feature includes: According to the depth prediction network and the image feature, obtaining the depth image feature; According to the semantic features of the map elements, the depth image features are sampled to obtain the depth image features of the map elements; Perform position embedding processing on the map element depth image feature to obtain a map element position embedding feature.

4. The method according to claim 2, characterized in that: The determining the first map element feature according to the map element semantic feature includes: Performing dimensionality-upgrading processing on the semantic features of the map elements; According to the result of the dimension-raising process, a first map element feature is obtained.

5. The method according to claim 2, characterized in that: The step of performing map element image feature extraction processing on the image feature according to the map element semantic feature to obtain a second map element feature includes: According to the semantic features of the map elements, sampling processing is performed on the image features; Obtaining map element image features according to the results of the sampling process; Performing dimensionality-upgrading processing on the map element image features; According to the result of the dimension increasing process, a second map element feature is obtained.

6. The method according to claim 1, characterized in that The preset map element detection model includes an attention mechanism network, and the updating process of the map element query feature to obtain the target map element query feature includes: According to the attention mechanism network, updating the map element query feature; Based on the result of the updating process, a target map element query feature is obtained.

7. The method according to claim 6, characterized in that The attention mechanism network includes a self-attention layer, a first cross-attention layer, and a second cross-attention layer. The updating process of the map element query feature according to the attention mechanism network includes: According to the self-attention layer, the map element query feature is subjected to category fusion processing to obtain a map element category query feature; According to the first cross attention layer, performing category interaction processing on the map element category query feature to obtain a map element category interaction query feature; Get the map element query feature queue of the previous frame of the current frame; According to the second cross attention layer and the map element query feature queue of the previous frame of the current frame, the map element category interactive query feature is subjected to inter-frame interactive processing to obtain the result of the inter-frame interactive processing; The result of the inter-frame interaction process is used as the result of the update process.

8. The method according to any one of claims 1 to 7, characterized in that The preset map element detection model also includes a classification network and a regression network. The output result includes map element category information and location information corresponding to the map element category information. The classification and regression processing is performed on the target map element query feature to obtain the output result of the preset map element detection model, including: Inputting the target map element query feature into the classification network to output the map element category information; The target map element query feature is input into the regression network to output the location information corresponding to the map element category information.

9. A method for model training, characterized in that: The method comprises: Get a sample surround image; Inputting the sample surround image into a map element detection model to be trained to perform feature extraction processing on the sample surround image to obtain image features; Performing map element detection processing on the image features to obtain map element query features; performing update processing on the map element query feature to obtain a target map element query feature; Performing classification and regression processing on the target map element query features to obtain an output result of the map element detection model to be trained; The map element detection model to be trained is iteratively trained according to the output result and the sample surround image to obtain a trained map element detection model.

10. A device for generating a high-precision map, characterized in that: The device comprises: A first acquisition unit, used to acquire a surround view image to be processed; A first extraction unit, used for inputting the surround view image into a preset map element detection model to perform feature extraction processing on the surround view image to obtain image features; A first detection unit, configured to perform map element detection processing on the image feature to obtain a map element query feature; A first updating unit, configured to update the map element query feature to obtain a target map element query feature; The first output unit is used to perform classification and regression processing on the target map element query feature to obtain the output result of the preset map element detection model, so as to generate the high-precision map according to the output result.

11. A model training device, characterized in that: The device comprises: A second acquisition unit is used to acquire a sample surround image; A second extraction unit is used to input the sample surround image into a map element detection model to be trained, so as to perform feature extraction processing on the sample surround image to obtain image features; A second detection unit is used to perform map element detection processing on the image feature to obtain a map element query feature; A second updating unit, configured to update the map element query feature to obtain a target map element query feature; A second output unit, used for performing classification and regression processing on the target map element query feature to obtain an output result of the map element detection model to be trained; The first training unit is used to iteratively train the map element detection model to be trained according to the output result and the sample surround image to obtain a trained map element detection model.

12. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-9.

14. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images