A method and system for real-time updating of high-precision maps based on roadside camera images

Abstract

The present invention relates to a method and system for real-time updating of high-precision maps based on road-side camera images. The present invention includes obtaining an initial image captured by a road-side camera, and determining the position and attitude parameters of the road-side camera in three-dimensional space based on the initial image; obtaining a real-time image captured by the road-side camera, and the real-time image includes the environment and map information of the current road section; based on the position and attitude parameters, corresponding a reference point in three-dimensional space to the reference pixel coordinates of the real-time image to obtain a position correspondence between the three-dimensional space and the image; performing feature extraction on the real-time image to obtain road-side feature representation information; generating fused feature information based on the road-side feature representation information and the feature representation information of the initial high-precision map; and updating the coordinate points of the initial high-precision map based on a query vector and the fused feature information to achieve real-time updating of the high-precision map. The present invention overcomes the problems of poor timeliness and high labor annotation costs in the high-precision map update method in the prior art.

Description

Technical Field

[0001] The present invention relates to the fields of high-precision maps and autonomous driving technologies, and in particular to a method and system for real-time updating of high-precision maps based on roadside camera images. Background Art

[0002] As a key component of autonomous driving systems, HD maps provide precise geographic information for vehicle positioning, navigation, and decision-making. Compared to standard maps, HD maps offer more detailed information on lanes, traffic signs, and the location of infrastructure such as overpasses.

[0003] Existing HD map acquisition methods primarily rely on professional mapping vehicles collecting data while driving on the road, followed by manual annotation. While this method produces high-quality, high-precision map data with relatively comprehensive information, the fact that all information must be collected on-site results in a long update cycle. Consequently, existing HD maps cannot be updated in real time for dynamic road conditions such as temporary lane closures, construction areas, and newly installed traffic signs, resulting in poor timeliness. Furthermore, the manual annotation process incurs significant time and labor costs. Summary of the Invention

[0004] To this end, the technical problem to be solved by the present invention is to overcome the problems of poor timeliness and high labor annotation costs existing in the high-precision map updating method in the prior art.

[0005] To solve the above technical problems, the present invention provides a method for real-time updating of high-precision maps based on roadside camera images, comprising:

[0006] Obtaining an initial image captured by a roadside camera, and determining a position and attitude parameters of the roadside camera in three-dimensional space based on the initial image;

[0007] Acquire real-time images captured by roadside cameras, the real-time images including the environment and map information of the current road section;

[0008] Based on the position and posture parameters, a reference point in the three-dimensional space is mapped to a reference pixel coordinate of the real-time image to obtain a position correspondence between the three-dimensional space and the image;

[0009] Based on the position correspondence between the three-dimensional space and the image, feature extraction is performed on the real-time image to obtain road end feature representation information;

[0010] generating fused feature information based on the road-end feature representation information and the feature representation information of the initial high-precision map;

[0011] Based on the query vector and the fused feature information, the coordinate points of the initial high-precision map are updated to achieve real-time updating of the high-precision map; wherein, the map includes a map query vector and a new element query vector, the map query vector is a vector generated by using the coordinate points of the initial high-precision map as prior coordinates and encoding through a multi-layer perceptron, and the new element query vector is a vector updated as the high-precision map real-time update network is updated in real time.

[0012] In one embodiment of the present invention, determining the position and attitude parameters of the roadside camera in three-dimensional space based on the initial image includes:

[0013] Based on the camera parameter prediction network, the image information of the initial image is input into the camera parameter prediction network to obtain the camera's extrinsic parameter matrix; wherein the extrinsic parameter matrix includes a translation vector (position parameter) and a rotation matrix (attitude parameter), which are used to convert the coordinates in the world coordinate system into the coordinates in the camera coordinate system.

[0014] In one embodiment of the present invention, the extrinsic parameter matrix is ​​used to convert the coordinates in the world coordinate system into the coordinates in the camera coordinate system, and is obtained by the following matrix transformation formula:

[0015]

[0016] Where K is the camera's intrinsic parameter matrix, including focal length, principal point (optical center) coordinates, distortion coefficient, etc.

[0017] R is the rotation matrix in the external parameter matrix;

[0018] T is the translation vector in the extrinsic matrix;

[0019] P c is the coordinate of the camera coordinate system;

[0020] P w The coordinates are in the world coordinate system.

[0021] In one embodiment of the present invention, extracting features from the real-time image to obtain roadside feature representation information includes:

[0022] Extracting image features of the real-time image based on a convolutional neural network;

[0023] Based on the position and posture parameters, mapping a reference point in the three-dimensional space to a reference pixel coordinate of the real-time image;

[0024] The image features are used as the input of the Transformer neural network model, and the cross-attention mechanism is used to fuse the reference pixel coordinates corresponding to the real-time image and the image features of the surrounding positions to obtain road end feature representation information.

[0025] In one embodiment of the present invention, generating fused feature information based on the road end feature representation information and the feature representation information of the initial high-precision map includes:

[0026] splicing the road end feature representation information and the feature representation information of the initial high-precision map to form a fused feature vector;

[0027] The fused feature vector is input into a multi-layer perceptron, and the high-dimensional fused feature vector is mapped to a lower dimension by the multi-layer perceptron to obtain the fused feature information.

[0028] In one embodiment of the present invention, updating the coordinate points of the initial high-precision map based on the query vector and the fused feature information to achieve real-time updating of the high-precision map includes:

[0029] Inputting the query vector and the fused feature information into a prediction network based on a Transformer decoder, predicting and outputting a coordinate change value, a coordinate change type, new element coordinates, and a confidence level corresponding to the coordinate change type; wherein the new element coordinates are predicted based on the new element query vector; and the coordinate change types include addition, modification, and deletion;

[0030] Based on the coordinate change value, the new element coordinates and the coordinate change type, determine whether the confidence corresponding to the coordinate change type is greater than a preset threshold. If so, update the prior coordinates of the coordinate point of the initial high-precision map and / or add the new element coordinates to obtain the updated target point coordinates; for each query vector, generate an updated high-precision map element framework line vector, and obtain the updated high-precision map element framework line based on the high-precision map element framework line vector.

[0031] In one embodiment of the present invention, the Transformer decoder-based prediction network performs enhancement processing on the initial high-precision map data during training, including:

[0032] Adding random coordinate noise, including modifying the coordinate values ​​in an existing HD map or offsetting the positions of coordinate points on an existing HD map;

[0033] Delete some map elements, including deleting some frame lines in existing high-precision maps.

[0034] The present invention also provides a high-precision map real-time update system based on roadside camera images, comprising:

[0035] A position and attitude parameter acquisition module is used to obtain an initial image captured by a roadside camera and determine the position and attitude parameters of the roadside camera in three-dimensional space based on the initial image;

[0036] A real-time image acquisition module is used to acquire real-time images collected by roadside cameras, wherein the real-time images include the environment and map information of the current road section;

[0037] A spatial position correspondence relationship acquisition module is used to correspond the reference point in the three-dimensional space to the reference pixel coordinates of the real-time image based on the position and posture parameters to obtain the position correspondence relationship between the three-dimensional space and the image;

[0038] A road end feature representation information acquisition module is used to extract features from the real-time image to obtain road end feature representation information;

[0039] a fusion feature information acquisition module, configured to generate fusion feature information based on the road-end feature representation information and the feature representation information of the initial high-precision map;

[0040] A high-precision map real-time update module is used to update the coordinate points of the initial high-precision map based on the query vector and the fused feature information to achieve real-time update of the high-precision map; wherein the map includes a map query vector and a new element query vector, the map query vector is a vector generated by using the coordinate points of the initial high-precision map as prior coordinates and encoding through a multi-layer perceptron, and the new element query vector is a vector updated as the high-precision map is updated in real time.

[0041] The above technical solution of the present invention has the following advantages over the prior art:

[0042] The present invention describes a real-time HD map update method and system based on roadside camera images. This method uses real-time roadside camera data to add, delete, and modify existing HD map elements, thereby generating a more accurate real-time HD map. By using computer vision methods to update HD maps in real time using data collected by roadside cameras, the system enhances the map's timeliness while reducing the need for manual data collection and annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.

[0044] Figure 1 This is a flow chart of the method for real-time updating of high-precision maps based on road-side camera images of the present invention.

[0045] Figure 2 It is a schematic diagram of the present invention predicting the camera's external parameters based on pictures taken by the roadside camera.

[0046] Figure 3 It is a schematic diagram of the present invention performing feature extraction on real-time images to obtain road-end feature representation information.

[0047] Figure 4 It is a schematic diagram of the present invention realizing real-time update of high-precision maps.

[0048] Figure 5 This is a schematic diagram of high-precision map training and reasoning based on the high-precision map real-time update method based on road-side camera images of the present invention. DETAILED DESCRIPTION

[0049] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0050] In the present invention, "several" means one or more, "multiple" means more than two, "greater than," "less than," "exceeds," etc. are understood to exclude the number itself; "above," "below," "within," etc. are understood to include the number itself. In the description of the present invention, the use of "first" or "second" is solely for the purpose of distinguishing technical features and is not to be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0051] Example 1

[0052] Reference Figure 1 As shown, a real-time update method for high-precision maps based on roadside camera images in this embodiment includes:

[0053] S1. Obtain an initial image captured by a roadside camera, and determine the position and attitude parameters of the roadside camera in three-dimensional space based on the initial image. Figure 2 As shown, specifically including:

[0054] Use a roadside camera to take one or a group of pictures as the initial image;

[0055] Based on a camera parameter prediction network, the image information of the initial image is input into the camera parameter prediction network to obtain an extrinsic parameter matrix of the camera.

[0056] The external parameter matrix includes the translation vector (position parameter) and the rotation matrix (attitude parameter), which is used to convert the coordinates in the world coordinate system into the coordinates in the camera coordinate system. It is obtained by the following matrix transformation formula:

[0057]

[0058] Where K is the camera's intrinsic parameter matrix, including focal length, principal point (optical center) coordinates, distortion coefficient, etc.

[0059] R is the rotation matrix in the external parameter matrix;

[0060] T is the translation vector in the extrinsic matrix;

[0061] P c is the coordinate of the camera coordinate system;

[0062] P w The coordinates are in the world coordinate system.

[0063] Through this process, the camera's external parameters are predicted based on the images taken by the road-side camera, and the position information of the road-side camera can be accurately obtained, which provides a basis for subsequent three-dimensional reconstruction of images and updating of high-precision maps.

[0064] As part of intelligent transportation infrastructure, roadside cameras can capture real-time image data of the road environment, offering high timeliness and wide coverage. By processing this data, portions of high-precision maps can be updated in real time, significantly improving map dynamics and accuracy. Automated processing and updating using roadside cameras also reduces significant labor and time costs.

[0065] S2. Acquire a real-time image captured by a roadside camera. The real-time image includes the environment and map information of the current road section.

[0066] S3. Based on the position and posture parameters, correspond the reference point in the three-dimensional space to the reference pixel coordinates of the real-time image to obtain a position correspondence between the three-dimensional space and the image.

[0067] Specifically, at each time point t, the real-time image P collected by the roadside camera is r , P r ={p t1 ,p t2 ,…,p tn}, input the real-time image into the camera parameter prediction network to predict the camera's posture; according to the aforementioned coordinate transformation formula, correspond the reference point in the three-dimensional space to the reference pixel coordinates of the real-time image.

[0068] The camera parameter prediction network can adopt the existing convolutional neural network (CNN) or other architectures to extract image features, identify key points and visual clues in the image, and then infer the camera's pose.

[0069] S4, based on the position correspondence between the three-dimensional space and the image, extract the features of the real-time image to obtain the road end feature representation information. Figure 3 shown.

[0070] In this embodiment, a convolutional neural network or a neural network with other architectures is used to extract features of real-time images, and then these features are input into a feature information extraction network. A neural network based on the Transformer architecture is used to process the feature information of multiple images through self-attention mechanism and cross-attention mechanism. Specifically:

[0071] Based on a convolutional neural network, image features of the real-time image are extracted; a multi-dimensional feature vector is generated, and the convolutional neural network slides in the image through its convolution kernel to extract feature information with local correlation, such as edges, textures, shapes, etc.

[0072] The main workflow of a convolutional neural network is as follows:

[0073] Convolution operation: Extract local information of the image through the convolution kernel. The convolution kernel is a small window that slides across the image and performs a weighted sum of local pixel values ​​to generate a feature map.

[0074] Activation function: ReLU activation function is usually used to increase the nonlinearity of the network to capture complex features.

[0075] Pooling: The pooling layer is used to downsample feature maps, reducing dimensionality and improving computational efficiency. A common pooling method is max pooling.

[0076] Generating multi-dimensional feature vectors: After multiple layers of convolution and pooling, CNNs ultimately generate a feature map containing multiple channels, each representing different feature information. These feature maps are flattened and output as multi-dimensional feature vectors, containing local information such as edges, textures, and shapes of the image.

[0077] Based on the position and posture parameters, a reference point in the three-dimensional space is mapped to a reference pixel coordinate of the real-time image.

[0078] The image features are used as the input of the Transformer neural network model. The Transformer model fuses the reference pixel coordinates corresponding to multiple images and the image features of the surrounding positions through the self-attention mechanism and the cross-attention mechanism to obtain the road end feature representation information F r , the specific form is a multi-dimensional feature vector. The specific process is as follows:

[0079] Input multidimensional feature vector: The multidimensional feature vector generated by the convolutional neural network is used as the input of the Transformer.

[0080] For each image, the Transformer processes its feature vector through a self-attention mechanism, ensuring that each feature point can interact with other feature points. This allows the integration of global information within the image.

[0081] To process data from different images, the Transformer's cross-attention mechanism integrates image features from different perspectives or time points. For example, different roadside cameras may capture the same scene from multiple angles. The cross-attention mechanism can combine the feature information of these different images to generate a more accurate roadside feature representation.

[0082] After processing self-attention and cross-attention, the Transformer outputs a feature vector that integrates global and cross-image relationships, obtaining road-end feature representation information. This is a multi-dimensional vector that covers key information such as the spatial layout, object shape, and positional relationships in the scene.

[0083] S5. Generate fusion feature information based on the road-end feature representation information and the feature representation information of the initial high-precision map (optional). Specifically:

[0084] The road end feature representation information F and the feature representation information F of the initial high-precision map are combined m Splice to form a fusion feature vector [F r ,F m ];

[0085] The fusion feature vector [F r ,F m ] is input into the multi-layer perceptron (MLP), which maps the high-dimensional fusion features to a lower dimension to obtain the fusion feature information F fuse Among them, F fuse =MLP([F r ,F m ]).

[0086] A multilayer perceptron (MLP) is a fully connected neural network composed of multiple linear layers. It is commonly used to reduce the dimensionality of high-dimensional data. Through the MLP, the system can extract more refined feature information for subsequent map updates. High-precision maps are composed of individual points and the framework lines that connect them, containing geographic information such as roads and buildings.

[0087] The working process of MLP is roughly as follows:

[0088] Input layer: Input the concatenated fusion feature vector [F r ,F m ];

[0089] Hidden layer: MLP maps high-dimensional feature vectors to lower dimensions through a series of linear transformations and nonlinear activation functions. Activation functions include ReLU, Sigmoid, etc.

[0090] Output layer: output fusion feature information F after dimensionality reduction fuse .

[0091] S6, based on the query vector and the fusion feature information generated in step S5, update the coordinate points of the initial high-precision map to achieve real-time update of the high-precision map; Figure 4 As shown, the process operates using a prediction network based on the Transformer decoder.

[0092] Wherein, the map includes a map query vector e m and the new element query vector e n , the map query vector e m The coordinate points of the initial high-precision map are used as the prior coordinates C m , and the vector generated by the multilayer perceptron encoding, and the new element query vector e n The query vector is a high-dimensional vector that is updated in real time with the HD map. The update of the new element query vector is dynamic and is constantly adjusted as the HD map is updated. The two are spliced ​​together to form the input query vector e in =[e m ,e n ].

[0093] The query vector e m and fusion feature information F fuse Input into the Transformer decoder to obtain the updated query vector, and then predict the coordinate change value C delta , coordinate change type T, new element coordinates C n and the corresponding confidence level P. There are three types of coordinate changes: addition (adding new map elements), modification (adjusting the coordinates of existing elements), and deletion (removing elements that no longer exist).

[0094] According to the coordinate change value C delta , coordinate change type T, new element coordinates C n And the corresponding confidence level P, to determine whether the confidence level is greater than the preset threshold (θ). If the confidence level meets the requirements, the coordinate points in the high-precision map are updated to generate a new target coordinate set C u This step also includes the prediction and insertion of new elements.

[0095] As you can understand, each coordinate change is associated with a confidence value, indicating the credibility of the change. An update is only performed when the confidence exceeds a preset threshold θ, ensuring the accuracy of the map update.

[0096] The details are explained below.

[0097] In step S6, based on the query vector e m and the fusion feature information e n , update the coordinate points of the initial high-precision map to achieve real-time update of the high-precision map, including:

[0098] The query vector e m and the fusion feature information e n Input to the prediction network based on Transformer decoder, predict and output the coordinate change value C delta , coordinate change type T, new element coordinates C n and a confidence level P corresponding to a coordinate change type T; wherein the new element coordinates are predicted based on the new element query vector; the coordinate change type T includes addition, modification, and deletion;

[0099] Based on the coordinate change value C delta , coordinate change type T, new element coordinates C n And the corresponding confidence P, update the coordinate points of the initial high-precision map.

[0100] It should be noted that the high-precision map is composed of points and the frame lines that connect the points; the coordinate points of the initial high-precision map (that is, the points that make up the high-precision map) are used as the prior coordinates C m ={(x1,y1),(x2,y2),…,(x n ,y n )};

[0101] Concatenate the prior coordinates of each frame line of the initial high-precision map into [x1′,y1′,x2′,y2′,…,x n ′,y n ′] is input into the multilayer perceptron for encoding to obtain the map query vector e m ={e m1 ,e m2 ,…,e mn In order to consider the newly emerged elements, a high-dimensional new element query vector e is introduced which can be continuously updated with training. n ={e n1 ,e n2 ,…,e nN}, the two are concatenated to get the input e in =[e m ,en ]. in and F fuse As the input of the prediction network based on the Transformer decoder, the coordinate change value C is predicted delta ={(Δx1,Δy1),(Δx2,Δy2),…,(Δx n ,Δy n )}, and the coordinate change type T of each coordinate is T = {t1, t2, ..., t n} and confidence P=={p1,p2,…,p n}.

[0102] The coordinate change value C is obtained delta , coordinate change type T, new element coordinates C n And the corresponding confidence level P, to update the coordinate points of the initial high-precision map, including:

[0103] According to the coordinate change value C delta , the new element coordinate C n and the coordinate change type T, determine whether the confidence P corresponding to the coordinate change type T is greater than a preset threshold θ, and if so, update the prior coordinates C of the coordinate point of the initial high-precision map m And / or add the new element coordinate C n , get the updated target point coordinates C in ;

[0104] Specifically, for the prior coordinate C m ={(x1,y1),(x2,y2),…,(x n ,y n )}, update or delete according to the change type, and obtain the updated target point coordinate set C u ={(x1+Δx1,y1+Δy1),(x2+Δx2,y2+Δy2),…,(x n +Δx n ,y n +Δy n )}. For new element queries, predict the new element coordinates C n ={(x1′,y1′),(x2′,y2′),…,(x n ′,y n ′)}. Finally, only the coordinate points with confidence greater than the threshold θ are retained. According to this method, an updated high-precision map element frame line vector is generated for each query vector, specifically in the form of [x′1,y1′,x2′,y2′,…,x n ′,y n′], and the coordinates of the points on this frame line are obtained by using this vector as (x1′,y1′),(x2′,y2′),…,(x n ′,y n ′), draw the frame line to achieve real-time update of the map.

[0105] By selectively accepting additions, deletions, and modifications based on confidence, we ensure that the quality of the updated map meets actual needs.

[0106] The above method uses image input from different angles of road-side cameras to automatically add, delete or modify elements of existing high-precision maps to generate real-time updated high-precision maps.

[0107] In addition, to improve the model's robustness in new environments and with different data patterns, the Transformer decoder-based prediction network performs enhanced processing on the initial HD map data during training, including the following:

[0108] Add random coordinate noise This includes modifying the coordinate values ​​in an existing high-precision map or offsetting the positions of coordinate points on an existing high-precision map;

[0109] Deleting some map elements, including removing some framework lines from existing HD maps. In HD maps, map elements typically consist of coordinate points and the framework lines connecting them. Deleting some framework lines simulates incomplete maps or missing data in the real world.

[0110] The purpose of adding random coordinate noise and deleting some map elements is to use the existing high-precision map to modify elements to construct an "initial map", using the unenhanced map as the true value to train the model's coordinate updating and generation capabilities.

[0111] In summary, refer to Figure 5 As shown in the figure, this real-time update method for high-precision maps based on roadside camera images constructs an "initial map" through data enhancement (such as adding coordinate noise, removing some map elements, etc.), uses the unenhanced map as the true value, trains the model's coordinate update and generation capabilities, and selectively accepts additions, deletions, and modifications based on confidence to ensure that the quality of the updated map meets actual needs.

[0112] Example 2

[0113] Based on the same inventive concept, this embodiment provides a high-precision map real-time update system based on road-side camera images. The principle of solving the problem is similar to the high-precision map real-time update method based on road-side camera images, and the repeated parts will not be repeated.

[0114] This embodiment provides a real-time update system for high-precision maps based on roadside camera images, including:

[0115] A position and attitude parameter acquisition module is used to obtain an initial image captured by a roadside camera and determine the position and attitude parameters of the roadside camera in three-dimensional space based on the initial image;

[0116] A real-time image acquisition module is used to acquire real-time images collected by roadside cameras, wherein the real-time images include the environment and map information of the current road section;

[0117] A spatial position correspondence relationship acquisition module is used to convert the real-time image into corresponding three-dimensional coordinates in the three-dimensional space based on the position and posture parameters, and obtain a position correspondence relationship between the three-dimensional space and the image;

[0118] A road end feature representation information acquisition module is used to extract features from the real-time image to obtain road end feature representation information;

[0119] a fusion feature information acquisition module, configured to generate fusion feature information based on the road-end feature representation information and the feature representation information of the initial high-precision map;

[0120] A high-precision map real-time update module is used to update the coordinate points of the initial high-precision map based on the query vector and the fused feature information to achieve real-time update of the high-precision map; wherein the map includes a map query vector and a new element query vector, the map query vector is a vector generated by using the coordinate points of the initial high-precision map as prior coordinates and encoding through a multi-layer perceptron, and the new element query vector is a vector updated as the high-precision map is updated in real time.

[0121] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0122] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0123] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0125] Finally, it should be noted that the above specific implementation methods are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to examples, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for real-time updating of high-precision maps based on roadside camera images, characterized in that: include: Obtaining an initial image captured by a roadside camera, and determining a position and attitude parameters of the roadside camera in three-dimensional space based on the initial image; Acquire real-time images captured by roadside cameras, the real-time images including the environment and map information of the current road section; Based on the position and posture parameters, a reference point in the three-dimensional space is mapped to a reference pixel coordinate of the real-time image to obtain a position correspondence between the three-dimensional space and the image; Based on the position correspondence between the three-dimensional space and the image, feature extraction is performed on the real-time image to obtain road end feature representation information; generating fused feature information based on the road-end feature representation information and the feature representation information of the initial high-precision map; Based on the query vector and the fused feature information, the coordinate points of the initial high-precision map are updated to achieve real-time updating of the high-precision map; wherein the map includes a map query vector and a new element query vector, the map query vector is a vector generated by using the coordinate points of the initial high-precision map as prior coordinates and encoding through a multi-layer perceptron, and the new element query vector is a vector updated in real time as the high-precision map real-time update network is updated; The updating of the coordinate points of the initial high-precision map based on the query vector and the fused feature information to achieve real-time updating of the high-precision map includes: Inputting the query vector and the fused feature information into a prediction network based on a Transformer decoder, predicting and outputting a coordinate change value, a coordinate change type, new element coordinates, and a confidence level corresponding to the coordinate change type; wherein the new element coordinates are predicted based on the new element query vector; and the coordinate change types include addition, modification, and deletion; Based on the coordinate change value, the new element coordinates and the coordinate change type, determine whether the confidence corresponding to the coordinate change type is greater than a preset threshold. If so, update the prior coordinates of the coordinate point of the initial high-precision map and / or add the new element coordinates to obtain the updated target point coordinates; for each query vector, generate an updated high-precision map element framework line vector, and obtain the updated high-precision map element framework line based on the high-precision map element framework line vector.

2. The method for real-time updating of high-precision maps based on roadside camera images according to claim 1, characterized in that: The determining, based on the initial image, the position and attitude parameters of the roadside camera in three-dimensional space includes: Based on the camera parameter prediction network, the image information of the initial image is input into the camera parameter prediction network to obtain the camera's extrinsic parameter matrix; wherein the extrinsic parameter matrix includes a translation vector, i.e., a position parameter, and a rotation matrix, i.e., a posture parameter, which is used to convert the coordinates in the world coordinate system into the coordinates in the camera coordinate system.

3. The method for real-time updating of high-precision maps based on roadside camera images according to claim 2, characterized in that: The external parameter matrix is ​​used to convert the coordinates in the world coordinate system into the coordinates in the camera coordinate system, which is obtained according to the following matrix formula: , Where K is the camera's intrinsic parameter matrix, including the focal length, the coordinates of the principal point (optical center), and the distortion coefficient; R is the rotation matrix in the external parameter matrix; T is the translation vector in the extrinsic matrix; is the coordinate of the camera coordinate system; The coordinates are in the world coordinate system.

4. The method for real-time updating of high-precision maps based on roadside camera images according to claim 2, characterized in that: The step of mapping the reference point in the three-dimensional space to the reference pixel coordinates of the real-time image based on the position and posture parameters to obtain a position correspondence between the three-dimensional space and the image includes: At each time point, inputting the real-time image into the camera parameter prediction network; Based on the extrinsic parameter matrix, the reference points in the three-dimensional space are mapped to the reference pixel coordinates of the real-time image to obtain a positional correspondence between the three-dimensional space and the image.

5. The method for real-time updating of high-precision maps based on roadside camera images according to claim 1, characterized in that: The extracting features from the real-time image to obtain road-end feature representation information includes: Extracting image features of the real-time image based on a convolutional neural network; Based on the position and posture parameters, mapping a reference point in the three-dimensional space to a reference pixel coordinate of the real-time image; The image features are used as the input of the Transformer neural network model, and the cross-attention mechanism is used to fuse the reference pixel coordinates corresponding to the real-time image and the image features of the surrounding positions to obtain road end feature representation information.

6. The method for real-time updating of high-precision maps based on roadside camera images according to claim 1, characterized in that: The generating of fused feature information based on the road-end feature representation information and the feature representation information of the initial high-precision map includes: splicing the road end feature representation information and the feature representation information of the initial high-precision map to form a fused feature vector; The fused feature vector is input into a multi-layer perceptron, and the high-dimensional fused feature vector is mapped to a lower dimension by the multi-layer perceptron to obtain the fused feature information.

7. The method for real-time updating of high-precision maps based on roadside camera images according to claim 1, characterized in that: During training, the Transformer decoder-based prediction network performs enhancement processing on the initial high-precision map data, including: Adding random coordinate noise, including modifying the coordinate values ​​in an existing HD map or offsetting the positions of coordinate points on an existing HD map; Delete some map elements, including deleting some frame lines in existing high-precision maps.

8. A high-precision map real-time update system based on roadside camera images, characterized in that: include: A position and attitude parameter acquisition module is used to obtain an initial image captured by a roadside camera and determine the position and attitude parameters of the roadside camera in three-dimensional space based on the initial image; A real-time image acquisition module is used to acquire real-time images collected by roadside cameras, wherein the real-time images include the environment and map information of the current road section; A spatial position correspondence relationship acquisition module is used to correspond the reference point in the three-dimensional space to the reference pixel coordinates of the real-time image based on the position and posture parameters to obtain the position correspondence relationship between the three-dimensional space and the image; A road end feature representation information acquisition module is used to extract features from the real-time image to obtain road end feature representation information; a fusion feature information acquisition module, configured to generate fusion feature information based on the road-end feature representation information and the feature representation information of the initial high-precision map; A high-precision map real-time update module, configured to update the coordinate points of the initial high-precision map based on the query vector and the fused feature information, thereby achieving real-time updates of the high-precision map; wherein the map includes a map query vector and a new element query vector, wherein the map query vector is a vector generated by encoding the coordinate points of the initial high-precision map as prior coordinates through a multi-layer perceptron, and the new element query vector is a vector updated as the high-precision map real-time update network is updated in real time; The updating of the coordinate points of the initial high-precision map based on the query vector and the fused feature information to achieve real-time updating of the high-precision map includes: Inputting the query vector and the fused feature information into a prediction network based on a Transformer decoder, predicting and outputting a coordinate change value, a coordinate change type, new element coordinates, and a confidence level corresponding to the coordinate change type; wherein the new element coordinates are predicted based on the new element query vector; and the coordinate change types include addition, modification, and deletion; Based on the coordinate change value, the new element coordinates and the coordinate change type, determine whether the confidence corresponding to the coordinate change type is greater than a preset threshold. If so, update the prior coordinates of the coordinate point of the initial high-precision map and / or add the new element coordinates to obtain the updated target point coordinates; for each query vector, generate an updated high-precision map element framework line vector, and obtain the updated high-precision map element framework line based on the high-precision map element framework line vector.

Images