Vehicle online mapping method, system, storage medium and vehicle

Abstract

The application discloses a kind of vehicle online mapping methods, it includes the steps: obtaining the look-around camera data of vehicle and the navigation map of the environment where vehicle is located;The look-around camera data and the navigation map are input into the mapping model that is trained in advance, obtain the semantic category and node position of map semantic element;According to the semantic category and node position of map semantic element obtained by different vehicles, multi-journey road network topological relation processing is carried out, and the vector map of the lane-level road network of the environment is constructed in real time.The application also discloses corresponding system, storage medium and vehicle.Implementation of the present application can improve the ability of autonomous parking of vehicle in passable area, and has the advantages of low cost, high efficiency and strong scalability.

Description

Technical Field

[0001] This invention relates to the technical field of parking, specifically to a method, system, storage medium, and vehicle for online vehicle mapping. Background Technology

[0002] Existing high-precision parking lot maps are typically built offline, using onboard sensors (LiDAR, cameras, inertial navigation sensors, etc.) of map-collecting vehicles to gather scene data. This data is then processed and vectorized semantically annotated using a combination of automated and manual methods. Alternatively, offline maps can be built within drivable areas using sensors (cameras or LiDAR) in intelligent driving vehicles employing Simultaneous Localization and Mapping (SLAM).

[0003] However, the existing offline mapping solutions have many shortcomings:

[0004] Data collection, processing, and labeling require significant manpower and time, and the process is cumbersome. Furthermore, offline SLAM mapping solutions require vehicles to re-enter the target area to complete autonomous parking, resulting in poor timeliness and difficulty in covering parking lot scenarios where vehicles are entering for the first time.

[0005] Meanwhile, because road information changes dynamically and offline maps have poor timeliness, they need to be updated frequently to ensure map accuracy. When expanding to large-scale, wide-area high-precision map applications, offline mapping requires significant investment in manpower, time, and maintenance costs. Summary of the Invention

[0006] The technical problem to be solved by this invention is to provide a method, system, storage medium, and vehicle for online vehicle mapping. This allows for real-time online mapping of locations such as parking lots, improves the ability of vehicles to park autonomously within passable areas, and has the advantages of low cost, high efficiency, and strong scalability.

[0007] To address the aforementioned technical problems, as one aspect of the present invention, a method for online vehicle mapping is provided, which includes at least the following steps:

[0008] Acquire surround-view camera data of the vehicle and navigation map of the vehicle's surroundings;

[0009] The surround-view camera data and the navigation map are input into a pre-trained mapping model to obtain the semantic categories and node locations of map semantic elements;

[0010] Based on the semantic categories and node locations of map semantic elements obtained from different vehicles, multi-journey road network topology association processing is performed to construct a vector map of the lane-level road network of the environment in real time.

[0011] The pre-trained mapping model is used to perform the following processing:

[0012] The surrounding camera data is subjected to spatiotemporal fusion processing to obtain a ground bird's-eye view of the vehicle's environment;

[0013] Feature extraction is performed on the ground bird's-eye view to obtain the map semantic elements of the environment, and the map semantic elements are classified and discretized to obtain the geometric structure of the map semantic elements;

[0014] Based on the geometric structure of the map semantic elements, semantic instance matching or node matching is performed. Combined with the road-level attribute information and topological relationships in the navigation map, the semantic category and node location of the map semantic elements are obtained.

[0015] The step of extracting features from the ground-level bird's-eye view to obtain map semantic elements of the surrounding environment, and then classifying and discretizing these map semantic elements to obtain their geometric structure, further includes:

[0016] The ground bird's-eye view is subjected to semantic feature detection and segmentation to extract map semantic elements, wherein the map semantic elements include at least: roads, lane lines, parking space lines, ground arrows, zebra crossings, and stop lines;

[0017] The map semantic elements are classified and discretized; zebra crossings, parking space lines, and traffic signs are discretized and represented as polygons; coarse road markings are discretized and represented as multiple line segments or a single line segment; continuous roads and lane markings are discretized and represented as multiple line segments.

[0018] The discretized map semantic elements are sampled at equal intervals, and each map semantic element is split into different nodes according to the arrangement and combination. The connection relationship and arrangement of the nodes are described according to the topological structure, thereby obtaining the geometric structure of the map semantic elements. The topological structure is the spatial relationship between line elements, which includes separation, adjacency, intersection and overlap.

[0019] The step of performing semantic instance matching or node matching based on the geometric structure of the map semantic elements, and combining road-level attribute information and topological relationships in the navigation map to obtain the semantic category and node location of the map semantic elements, further includes:

[0020] For features represented by polygons, each segment in the polygon is matched with the outline pixels of a semantic instance on the ground bird's-eye view;

[0021] For elements represented by multiple line segments or a single line segment, node matching is performed. For each map semantic element, the optimal permutation is found from the set of equivalent permutations, such that the predicted nodes correspond one-to-one with the actual nodes and the sum of the distances between point pairs is minimized.

[0022] Obtain road-level precision attribute information and topological relationships in the navigation map. The attribute information includes road curvature, slope information, number of lane lines, and width; the topological relationships include at least: topological connectivity relationships.

[0023] The attribute information and topological relationship are unified with the image feature information obtained by feature extraction from the ground bird's-eye view through the embedding operation into a feature vector space;

[0024] Based on the matching results, constraints are imposed on the category prediction, positional relationship, and directional consistency of map semantic elements, and the semantic category and node position of map semantic elements are output.

[0025] The step of performing multi-journey road network topology association processing based on the semantic categories and node locations of map semantic elements obtained from different vehicles, and constructing and outputting a vector map of the lane-level road network of the environment in real time, further includes:

[0026] Based on the semantic categories and node positions of map semantic elements obtained from different vehicles' observations of the environment from different spatial angles and times, multiple journey trajectories are associated with the connection relationships and arrangement of nodes according to the same set of specifications of topological structure description, forming a lane-level road network, and obtaining and outputting a vector map of the environment.

[0027] The step of performing spatiotemporal fusion processing on the surround-view camera data to obtain a ground bird's-eye view of the vehicle's environment further includes:

[0028] The surround-view camera data is mapped onto a unified vehicle coordinate system and fused to form a ground bird's-eye view of the environment, with time information annotated.

[0029] This further includes:

[0030] Obtain the initial mapping model;

[0031] Establish the following loss function:

[0032] L = a*L1 + b*L2 + c*L3 + d*L4

[0033] Where a, b, c, and d are the weights that balance different loss terms, L1 is the classification loss function, L2 is the distance loss function, L3 is the orientation loss function, and L4 is the topological similarity loss function; the topological similarity loss function is used to calculate the statistical mean difference between the topological relationships contained in the predicted line group and the ground truth line group.

[0034] Input a ring view image sample into the initial mapping network model, and use a loss function to make the network output match the expected value, thereby obtaining the optimal weight parameters in the initial mapping model and thus obtaining a trained mapping model.

[0035] Accordingly, another aspect of the present invention also provides a method for online vehicle mapping, characterized by comprising the following steps:

[0036] Determine the target parking area based on the parking lot entrance on the navigation map;

[0037] After determining that the vehicle has entered the target parking area, it is checked whether there is an offline high-precision map of the drivable area. If there is no map of the drivable area, an online vector map is generated in real time using the method described above for parking planning.

[0038] If a high-precision map exists for the drivable area, an online vector map is generated in real time using the method described above. The real-time online vector map is then projected onto the same coordinate system and fused with the offline high-precision map to obtain the latest real-time vector map for use in parking planning.

[0039] Accordingly, another aspect of the present invention also provides an online vehicle mapping system, comprising:

[0040] The initial data acquisition unit is used to acquire surround-view camera data of the vehicle and navigation maps of the vehicle's surrounding environment.

[0041] The model processing unit is used to input the surround-view camera data and the navigation map into a pre-trained mapping model to obtain the semantic categories and node locations of map semantic elements;

[0042] The vector map acquisition unit performs multi-journey road network topology association processing based on the semantic category and node location of the map semantic elements obtained by different vehicles, and constructs a vector map of the lane-level road network of the environment in real time.

[0043] The model processing unit further includes:

[0044] The preprocessing module is used to perform spatiotemporal fusion processing on the surround-view camera data to obtain a ground bird's-eye view of the vehicle's environment.

[0045] The feature extraction module is used to extract features from the ground bird's-eye view, obtain the map semantic elements of the environment, and classify and discretize the map semantic elements to obtain the geometric structure of the map semantic elements.

[0046] The matching module is used to perform semantic instance matching or node matching based on the geometric structure of the map semantic elements, and combine the road-level attribute information and topological relationships in the navigation map to obtain the semantic category and node position of the map semantic elements.

[0047] Accordingly, in another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method as described above.

[0048] Accordingly, another aspect of the present invention also provides a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the method described above.

[0049] Accordingly, another aspect of the present invention provides a vehicle on which an online vehicle mapping system as described above is deployed.

[0050] Implementing the embodiments of the present invention has the following beneficial effects:

[0051] This invention provides a method, system, storage medium, and vehicle for online vehicle mapping. By acquiring vehicle surround-view fisheye camera data and combining it with low-precision road-level navigation map data, the semantic categories and node locations of map semantic elements can be output, lane-level road networks can be constructed in real time, and vector maps can be generated online and in real time by vehicles. In this invention, map semantic elements are geometrically represented by categories, and the semantic categories and node locations of map semantic elements are obtained by matching according to the categories. Furthermore, road-level attribute information and topological relationships are added to the network. The network structure is used to interact with these feature information to obtain more accurate location information and more high-precision map elements for output, thereby realizing the online and real-time generation of vector maps by vehicles.

[0052] This invention can be used in areas where maps cannot fully cover, such as parking lots. By constructing vector maps online in real time for vehicles, it more flexibly solves the problem of autonomous parking within passable areas. It optimizes autonomous parking map and positioning technology solutions, making it more flexible and eliminating the need for complicated user operations, thus improving the user experience of autonomous parking.

[0053] The online mapping method provided by this invention requires no manual processing, annotation, or maintenance. Compared with traditional offline mapping methods, it has the advantages of low cost, strong scalability, ease of use, and high efficiency, without requiring additional hardware costs. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, obtaining other drawings based on these drawings without creative effort still falls within the scope of the present invention.

[0055] Figure 1 This is a schematic diagram of the main process of an embodiment of the online vehicle mapping method provided by the present invention;

[0056] Figure 2 for Figure 1 A more detailed flowchart of step S11;

[0057] Figure 3 To adopt Figure 1 A schematic diagram illustrating the principle of the method;

[0058] Figure 4 To adopt Figure 1 A flowchart illustrating the parking process.

[0059] Figure 5 This is a schematic diagram of a structure of an embodiment of an online vehicle mapping system provided by the present invention;

[0060] Figure 6 for Figure 5 A schematic diagram of the structure of the model processing unit. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings.

[0062] like Figure 1 The diagram shown illustrates the main flow of an embodiment of an online vehicle mapping method provided by the present invention. (In conjunction with...) Figure 2 , Figure 3 As shown, in this embodiment, the online vehicle mapping method can be used in parking scenarios and also in driving scenarios. Specifically, the method includes at least the following steps:

[0063] Step S10: Obtain the vehicle's surround view camera data and the navigation map of the vehicle's surrounding environment;

[0064] In a specific example, step S10 further includes:

[0065] Acquire real-time images captured by multiple fisheye cameras mounted on the vehicle, and acquire surround-view camera data around the vehicle; in a specific example, multiple fisheye cameras can be set around the vehicle, such as 4, 6 or 8; the number of cameras is unlimited, but they are required to cover a 360-degree field of view of the vehicle body.

[0066] At the same time, in this step, a navigation map of the vehicle's current environment is obtained. It is understood that the current environment can be such as a parking lot or a road on which the vehicle is driving.

[0067] Step S11: Input the surround-view camera data and the navigation map into the pre-trained mapping model to obtain the semantic categories and node locations of the map semantic elements;

[0068] Among them, such as Figure 2 As shown, in step S11, the pre-trained mapping model is used to perform the following processing:

[0069] Step S110: Perform spatiotemporal fusion processing on the surround-view camera data to obtain a ground bird's-eye view of the vehicle's environment;

[0070] Specifically, the surround-view camera data is mapped onto a unified vehicle coordinate system and fused to form a ground-level bird's-eye view of the parking lot, with time information annotated. Specifically, the two-dimensional image data from the camera sensors is converted into a three-dimensional bird's-eye view and then incorporating a time dimension, thus unifying space and time.

[0071] Step S111: Extract features from the ground bird's-eye view to obtain the map semantic elements of the environment, and classify and discretize the map semantic elements to obtain the geometric structure of the map semantic elements;

[0072] In a specific example, step S111 further includes:

[0073] Semantic feature detection is performed using the CNN neural network in the trained neural network model, and segmentation is performed using the U-Net network in the trained neural network model to extract semantic elements of the parking lot map. These semantic elements include at least: roads, lane lines, parking space lines, ground arrows, zebra crossings, and stop lines. It is understood that the high-precision map geometric elements used by autonomous vehicles are typically composed of points, lines, and polygons. For autonomous parking scenarios, road boundary lines, lane lines, parking space lines, traffic signs, zebra crossings, and speed bumps are distinguishable and stable feature elements. The preceding semantic feature extraction method is not limited here; both CNN and U-Net can be used for semantic feature detection or semantic element extraction, and other networks can also be employed.

[0074] The map elements are categorized and discretized; zebra crossings, parking space lines, and traffic signs (such as arrows) in the parking scene are discretized and represented as polygons; coarse road markings (such as speed bumps) in the parking scene are discretized and represented as multiple line segments or a single line segment; continuous roads and lane markings (such as lane lines and road edge lines) in the parking scene are discretized and represented as multiple line segments.

[0075] The discretized map is sampled at equal intervals, and each map element is split into different nodes according to the arrangement and combination. The connection relationship and arrangement of the nodes are described according to the topological structure, so as to obtain the geometric structure of the map semantic elements. Among them, the spatial topological relationship between road line elements is the spatial relationship of separation, adjacency, intersection and overlap between line elements. These topological relationships are the spatial connection, inclusion and proximity relationships between geographic entities that are not affected by changes such as plane stretching or rotation.

[0076] Understandably, in this step, the map semantic elements of different types of parking lots are discretized into geometric representations of categories; structured modeling is used to generate topological connections between key points to improve the accuracy of the description of the topological relationships between key points of road lane lines in the parking lot environment and to output more high-precision map elements.

[0077] Step S112: Perform semantic instance matching or node matching based on the geometric structure of the map semantic elements, and combine the road-level attribute information and topological relationships in the navigation map to obtain the semantic category and node location of the map semantic elements.

[0078] In a specific example, step S112 further includes:

[0079] Different matching mechanisms are used for label assignment based on different types of map semantic features, including semantic instance matching and node matching. Specifically, for features represented by polygons, each segment (or edge) of the polygon is matched with the outline pixels of the semantic instance on the 3D bird's-eye view; this is semantic instance matching. For features represented by multiple line segments or a single line segment, node matching is performed. For each map semantic feature, the optimal permutation is found from the equivalent permutation set, such that the predicted nodes correspond one-to-one with the actual nodes and the sum of the distances between point pairs is minimized. The equivalent permutation set contains all equivalent connection relationships between nodes and is used to describe the topological structure of the map features.

[0080] Obtain attribute information and topological relationships of roads at the level of precision in the vehicle navigation map. The attribute information includes the curvature, slope information, number of lane lines, and width of the roads. The topological relationships include at least the topological connectivity relationships.

[0081] Embedding operations unify the attribute information and topological relationships with the feature information obtained from feature extraction of the ground bird's-eye view into the same feature vector space, thus achieving the fusion of geometric elements and attribute information features. This fusion is then input into the network structure for decoding and output. Specifically, embedding operations can map a high-dimensional vector to a low-dimensional space, achieving dimensionality reduction and feature fusion. In image processing, embedding operations can be implemented by adding a fully connected layer after the convolutional layer. During this process, auxiliary information such as attributes and topological information can be introduced and embedded into the same feature vector space as the visual information, thereby achieving the organic fusion of visual features and attribute information. In this way, the entire processing of visual and non-visual information is maintained within the same feature vector space, achieving full information fusion and balanced processing.

[0082] Based on the matching results, constraints are imposed on the category prediction, positional relationship, and directional consistency of map semantic elements, and the semantic category and node position of map semantic elements are output.

[0083] Understandably, in this step, road-level attribute information is added to the network structure, and the network structure is used to interact with this feature information to obtain more accurate location information and more high-precision map element output, rather than being limited to real-time vectorized output of only high-precision map line elements such as lane lines and pedestrian crossings.

[0084] Step S12: Based on the semantic categories and node locations of the map semantic elements obtained by different vehicles, perform multi-journey road network topology association processing to construct a vector map of the lane-level road network of the environment in real time.

[0085] In a specific example, step S12 further includes:

[0086] Based on the semantic categories and node locations of map semantic elements obtained from different vehicles' observations of the parking lot from different spatial angles and times, multiple journey trajectories are linked together according to a common set of topological structures describing the connections and arrangements between nodes to form a lane-level road network, thus obtaining a vector map of the surrounding environment. Specifically, this includes operations such as geometric matching between maps and joint optimization of results.

[0087] Understandably, the final vector map of the parking lot's lane-level road network can include information such as parking space layout, parking lot structure, traffic information, safety facilities, and other parking facilities. Parking space layout can specifically include parking methods (horizontal parking, perpendicular parking, angled parking, etc.), space numbers, and the number of spaces. Parking lot structure refers to the internal structure and functional areas of the parking lot, such as entrance and exit lanes, payment areas, passageways, and entrance / exit lanes. Traffic information includes routes for entering and exiting lanes, speed limit signs, and traffic lights. Safety facilities include surveillance cameras, alarms, and signs. Other parking facilities can include repair shops, gas stations, car washes, and restrooms. This information in the vector map helps in-vehicle navigation systems understand the parking lot's internal structure, providing accurate navigation and guidance, making it easier for drivers to find available parking spaces and complete parking operations.

[0088] Meanwhile, it is understood that in the method provided by this invention, a neural network needs to be pre-established and trained to form a trained neural network model. The steps include:

[0089] Establish a neural network model that includes CNN neural networks and U-Net networks;

[0090] The loss function is constructed as follows, consisting of four parts, with weights assigned to each part:

[0091] L = a*L1 + b*L2 + c*L3 + d*L4

[0092] Where a, b, c, and d are the weights that balance different loss terms, L1 is the classification loss function, L2 is the distance loss function, L3 is the orientation loss function, and L4 is the topological similarity loss function.

[0093] The loss function measures the degree of deviation between the model's predictions and the true values. This approach calculates the degree of deviation from the above-mentioned dimensions.

[0094] Let N represent the number of map features in a scene. The predicted values ​​and the ground truth values ​​are considered as two sets of points. and S i The predicted map feature categories are: The map feature category in the truth value is C. i The connection line between adjacent points in the prediction result is The line connecting adjacent points in the truth value is e. i .

[0095] More specifically, classification loss functions are commonly used in classification problems. They measure the difference between the class predicted by the neural network model and the expected class. Commonly used classification loss functions include cross-entropy loss and log loss. In this embodiment, based on the best matching result at the instance level, each predicted map element is assigned a class label, and the classification loss is calculated using focus loss. The class label can be empty. Specifically, L1 can be calculated using the following formula:

[0096]

[0097] Distance loss functions are commonly used in regression problems. They measure the distance or difference between the output of a neural network model and the actual value, and are a commonly used metric for evaluating the performance of regression models. Commonly used distance loss functions include mean squared error (MSE) and mean absolute error (MAE). In this embodiment, the distance loss function L2 is used to calculate the distance loss between each assigned pair of points using the Manhattan distance between the prediction and the true value. For polyline and polygon nodes, the aim is to limit the position of each predicted point (polyline and polygon node), where the loss of the point pair is defined as the Manhattan distance calculated between each assigned pair of points. Point set S and point set i The block distance is the sum of the projections of the line segments formed by two points in a fixed rectangular coordinate system of Euclidean space onto the axes. Specifically, L2 can be calculated using the following formula:

[0098]

[0099] Orientation loss functions are typically used in tasks where directional information is crucial, such as different directional angles in object detection and vehicle movement direction in autonomous driving. Commonly used orientation loss functions include cosine loss functions and angle loss functions. In this embodiment, the orientation loss function L3 is used to design edge orientation loss for the connection lines between adjacent points to constrain the geometry of higher edge levels and calculate the cosine similarity between the prediction and the ground truth. Specifically, L3 can be calculated using the following formula:

[0100]

[0101] Topological similarity loss functions are commonly used in tasks such as image segmentation to measure the topological similarity of segmented images. Topological similarity measures the degree of change in the topological structure of the same object in the original and segmented images, such as the similarity in quantity, shape, and position. In this embodiment, the topological relationships between line elements are assigned corresponding values ​​based on their proximity, thereby establishing a topological difference matrix. The statistical mean difference of the topological relationships contained in the line group is calculated as the topological similarity loss L4. Specifically, L4 can be calculated using the following formula:

[0102]

[0103] Input the toroidal image samples into the initial mapping model, use the loss function to make the network output match the expected value, obtain the optimal weight parameters (weights and biases) in the initial mapping model, and thus obtain the trained initial mapping model.

[0104] In practical applications, the online vehicle mapping method for parking scenarios can be implemented using the following steps:

[0105] Determine the target parking area based on the parking lot entrance on the navigation map;

[0106] After determining that the vehicle has entered the target parking area, it is checked whether there is an offline high-precision map of the drivable area. If there is no map of the drivable area, an online vector map is generated in real time using the method described above for parking planning.

[0107] If a high-precision map exists for the drivable area, an online vector map is generated in real time using the method described above. The real-time online vector map is then projected onto the same coordinate system and fused with the offline high-precision map to obtain the latest real-time vector map for use in parking planning.

[0108] It is understandable that they were combined together. Figure 3 As shown, in an embodiment of the present invention, the overall idea is as follows: First, surround view camera data, i.e., images from the front, rear, left, and right perspectives, are obtained through a vehicle surround view fisheye camera, and input into a pre-trained mapping model along with the navigation map of the current environment; the mapping model performs unified multi-view spatiotemporal fusion on the surround view camera data to obtain an element geometric structure model, performs geometric representation of map elements by category, and matches the semantic categories and node positions of map elements according to the categories, and further adds road-level attribute information and topological relationships to the network, and finally generates a vector map of the lane-level road network of the environment.

[0109] More specifically, such as Figure 4 As shown, an example of using Figure 1The following is a flowchart illustrating the parking process. It includes the following steps:

[0110] Step S1: After completing the preliminary steps such as powering on the vehicle, system self-check, and driving by the user, the user sets the parking lot entrance as the destination in the navigation map, that is, determines the target parking area, such as a parking lot of a residential area or a shopping mall.

[0111] Step S2: The system detects whether the vehicle has entered the target geofence. Control conditions: The navigation map needs to be associated with a Area of ​​Interest (POI) layer such as a residential area / shopping mall; the system obtains real-time vehicle position and attitude data from the vehicle's positioning module, such as GNSS data and IMU data, and combines the vehicle's position and attitude data to determine whether the vehicle has entered the drivable area.

[0112] Step S3: Initialize positioning activation. The control condition is: vehicle detection and recognition pass through the entrance gate of the drivable area.

[0113] Step S4: The system checks whether a high-precision map exists in the drivable area.

[0114] Step S5: If the drivable area is not on the map, proceed as described above. Figures 1 to 2 The provided method generates real-time online vector base maps. If a map exists for the drivable area, the real-time map and the offline map can be projected onto the same coordinate system and fused, with the offline map serving as the base map and the real-time map as the update, continuously ensuring map freshness. The fusion process, or stitching and reconstruction, primarily employs methods such as coarse alignment, correlation, joint nonlinear optimization, and surface optimization. Through this step |S5, a 6-DOF high-frequency continuous trajectory output can be obtained on the vehicle.

[0115] Step S6: Perform route planning at the vehicle end according to the target parking space selected by the user.

[0116] Step S7: Complete parking.

[0117] Accordingly, such as Figure 5 The diagram shown illustrates a structural schematic of an embodiment of a vehicle online mapping system provided by the present invention. (In conjunction with...) Figure 6 As shown, in this embodiment, the online vehicle mapping system 1 includes at least:

[0118] The initial data acquisition unit 10 is used to acquire surround-view camera data of the vehicle and a navigation map of the vehicle's surrounding environment.

[0119] The model processing unit 11 is used to input the surround-view camera data and the navigation map into a pre-trained mapping model to obtain the semantic category and node position of the map semantic elements;

[0120] The vector map acquisition unit 12 performs multi-journey road network topology association processing based on the semantic category and node position of the map semantic elements obtained by different vehicles, and constructs a vector map of the lane-level road network of the environment in real time.

[0121] Combined Figure 6 As shown, in one example, the model processing unit 11 further includes:

[0122] Preprocessing module 110 is used to perform spatiotemporal fusion processing on the surround view camera data to obtain a ground bird's-eye view of the environment where the vehicle is located.

[0123] The feature extraction module 111 is used to extract features from the ground bird's-eye view, obtain the map semantic elements of the environment, and classify and discretize the map semantic elements to obtain the geometric structure of the map semantic elements.

[0124] The matching module 112 is used to perform semantic instance matching or node matching based on the geometric structure of the map semantic elements, and combine the road-level attribute information and topological relationships in the navigation map to obtain the semantic category and node position of the map semantic elements.

[0125] For more details, please refer to and combine with the above. Figures 1 to 4 The description will not be repeated here.

[0126] Accordingly, in another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned... Figures 1 to 4 The steps of the described method. For more details, please refer to and combine with the foregoing descriptions. Figures 1 to 4 The description will not be repeated here.

[0127] Accordingly, another aspect of the present invention also provides a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the aforementioned... Figures 1 to 4 The steps of the method are described. For more details, please refer to and combine with the foregoing descriptions. Figures 1 to 3 The description will not be repeated here.

[0128] Accordingly, in another aspect, the present invention also provides a vehicle having the aforementioned configuration deployed thereon. Figures 5 to 6 The system described is an online vehicle mapping system. For more details, please refer to and combine with the aforementioned... Figures 5 to 6 The description will not be repeated here.

[0129] Implementing the embodiments of the present invention has the following beneficial effects:

[0130] This invention provides a method, system, storage medium, and vehicle for online vehicle mapping. By acquiring vehicle surround-view fisheye camera data and combining it with low-precision road-level navigation map data, the semantic categories and node locations of map semantic elements can be output, lane-level road networks can be constructed in real time, and vector maps can be generated online and in real time by vehicles. In this invention, map semantic elements are geometrically represented by categories, and the semantic categories and node locations of map semantic elements are obtained by matching according to the categories. Furthermore, road-level attribute information and topological relationships are added to the network. The network structure is used to interact with these feature information to obtain more accurate location information and more high-precision map elements for output, thereby realizing the online and real-time generation of vector maps by vehicles.

[0131] This invention can be used in areas where maps cannot fully cover, such as parking lots. By constructing vector maps online in real time for vehicles, it more flexibly solves the problem of autonomous parking within passable areas. It optimizes autonomous parking map and positioning technology solutions, making it more flexible and eliminating the need for complicated user operations, thus improving the user experience of autonomous parking.

[0132] The online mapping method provided by this invention requires no manual processing, annotation, or maintenance. Compared with traditional offline mapping methods, it has the advantages of low cost, strong scalability, ease of use, and high efficiency, without requiring additional hardware costs.

[0133] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0135] The above description is merely a preferred embodiment of the present invention and should not be construed as limiting the scope of the invention. Therefore, any equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for online vehicle mapping, characterized in that, It should include at least the following steps: Acquire surround-view camera data of the vehicle and navigation map of the vehicle's surroundings; The surround-view camera data and the navigation map are input into a pre-trained mapping model to obtain the semantic categories and node locations of map semantic elements; Based on the semantic categories and node locations of map semantic elements obtained from different vehicles, multi-journey road network topology association processing is performed to construct a vector map of the lane-level road network of the environment in real time. The pre-trained mapping model is used to perform the following processing: The surrounding camera data is subjected to spatiotemporal fusion processing to obtain a ground bird's-eye view of the vehicle's environment; Feature extraction is performed on the ground bird's-eye view to obtain the map semantic elements of the environment, and the map semantic elements are classified and discretized to obtain the geometric structure of the map semantic elements; Based on the geometric structure of the map semantic elements, semantic instance matching or node matching is performed. Combined with the road-level attribute information and topological relationships in the navigation map, the semantic category and node location of the map semantic elements are obtained.

2. The method as described in claim 1, characterized in that, The step of extracting features from the ground-level bird's-eye view to obtain map semantic elements of the surrounding environment, and then classifying and discretizing these map semantic elements to obtain their geometric structure, further includes: The ground bird's-eye view is subjected to semantic feature detection and segmentation to extract map semantic elements, wherein the map semantic elements include at least: roads, lane lines, parking space lines, ground arrows, zebra crossings, and stop lines; The map semantic elements are classified and discretized; zebra crossings, parking space lines, and traffic signs are discretized and represented as polygons; coarse road markings are discretized and represented as multiple line segments or a single line segment; continuous roads and lane markings are discretized and represented as multiple line segments. The discretized map semantic elements are sampled at equal intervals, and each map semantic element is split into different nodes according to the arrangement and combination. The connection relationship and arrangement of the nodes are described according to the topological structure, thereby obtaining the geometric structure of the map semantic elements. The topological structure is the spatial relationship between line elements, which includes separation, adjacency, intersection and overlap.

3. The method as described in claim 2, characterized in that, The step of performing semantic instance matching or node matching based on the geometric structure of the map semantic elements, and combining road-level attribute information and topological relationships in the navigation map to obtain the semantic category and node location of the map semantic elements, further includes: For features represented by polygons, each segment in the polygon is matched with the outline pixels of a semantic instance on the ground bird's-eye view; For elements represented by multiple line segments or a single line segment, node matching is performed. For each map semantic element, the optimal permutation is found from the set of equivalent permutations, such that the predicted nodes correspond one-to-one with the actual nodes and the sum of the distances between point pairs is minimized. Obtain road-level precision attribute information and topological relationships in the navigation map. The attribute information includes road curvature, slope information, number of lane lines, and width; the topological relationships include at least: topological connectivity relationships. The attribute information and topological relationship are unified with the image feature information obtained by feature extraction from the ground bird's-eye view through the embedding operation into a feature vector space; Based on the matching results, constraints are imposed on the category prediction, positional relationship, and directional consistency of map semantic elements, and the semantic category and node position of map semantic elements are output.

4. The method as described in claim 1, characterized in that, The step of performing multi-journey road network topology association processing based on the semantic categories and node locations of map semantic elements obtained from different vehicles, and constructing and outputting a vector map of the lane-level road network of the environment in real time, further includes: Based on the semantic categories and node positions of map semantic elements obtained from different vehicles' observations of the environment from different spatial angles and times, multiple journey trajectories are associated with the connection relationships and arrangement of nodes according to the same set of specifications of topological structure description, forming a lane-level road network, and obtaining and outputting a vector map of the environment.

5. The method according to any one of claims 1 to 4, characterized in that, The step of performing spatiotemporal fusion processing on the surround-view camera data to obtain a ground bird's-eye view of the vehicle's environment further includes: The surround-view camera data is mapped onto a unified vehicle coordinate system and fused to form a ground bird's-eye view of the environment, with time information annotated.

6. The method as described in claim 5, characterized in that, Further includes: Obtain the initial mapping model; Establish the following loss function: L = a*L1 + b*L2 + c*L3 + d*L4 Where a, b, c, and d are the weights that balance different loss terms, L1 is the classification loss function, L2 is the distance loss function, L3 is the orientation loss function, and L4 is the topological similarity loss function; the topological similarity loss function is used to calculate the statistical mean difference between the topological relationships contained in the predicted line group and the ground truth line group. Input a ring view image sample into the initial mapping network model, and use a loss function to make the network output match the expected value, thereby obtaining the optimal weight parameters in the initial mapping model and thus obtaining a trained mapping model.

7. A method for online vehicle mapping, characterized in that, Includes the following steps: Determine the target parking area based on the parking lot entrance on the navigation map; After determining that the vehicle has entered the target parking area, it is detected whether there is an offline high-precision map of the drivable area. If there is no map of the drivable area, an online vector map is generated in real time using the method described in any one of claims 1 to 6 for use in parking planning. If a high-precision map exists in the drivable area, an online vector map is generated in real time using the method described in any one of claims 1 to 6. The real-time online vector map is then projected onto the same coordinate system and fused with the offline high-precision map to obtain the latest real-time vector map for use in parking planning.

8. A vehicle online mapping system, characterized in that, include: The initial data acquisition unit is used to acquire surround-view camera data of the vehicle and navigation maps of the vehicle's surrounding environment. The model processing unit is used to input the surround-view camera data and the navigation map into a pre-trained mapping model to obtain the semantic categories and node locations of map semantic elements; The vector map acquisition unit performs multi-journey road network topology association processing based on the semantic category and node location of the map semantic elements obtained by different vehicles, and constructs a vector map of the lane-level road network of the environment in real time. The model processing unit further includes: The preprocessing module is used to perform spatiotemporal fusion processing on the surround-view camera data to obtain a ground bird's-eye view of the vehicle's environment. The feature extraction module is used to extract features from the ground bird's-eye view, obtain the map semantic elements of the environment, and classify and discretize the map semantic elements to obtain the geometric structure of the map semantic elements. The matching module is used to perform semantic instance matching or node matching based on the geometric structure of the map semantic elements, and combine the road-level attribute information and topological relationships in the navigation map to obtain the semantic category and node position of the map semantic elements.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

11. A vehicle, characterized in that, It is equipped with the vehicle online mapping system as described in claim 8.

Images