Vehicle online mapping method and system, storage medium and vehicle

Abstract

The invention discloses a vehicle online mapping method. The method comprises the following steps: acquiring data of a looking-around camera of a vehicle and a navigation map of an environment where the vehicle is located; inputting the surround view camera data and the navigation map into a pre-trained mapping model to obtain semantic categories and node positions of map semantic elements; and according to semantic categories and node positions of map semantic elements obtained by different vehicles, carrying out multi-journey road network topological association processing, and constructing a vector map of a lane-level road network of the environment in real time. The invention further discloses a corresponding system, a storage medium and a vehicle. According to the invention, the autonomous parking capability of the vehicle in the passable area can be improved, and the method has the advantages of low cost, high efficiency and strong expansibility.

Description

Technical Field

[0001] The present invention relates to the technical field related to parking, and in particular to a vehicle online mapping method, system, storage medium and vehicle. Background Art

[0002] Existing parking lot high-precision maps are usually constructed offline, that is, using the on-board sensors (lidar, camera, inertial navigation sensor, etc.) of the map collection vehicle to collect scene data, and then process and vectorize the data semantically through a combination of automation and manual methods. Alternatively, offline maps are constructed in the drivable area through the simultaneous localization and mapping (SLAM) method using sensors on the smart driving vehicle (camera or lidar).

[0003] However, the existing offline mapping solution has many shortcomings:

[0004] Collecting, processing and labeling data all require a lot of manpower and time costs, and the process is cumbersome; at the same time, the SLAM offline mapping solution requires the vehicle to enter the target area again to complete autonomous parking, which has poor timeliness and is difficult to cover the parking lot scene where the vehicle enters for the first time.

[0005] At the same time, since road information changes dynamically and offline-built maps have poor timeliness, high-frequency updates are required to ensure map accuracy. When expanding to large-scale, wide-ranging high-precision map applications, offline map construction requires a lot of manpower, time, and maintenance costs. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a vehicle online mapping method, system, storage medium and vehicle. It can realize real-time online mapping of parking lots and other places, improve the ability of vehicles to park autonomously in passable areas, and has the advantages of low cost, high efficiency and strong scalability.

[0007] In order to solve the above technical problems, as one aspect of the present invention, a vehicle online mapping method is provided, which at least comprises the following steps:

[0008] Obtain the vehicle's surround view camera data and a navigation map of the vehicle's environment;

[0009] Inputting the surround camera data and the navigation map into a pre-trained mapping model to obtain semantic categories and node positions of map semantic elements;

[0010] According to the semantic categories and node positions of the map semantic elements obtained by different vehicles, multi-trip road network topology association processing is performed to construct a vector map of the lane-level road network of the environment in real time;

[0011] The pre-trained mapping model is used to perform the following processing:

[0012] Performing spatiotemporal fusion processing on the surround view camera data to obtain a ground bird's-eye view of the environment where the vehicle is located;

[0013] Extracting features from the ground bird's-eye view to obtain map semantic elements of the environment, and classifying and discretizing the map semantic elements to obtain the geometric structure of the map semantic elements;

[0014] Semantic instance matching or node matching is performed according to the geometric structure of the map semantic element, and the semantic category and node position of the map semantic element are obtained by combining the road-level attribute information and the topological relationship in the navigation map.

[0015] The step of extracting features from the ground bird's-eye view to obtain map semantic elements of the environment, and classifying and discretizing the map semantic elements to obtain the geometric structure of the map semantic elements further includes:

[0016] Performing semantic feature detection on the ground bird's-eye view map, and performing segmentation to extract map semantic elements, wherein the map semantic elements at least include: roads, lane lines, parking space lines, ground arrows, zebra crossings, and stop lines;

[0017] The map semantic elements are classified and discretized; zebra crossings, parking space lines, and traffic signs are discretized and represented by polygons; rough road markings are discretized and represented by multiple line segments or one line segment; continuous roads and lane markings are discretized and represented by multiple line segments;

[0018] The discretized map semantic elements are sampled at equal intervals, each map semantic element is split into different nodes in a permutation and combination manner, and the connection relationship and arrangement manner between the nodes are described according to the topological structure, so as to obtain the geometric structure of the map semantic elements; wherein the topological structure is the spatial relationship of separation, adjacency, intersection and overlap between line elements.

[0019] The performing of semantic instance matching or node matching according to the geometric structure of the map semantic element and combining the road-level attribute information and topological relationship in the navigation map to obtain the semantic category and node position of the map semantic element further includes:

[0020] For a feature represented by a polygon, each segment in the polygon is matched with a contour pixel of a semantic instance on the ground bird's-eye view image;

[0021] For features represented by multiple line segments or a single line segment, node matching is performed. For each map semantic feature, the optimal arrangement is found from the set of equivalent arrangements so that the predicted nodes correspond to the real nodes one by one and the sum of the distances between the point pairs is minimized.

[0022] Obtaining attribute information and topological relationships of road level accuracy in a navigation map, wherein the attribute information includes curvature, slope information, number and width of lane lines of the road; the topological relationships at least include: topological connection relationships;

[0023] Unifying the attribute information and the topological relationship and the image feature information obtained by extracting features from the ground bird's-eye view into a feature vector space through an embedding operation;

[0024] According to the matching results, the category prediction, position relationship and direction consistency of the map semantic elements are constrained, and the semantic category and node position of the map semantic elements are output.

[0025] The method further comprises: performing multi-trip road network topology association processing according to the semantic categories and node positions of the map semantic elements obtained by different vehicles, constructing and outputting a vector map of the lane-level road network of the environment in real time, and:

[0026] Based on the semantic categories and node positions of map semantic elements obtained by different vehicles observing the environment at different spatial angles and times, multiple journey trajectories are associated with the connection relationship and arrangement method between nodes according to the topological structure description of the same set of specifications to form a lane-level road network, and a vector map of the parking lot is obtained and output.

[0027] The step of performing spatiotemporal fusion processing on the surround view camera data to obtain a ground bird's-eye view of the environment where the vehicle is located further includes:

[0028] The surround view camera data is mapped into a unified vehicle coordinate system, fused to form a ground bird's-eye view of the environment, and annotated with time information.

[0029] Among them, further include:

[0030] Get the initial mapping model;

[0031] Establish the following loss function:

[0032] L=a*L1+b*L2+c*L3+d*L4

[0033] Wherein a, b, c, d are weights for balancing different loss items, L1 is the classification loss function, L2 is the distance loss function, L3 is the direction loss function, and L4 is the topological similarity loss function; the topological similarity loss function is used to calculate the statistical mean difference between the topological relationships contained in the predicted line group and the true value line group;

[0034] The surround image samples are input into the initial mapping network model, the output of the network is made consistent with the expected value through the loss function, the optimal weight parameters in the initial mapping model are obtained, and thus a trained mapping model is obtained.

[0035] Accordingly, another aspect of the present invention further provides a vehicle online mapping method, characterized in that it comprises the following steps:

[0036] Determine the target parking area based on the parking lot entrance in the navigation map;

[0037] After determining that the vehicle enters the target parking area, detecting whether there is an offline high-precision map of the drivable area, if there is no map of the drivable area, using the above method to generate an online vector map in real time for parking planning;

[0038] If a high-precision map exists in the drivable area, an online vector map is generated in real time using the method described above, and the real-time online vector map and the offline high-precision map are projected into the same coordinate system for fusion to obtain the latest real-time vector map for parking planning.

[0039] Accordingly, another aspect of the present invention further provides a vehicle online mapping system, comprising:

[0040] The initial data acquisition unit is used to acquire the vehicle's surround view camera data and the navigation map of the vehicle's environment.

[0041] A model processing unit, used to input the surround camera data and the navigation map into a pre-trained mapping model to obtain semantic categories and node positions of map semantic elements;

[0042] A vector map acquisition unit performs multi-trip road network topology association processing according to the semantic categories and node positions of map semantic elements obtained by different vehicles, and constructs a vector map of the lane-level road network of the environment in real time;

[0043] Wherein, the model processing unit further includes:

[0044] A preprocessing module, used for performing spatiotemporal fusion processing on the surround view camera data to obtain a ground bird's-eye view of the environment where the vehicle is located;

[0045] A feature extraction module is used to extract features from the ground bird's-eye view to obtain map semantic elements of the environment, and to classify and discretize the map semantic elements to obtain the geometric structure of the map semantic elements;

[0046] The matching module is used to perform semantic instance matching or node matching according to the geometric structure of the map semantic elements, and to obtain the semantic category and node position of the map semantic elements in combination with the road-level attribute information and topological relationship in the navigation map.

[0047] Correspondingly, another aspect of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the aforementioned method when executed by a processor.

[0048] Correspondingly, another aspect of the present invention further provides a computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the aforementioned method when executing the computer program.

[0049] Correspondingly, another aspect of the present invention further provides a vehicle on which the above-mentioned vehicle online mapping system is deployed.

[0050] The implementation of the embodiments of the present invention has the following beneficial effects:

[0051] The present invention provides a vehicle online mapping method, system, storage medium and vehicle. By acquiring the vehicle surround fisheye camera data and combining it with the low-precision road-level navigation map data, the semantic categories and node positions of the map semantic elements can be output, and the lane-level road network can be constructed in real time, so that the vehicle can generate a vector map online in real time. In the present invention, the map semantic elements are classified and geometrically represented, and the semantic categories and node positions of the map semantic elements are obtained by matching according to the categories, and the road-level attribute information and topological relationships are further added to the network. The network structure is used to interact with these feature information to obtain more accurate location information and more high-precision map element outputs, so that the vehicle can generate a vector map online in real time.

[0052] The present invention can be used in areas that cannot be fully covered by maps, such as parking lots. By constructing vector maps online in real time by vehicles, the ability of vehicles to park autonomously in passable areas can be more flexibly solved, and the autonomous parking map and positioning technology solutions are optimized. The flexibility is greater and there is no need for complicated operations by users, thereby improving the user experience of autonomous parking.

[0053] The online mapping method provided by the present invention does not require manual processing, annotation and maintenance. Compared with the traditional offline mapping method, it has the advantages of low cost, strong scalability, ease of use and high efficiency, and does not require additional hardware costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, without paying creative labor, other drawings obtained based on these drawings still belong to the scope of the present invention.

[0055] Figure 1 A schematic diagram of the main process of an embodiment of a vehicle online mapping method provided by the present invention;

[0056] Figure 2 for Figure 1 A more detailed flowchart of step S11;

[0057] Figure 3 To adopt Figure 1 Schematic diagram of the principle of the method;

[0058] Figure 4 To adopt Figure 1 A schematic diagram of a process of parking by a method;

[0059] Figure 5 A schematic diagram of the structure of an embodiment of a vehicle online mapping system provided by the present invention;

[0060] Figure 6 for Figure 5 Schematic diagram of the structure of the processing unit in the model. DETAILED DESCRIPTION

[0061] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings.

[0062] like Figure 1 FIG. 1 is a schematic diagram showing the main process of an embodiment of a vehicle online mapping method provided by the present invention. Figure 2 , Figure 3 As shown, in this embodiment, the vehicle online mapping method can be used in parking scenarios and driving scenarios. Specifically, the method at least includes the following steps:

[0063] Step S10, obtaining surround view camera data of the vehicle and a navigation map of the environment where the vehicle is located;

[0064] In a specific example, the step S10 further includes:

[0065] Acquire real-time images captured by multiple fisheye cameras mounted on the vehicle, and acquire surround camera data around the vehicle; in a specific example, multiple fisheye cameras may be set around the vehicle, for example, 4, 6, or 8 cameras may be set; wherein the number of cameras is not limited, but is required to cover a 360-degree viewing angle of the vehicle body;

[0066] At the same time, in this step, a navigation map of the vehicle's current environment is obtained. It can be understood that the current environment may be, for example, a parking lot and a driving road.

[0067] Step S11, inputting the surround camera data and the navigation map into a pre-trained mapping model to obtain semantic categories and node positions of map semantic elements;

[0068] Among them, Figure 2 As shown, in step S11, the pre-trained mapping model is used to perform the following processing:

[0069] Step S110, performing spatiotemporal fusion processing on the surround view camera data to obtain a ground bird's-eye view of the environment where the vehicle is located;

[0070] Specifically, the surround camera data is mapped to a unified vehicle coordinate system, fused to form a ground bird's-eye view of the parking lot, and annotated with time information. Specifically, the two-dimensional image data in the camera sensor is converted into a three-dimensional bird's-eye view, and the time dimension is integrated to unify space and time.

[0071] Step S111, extracting features from the ground bird's-eye view to obtain map semantic elements of the environment, and classifying and discretizing the map semantic elements to obtain the geometric structure of the map semantic elements;

[0072] In a specific example, the step S111 further includes:

[0073] The CNN neural network in the trained neural network model is used to perform semantic feature detection, and the U-Net network in the trained neural network model is used to segment and extract the semantic elements of the parking lot map, wherein the semantic elements of the parking lot map include at least: roads, lane lines, parking space lines, ground arrows, zebra crossings, and stop lines; it can be understood that the high-precision map geometric elements used by autonomous driving vehicles are usually composed of points, lines, and surfaces (polygons). For the autonomous parking scene of parking lots, road boundary lines, lane lines, parking space lines, traffic signs, zebra crossings, speed bumps, etc. are distinguishable and stable feature elements. The preceding semantic feature extraction method is not limited here. Both CNN and U-NET can perform semantic feature detection or semantic element extraction, and other networks can also be used for semantic feature detection or semantic element extraction.

[0074] The map elements are classified and discretized; the zebra crossings, parking space lines, and traffic signs (such as arrows) in the parking scene are discretized and represented by polygons; the rough road markings (such as speed bumps) in the parking scene are discretized and represented by multiple line segments or one line segment; the continuous roads and lane markings (such as lane lines and road side lines) in the parking scene are discretized and represented by multiple line segments;

[0075] The discretized map is sampled at equal intervals, each map element is split into different nodes in an arrangement and combination manner, and the connection relationship and arrangement manner between the nodes are described according to the topological structure, so as to obtain the geometric structure of the map semantic elements: wherein the spatial topological relationship between the road line elements is the spatial relationship of separation, proximity, intersection and overlap between the line elements, and these topological relationships are the spatial connection, inclusion and proximity relationships between geographic entities that are not affected by changes such as plane stretching or rotation.

[0076] It can be understood that in this step, different types of map semantic elements of parking lots are discretized into geometric expressions of classification; structured modeling is used to generate topological connection relationships between key points to improve the accuracy of the description of the topological structure relationship between key points of road lane lines in parking lot environments and output more high-precision map elements.

[0077] Step S112, performing semantic instance matching or node matching according to the geometric structure of the map semantic element, combining the road-level attribute information and topological relationship in the navigation map, to obtain the semantic category and node position of the map semantic element.

[0078] In a specific example, the step S112 further includes:

[0079] Different matching mechanisms are used for label assignment for different types of map semantic elements, including semantic instance matching and node matching. Specifically, for elements represented by polygons, each segment (or edge) in the polygon is matched with the contour pixels of the semantic instance on the three-dimensional bird's-eye view, which is semantic instance matching; for elements represented by multiple line segments or a line segment, node matching is performed, and the optimal arrangement of each map semantic element is found from the equivalent arrangement set, so that the predicted nodes and the real nodes correspond one-to-one and the sum of the distances between the point pairs is minimized; wherein, the equivalent arrangement set contains all equivalent connection relationships between nodes, which is used to describe the topological structure of map elements.

[0080] Obtaining attribute information and topological relationships of road level accuracy in the vehicle navigation map, wherein the attribute information includes the curvature, slope information, number and width of lane lines of the road; the topological relationships at least include: topological connection relationships;

[0081] Through the embedding operation, the attribute information and topological relationship are unified with the feature information obtained by feature extraction of the ground bird's-eye view into the same feature vector space, that is, the fusion of geometric elements and attribute information features is realized, and it is input into the network structure for decoding and output; specifically, the embedding operation can map a high-dimensional vector to a low-dimensional space to achieve dimensionality reduction and feature fusion of information. In image processing, the embedding operation can be implemented by adding a fully connected layer after the convolution layer. In this process, auxiliary information such as attribute and topological information can be introduced and embedded into the same feature vector space as the visual information, thereby realizing the organic fusion of visual features and attribute information. In this way, the entire processing process of visual and non-visual information can be maintained in the same feature vector space, achieving full fusion and balanced processing of information.

[0082] According to the matching results, the category prediction, position relationship and direction consistency of the map semantic elements are constrained, and the semantic category and node position of the map semantic elements are output.

[0083] It can be understood that in this step, road-level attribute information is added to the network structure, and the network structure is used to interact with these feature information to obtain more accurate location information and more high-precision map element outputs, rather than being limited to real-time vectorization output of high-precision map linear elements such as lane lines and crosswalks.

[0084] Step S12, performing multi-trip road network topology association processing according to the semantic categories and node positions of the map semantic elements obtained by different vehicles, and constructing a vector map of the lane-level road network of the environment in real time.

[0085] In a specific example, the step S12 further includes:

[0086] Based on the semantic categories and node positions of map semantic elements observed by different vehicles at different spatial angles and times of the parking lot, multiple journey trajectories are associated with the connection relationship and arrangement between nodes according to the topological structure description of the same set of specifications to form a lane-level road network and obtain a vector map of the parking lot. Specifically, it needs to include operations such as geometric matching between maps and joint optimization of results.

[0087] It is understandable that the final vector map of the lane-level road network of the parking lot may include information such as parking space layout, parking lot structure, traffic information, safety facilities, and other parking facilities. Among them, the parking space layout may specifically include parking methods (horizontal parking, vertical parking, diagonal parking, etc.), parking space numbers, the number of parking spaces, and other information. The parking lot structure refers to the internal structure of the parking lot and various functional areas, such as import and export channels, payment areas, channels, entrance and exit lanes, etc. Traffic information includes information such as routes for entering and exiting channels, speed limit signs, and traffic lights. Safety facilities include surveillance cameras, alarms, signs, etc. Other parking facilities may be repair shops, gas stations, car wash stations, toilets, etc. These contents in the vector map can help the vehicle navigation system understand the internal structure of the parking lot, provide accurate navigation and guidance, and make it easier for drivers to find free parking spaces and complete parking operations.

[0088] At the same time, it can be understood that in the method provided by the present invention, it is necessary to establish a neural network in advance and perform training to form a trained neural network model, and the steps include:

[0089] Establish a neural network model including CNN neural network and U-Net network;

[0090] The following loss function is established, which consists of four parts and assigns weights to each part:

[0091] L=a*L1+b*L2+c*L3+d*L4

[0092] Among them, a, b, c, and d are weights for balancing different loss terms, L1 is the classification loss function, L2 is the distance loss function, L3 is the direction loss function, and L4 is the topological similarity loss function;

[0093] The loss function is used to measure the degree of deviation between the prediction made by the model and the true value. This solution calculates the degree of deviation from the above dimensions.

[0094] N represents the number of map elements in a scene. The predicted value and the true value are regarded as two sets of points. and S i , the predicted map feature category is The map feature category in the true value is C i , the connection line between adjacent points of the prediction result is The connecting line between adjacent points in the true value is e i .

[0095] More specifically, the classification loss function is often used for classification problems, which measures the gap between the category predicted by the neural network model and the expected category. Commonly used classification loss functions include cross entropy loss function, logarithmic loss function, etc. In this embodiment, according to the best matching result at the instance level, a category label is assigned to each predicted map element, and the classification loss is calculated by focal loss. Among them, the category label can be empty. Specifically, L1 can be calculated by the following formula:

[0096]

[0097] The distance loss function is usually used for regression problems. It measures the distance or difference between the output of the neural network model and the actual value. It is a commonly used indicator to evaluate the performance of the regression model. Common distance loss functions include mean square error (MSE), mean absolute error (MAE), etc. In this embodiment, the distance loss function L2 is used to calculate the distance loss between each assigned point pair through the Manhattan distance (Manhattan Distance) between the prediction and the true value. For the nodes of the polyline and polygon, the purpose is to limit the position of each predicted point (node ​​of the polyline and polygon), where the loss of the point pair is defined as the Manhattan distance calculated between each assigned point pair. Point set and point set S i The block distance is the sum of the distances of the projections of the line segment formed by two points on the axis in the fixed rectangular coordinate system of the Euclidean space. Specifically, L2 can be calculated by the following formula:

[0098]

[0099] Direction loss functions are usually used in tasks that require attention to direction information, such as different direction angles in object detection, vehicle movement direction in autonomous driving, etc. Commonly used direction loss functions include cosine loss function, angle loss function, etc. In this embodiment, the direction loss function L3 is used to design edge direction loss for the connecting line between adjacent points to limit the geometry of higher edge levels and calculate the cosine similarity of the prediction and the true value. Specifically, L3 can be calculated by the following formula:

[0100]

[0101] The topological similarity loss function is generally used in tasks such as image segmentation to measure the topological structure similarity of image segmentation. Among them, the topological structure similarity is used to measure the degree of change in the topological structure of the same target in the original image and the segmented image, such as the similarity of quantity, shape, position, etc. In this embodiment, the topological relationships between line elements are assigned corresponding values ​​according to the proximity between them, so as to establish a topological difference matrix, and calculate the statistical mean difference of the topological relationships contained in the line group as the topological similarity loss L4. Specifically, L4 can be calculated by the following formula:

[0102]

[0103] The surround image samples are input into the initial mapping model, and the output of the network is made consistent with the expected value through the loss function to obtain the optimal weight parameters (weights and biases) in the initial mapping model, thereby obtaining a trained initial mapping model.

[0104] In specific applications, the online vehicle mapping method for parking scenarios may adopt the following application steps:

[0105] Determine the target parking area based on the parking lot entrance in the navigation map;

[0106] After determining that the vehicle enters the target parking area, detecting whether there is an offline high-precision map of the drivable area, if there is no map of the drivable area, using the above method to generate an online vector map in real time for parking planning;

[0107] If a high-precision map exists in the drivable area, an online vector map is generated in real time using the method described above, and the real-time online vector map and the offline high-precision map are projected into the same coordinate system for fusion to obtain the latest real-time vector map for parking planning.

[0108] It is understandable that, combined with Figure 3 As shown, in an embodiment of the present invention, the overall idea is: first, the surround view camera data around the vehicle, i.e., the images of the front, back, left and right perspectives, are obtained through a self-view fisheye camera, and the images are input into a pre-trained mapping model together with the navigation map of the current environment; the surround view camera data are subjected to unified multi-view spatiotemporal fusion through the mapping model, and an element geometric structure model is obtained, the map elements are classified and geometrically represented, and the semantic categories and node positions of the map elements are obtained by matching according to the categories, and the road-level attribute information and topological relationships are further added to the network, and finally a vector map of the lane-level road network of the environment is generated.

[0109] More specifically, if Figure 4 An example is shown in FIG. Figure 1The process flow diagram of parking by the method of FIG. It includes the following steps:

[0110] Step S1, after completing the previous steps such as vehicle power-on, system self-check, and user driving, the user sets the parking lot entrance as the end point in the navigation map, that is, determines the target parking area, such as a residential area, a shopping mall parking lot, etc.

[0111] Step S2: The system detects whether the vehicle has entered the target geographic fence. Control conditions: The navigation map needs to be associated with the area of ​​interest (POI) layer such as the community / shopping mall; the vehicle's real-time position and posture data in the vehicle-side positioning module such as the vehicle's GNSS data and IMU data are obtained, and the vehicle's position and posture data are combined to determine whether the vehicle has entered the drivable area.

[0112] Step S3, initialization positioning activation. The control condition is: the vehicle is detected and identified to pass through the entrance gate of the drivable area.

[0113] Step S4: The system detects whether there is a high-precision map in the drivable area.

[0114] Step S5: If there is no map for the drivable area, Figure 1 to Figure 2 The method provided is used to generate a real-time online vector base map; if there is a map in the drivable area, the real-time map can be projected into the same coordinate system with the offline map for fusion, and the offline map can be regarded as the base map, and the real-time map can be used as an update to continuously ensure the freshness of the map. Among them, fusion is splicing and reconstruction, and the main methods include coarse alignment, association, joint nonlinear optimization, surface optimization, etc. Through this step | S5, a 6-degree-of-freedom high-frequency continuous trajectory output can be obtained on the vehicle side.

[0115] Step S6, the vehicle performs path planning according to the target parking space selected by the user.

[0116] Step S7, parking is completed.

[0117] Accordingly, if Figure 5 FIG. 1 is a schematic diagram showing a structure of an embodiment of a vehicle online mapping system provided by the present invention. Figure 6 As shown, in this embodiment, the vehicle online mapping system 1 at least includes:

[0118] The initial data acquisition unit 10 is used to acquire the surround view camera data of the vehicle and the navigation map of the environment where the vehicle is located.

[0119] A model processing unit 11 is used to input the surround camera data and the navigation map into a pre-trained mapping model to obtain semantic categories and node positions of map semantic elements;

[0120] A vector map acquisition unit 12 performs multi-trip road network topology association processing according to the semantic categories and node positions of the map semantic elements obtained by different vehicles, and constructs a vector map of the lane-level road network of the environment in real time;

[0121] Combined Figure 6 As shown, in one example, the model processing unit 11 further includes:

[0122] A preprocessing module 110 is used to perform spatiotemporal fusion processing on the surround view camera data to obtain a ground bird's-eye view of the environment where the vehicle is located;

[0123] The feature extraction module 111 is used to extract features from the ground bird's-eye view to obtain map semantic elements of the environment, and classify and discretize the map semantic elements to obtain the geometric structure of the map semantic elements;

[0124] The matching module 112 is used to perform semantic instance matching or node matching according to the geometric structure of the map semantic element, and obtain the semantic category and node position of the map semantic element in combination with the road-level attribute information and topological relationship in the navigation map.

[0125] For more details, please refer to and combine the above Figures 1 to 4 The description will not be repeated here.

[0126] Accordingly, in another aspect of the present invention, there is also provided a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the above-mentioned Figures 1 to 4 For more details, please refer to and combine the above Figures 1 to 4 The description will not be repeated here.

[0127] Accordingly, another aspect of the present invention further provides a computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the computing device implements the above-mentioned Figures 1 to 4 Describe the steps of the method. For more details, please refer to and combine the above Figures 1 to 3 The description will not be repeated here.

[0128] Accordingly, another aspect of the present invention further provides a vehicle on which the aforementioned Figure 5 to Figure 6 For more details, please refer to and combine the above Figure 5 to Figure 6 The description will not be repeated here.

[0129] The implementation of the embodiments of the present invention has the following beneficial effects:

[0130] The present invention provides a vehicle online mapping method, system, storage medium and vehicle. By acquiring the vehicle surround fisheye camera data and combining it with the low-precision road-level navigation map data, the semantic categories and node positions of the map semantic elements can be output, and the lane-level road network can be constructed in real time, so that the vehicle can generate a vector map online in real time. In the present invention, the map semantic elements are classified and geometrically represented, and the semantic categories and node positions of the map semantic elements are obtained by matching according to the categories, and the road-level attribute information and topological relationships are further added to the network. The network structure is used to interact with these feature information to obtain more accurate location information and more high-precision map element outputs, so that the vehicle can generate a vector map online in real time.

[0131] The present invention can be used in areas that cannot be fully covered by maps, such as parking lots. By constructing vector maps online in real time by vehicles, the ability of vehicles to park autonomously in passable areas can be more flexibly solved, and the autonomous parking map and positioning technology solutions are optimized. The flexibility is greater and there is no need for complicated operations by users, thereby improving the user experience of autonomous parking.

[0132] The online mapping method provided by the present invention does not require manual processing, annotation and maintenance. Compared with the traditional offline mapping method, it has the advantages of low cost, strong scalability, ease of use and high efficiency, and does not require additional hardware costs.

[0133] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] The above disclosure is only a preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A vehicle online mapping method, characterized in that: At least the following steps are included: Obtain the vehicle's surround view camera data and a navigation map of the vehicle's environment; Inputting the surround camera data and the navigation map into a pre-trained mapping model to obtain semantic categories and node positions of map semantic elements; According to the semantic categories and node positions of the map semantic elements obtained by different vehicles, multi-trip road network topology association processing is performed to construct a vector map of the lane-level road network of the environment in real time; The pre-trained mapping model is used to perform the following processing: Performing spatiotemporal fusion processing on the surround view camera data to obtain a ground bird's-eye view of the environment where the vehicle is located; Extracting features from the ground bird's-eye view to obtain map semantic elements of the environment, and classifying and discretizing the map semantic elements to obtain the geometric structure of the map semantic elements; Semantic instance matching or node matching is performed according to the geometric structure of the map semantic element, and the semantic category and node position of the map semantic element are obtained by combining the road-level attribute information and the topological relationship in the navigation map.

2. The method according to claim 1, characterized in that The step of extracting features from the ground bird's-eye view to obtain map semantic elements of the environment, and classifying and discretizing the map semantic elements to obtain the geometric structure of the map semantic elements further includes: Performing semantic feature detection on the ground bird's-eye view map, and performing segmentation to extract map semantic elements, wherein the map semantic elements at least include: roads, lane lines, parking space lines, ground arrows, zebra crossings, and stop lines; The map semantic elements are classified and discretized; zebra crossings, parking space lines, and traffic signs are discretized and represented by polygons; rough road markings are discretized and represented by multiple line segments or one line segment; continuous roads and lane markings are discretized and represented by multiple line segments; The discretized map semantic elements are sampled at equal intervals, each map semantic element is split into different nodes in a permutation and combination manner, and the connection relationship and arrangement manner between the nodes are described according to the topological structure, so as to obtain the geometric structure of the map semantic elements; wherein the topological structure is the spatial relationship of separation, adjacency, intersection and overlap between line elements.

3. The method according to claim 2, characterized in that The performing of semantic instance matching or node matching according to the geometric structure of the map semantic element and combining the road-level attribute information and topological relationship in the navigation map to obtain the semantic category and node position of the map semantic element further includes: For a feature represented by a polygon, each segment in the polygon is matched with a contour pixel of a semantic instance on the ground bird's-eye view image; For features represented by multiple line segments or a single line segment, node matching is performed. For each map semantic feature, the optimal arrangement is found from the set of equivalent arrangements so that the predicted nodes correspond to the real nodes one by one and the sum of the distances between the point pairs is minimized. Obtaining attribute information and topological relationships of road level accuracy in a navigation map, wherein the attribute information includes curvature, slope information, number and width of lane lines of the road; the topological relationships at least include: topological connection relationships; Unifying the attribute information and the topological relationship and the image feature information obtained by extracting features from the ground bird's-eye view into a feature vector space through an embedding operation; According to the matching results, the category prediction, position relationship and direction consistency of the map semantic elements are constrained, and the semantic category and node position of the map semantic elements are output.

4. The method according to claim 1, characterized in that The method further comprises: performing multi-trip road network topology association processing according to the semantic categories and node positions of the map semantic elements obtained by different vehicles, constructing and outputting a vector map of the lane-level road network of the environment in real time, and: Based on the semantic categories and node positions of map semantic elements obtained by different vehicles observing the environment at different spatial angles and times, multiple journey trajectories are associated with the connection relationship and arrangement method between nodes according to the topological structure description of the same set of specifications to form a lane-level road network, and a vector map of the parking lot is obtained and output.

5. The method according to any one of claims 1 to 4, characterized in that: The performing spatiotemporal fusion processing on the surround view camera data to obtain a ground bird's-eye view of the environment where the vehicle is located further includes: The surround view camera data is mapped into a unified vehicle coordinate system, fused to form a ground bird's-eye view of the environment, and annotated with time information.

6. The method according to claim 5, characterized in that Further including: Get the initial mapping model; Establish the following loss function: L=a*L1+b*L2+c*L3+d*L4 Wherein a, b, c, d are weights for balancing different loss items, L1 is the classification loss function, L2 is the distance loss function, L3 is the direction loss function, and L4 is the topological similarity loss function; the topological similarity loss function is used to calculate the statistical mean difference between the topological relationships contained in the predicted line group and the true value line group; The surround image samples are input into the initial mapping network model, the output of the network is made consistent with the expected value through the loss function, the optimal weight parameters in the initial mapping model are obtained, and thus a trained mapping model is obtained.

7. A vehicle online mapping method, characterized in that: The steps include: Determine the target parking area based on the parking lot entrance in the navigation map; After determining that the vehicle enters the target parking area, detecting whether there is an offline high-precision map of the drivable area, if there is no map of the drivable area, using the method according to any one of claims 1 to 6 to form an online vector map in real time for parking planning; If a high-precision map exists in the drivable area, an online vector map is generated in real time using the method described in any one of claims 1 to 6, and the online vector map obtained in real time is projected and fused with the offline high-precision map into the same coordinate system to obtain the latest real-time vector map for parking planning.

8. A vehicle online mapping system, characterized in that: include: The initial data acquisition unit is used to acquire the vehicle's surround view camera data and the navigation map of the vehicle's environment. A model processing unit, used to input the surround camera data and the navigation map into a pre-trained mapping model to obtain semantic categories and node positions of map semantic elements; A vector map acquisition unit performs multi-trip road network topology association processing according to the semantic categories and node positions of map semantic elements obtained by different vehicles, and constructs a vector map of the lane-level road network of the environment in real time; Wherein, the model processing unit further includes: A preprocessing module, used for performing spatiotemporal fusion processing on the surround view camera data to obtain a ground bird's-eye view of the environment where the vehicle is located; A feature extraction module is used to extract features from the ground bird's-eye view to obtain map semantic elements of the environment, and to classify and discretize the map semantic elements to obtain the geometric structure of the map semantic elements; The matching module is used to perform semantic instance matching or node matching according to the geometric structure of the map semantic elements, and to obtain the semantic category and node position of the map semantic elements in combination with the road-level attribute information and topological relationship in the navigation map.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A vehicle, characterized in that: The vehicle online mapping system as claimed in claim 8 is deployed thereon.

Images