Method, apparatus, and electronic device for constructing a semantic map of a parking lot

Abstract

The present invention provides a method, apparatus, and electronic device for constructing a semantic map of a parking lot. In this method, semantic point clouds and lane solid line features of the parking lot are extracted, and then map building is performed based on the semantic point clouds and lane solid line features. Finally, in the obtained semantic map of the parking lot, the semantic point clouds and lane solid line features can be retained, improving the accuracy of subsequent positioning. It can be seen that in the method for constructing the semantic map of the parking lot of the present invention, the semantic point clouds and lane solid line features are more reasonable than the parking space features of the traditional solution and can adapt to various parking lot scenarios, with good universality.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular, to a method, device, and electronic device for constructing a semantic map of a parking lot. Background Art

[0002] In recent years, with the progress of autonomous driving technology, the automatic valet parking technology has also developed rapidly. Vehicles with the function of automatic valet parking can achieve autonomous low-speed cruising, searching for parking spaces, and parking in and out, which is an effective solution to the problem of difficult parking. Among them, the creation of a semantic SLAM positioning map for a parking lot is a key technology.

[0003] Currently, in terms of constructing a parking lot environment map, there are mainly the following two solutions: One is to use a vehicle surround fisheye camera to splice and obtain a bird's-eye view, then extract the parking space features, and then generate a parking space map based on the matching of the parking space features. This method only retains the enhanced features of the parking spaces as map elements, resulting in poor subsequent positioning accuracy and only being applicable to scenarios with parking spaces; the other is to use a front camera and a vehicle surround fisheye camera, process and splice to obtain a bird's-eye view, and perform semantic feature extraction and matching, parking space corner point extraction and matching, and column feature extraction and matching based on the bird's-eye view. This method must introduce a front camera, increasing the consumption of processing resources, and since column features are required, it is only applicable to indoor parking lot environments and is not very applicable to outdoor parking lot environments.

[0004] In summary, the existing methods for constructing a parking lot environment map have technical problems such as unreasonable feature selection and poor universality. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method, device, and electronic device for constructing a semantic map of a parking lot to alleviate the technical problems of unreasonable feature selection and poor universality in the existing methods for constructing a parking lot environment map.

[0006] In a first aspect, an embodiment of the present invention provides a method for constructing a semantic map of a parking lot, including:

[0007] Splicing the current frame image collected by the vehicle surround camera to generate a current frame bird's-eye view of the parking lot;

[0008] Performing semantic segmentation on the current frame bird's-eye view to obtain a corresponding semantic segmentation map, and determining a semantic point cloud and a lane solid line feature in the vehicle coordinate system based on the semantic segmentation map;

[0009] Match the current frame bird's-eye view and the previous frame bird's-eye view based on the semantic point cloud in the vehicle coordinate system corresponding to the current frame bird's-eye view, the lane solid line feature in the vehicle coordinate system corresponding to the current frame bird's-eye view, the semantic point cloud in the vehicle coordinate system corresponding to the previous frame bird's-eye view, the pose corresponding to the previous frame bird's-eye view, and the lane solid line feature in the vehicle coordinate system corresponding to the previous frame bird's-eye view, and calculate the pose corresponding to the current frame bird's-eye view;

[0010] When the current frame bird's-eye view is a key frame and the number of key frames reaches a preset value, use the current frame bird's-eye view as the current key frame, and use the current key frame and the previous first preset number of key frames as the local map;

[0011] Use the method of graph optimization to perform matching optimization on the local map, and obtain the lane solid line feature in the optimized world coordinate system and the optimized poses corresponding to each key frame;

[0012] Construct a parking lot semantic map according to the lane solid line feature in the optimized world coordinate system, the semantic point cloud in the vehicle coordinate system corresponding to each key frame, and the optimized poses corresponding to each key frame.

[0013] Further, determining the semantic point cloud and the lane solid line feature in the vehicle coordinate system based on the semantic segmentation map includes:

[0014] Convert the pixel coordinates of the points in the semantic segmentation map into the coordinates of the points in the vehicle coordinate system, and then obtain the semantic point cloud in the vehicle coordinate system;

[0015] Extract the lane solid line image from the semantic segmentation map, and perform binarization processing on the lane solid line image to obtain a lane solid line binarized image;

[0016] Traverse the lane solid line binarized image row by row. If n adjacent pixels all represent lane solid line pixels, set the n adjacent pixels as non-lane solid line pixels to obtain a lane solid line binarized image parallel to the vehicle's forward direction, where n is a preset integer;

[0017] Traverse the lane solid line binarized image parallel to the vehicle's forward direction according to a preset width, and determine the number of all lane solid line pixels in each preset width area;

[0018] Determine a second preset number of target preset width areas and the pixel coordinates of the lane solid line pixels in each target preset width area according to the number of all lane solid line pixels in each preset width area, where the number of lane solid line pixels in the target preset width area is the largest;

[0019] Convert the pixel coordinates of the lane solid line pixels within each of the target preset width regions into the coordinates of points in the vehicle coordinate system, thereby obtaining each lane solid line point cloud in the vehicle coordinate system;

[0020] Solve for the unknown parameters in the preset straight line function based on each lane solid line point cloud to obtain the lane solid line feature in the vehicle coordinate system.

[0021] Further, match the current frame bird's-eye view and the previous frame bird's-eye view based on the semantic point cloud in the vehicle coordinate system corresponding to the current frame bird's-eye view, the lane solid line feature in the vehicle coordinate system corresponding to the current frame bird's-eye view, the semantic point cloud in the vehicle coordinate system corresponding to the previous frame bird's-eye view, the pose corresponding to the previous frame bird's-eye view, and the lane solid line feature in the vehicle coordinate system corresponding to the previous frame bird's-eye view, and calculate the pose corresponding to the current frame bird's-eye view, including:

[0022] Convert the semantic point cloud in the vehicle coordinate system corresponding to the previous frame bird's-eye view to the world coordinate system according to the pose corresponding to the previous frame bird's-eye view, to obtain the semantic point cloud in the world coordinate system corresponding to the previous frame bird's-eye view;

[0023] Convert the semantic point cloud in the vehicle coordinate system corresponding to the current frame bird's-eye view to the world coordinate system according to the initial pose corresponding to the current frame bird's-eye view preset, to obtain the semantic point cloud in the world coordinate system corresponding to the current frame bird's-eye view;

[0024] Determine the matching relationship between the semantic point cloud in the world coordinate system corresponding to the previous frame bird's-eye view and the semantic point cloud in the world coordinate system corresponding to the current frame bird's-eye view through nearest neighbor search, and calculate the nearest neighbor distance error between the matching semantic point clouds, and use the weighted sum of the nearest neighbor distance errors as the first constraint relationship, where the weights of the nearest neighbor distance errors corresponding to different categories of semantic point clouds are different;

[0025] Determine the lane solid line endpoints in the vehicle coordinate system corresponding to the current frame bird's-eye view according to the lane solid line feature in the vehicle coordinate system corresponding to the current frame bird's-eye view;

[0026] Convert the lane solid line endpoints in the vehicle coordinate system corresponding to the current frame bird's-eye view to the world coordinate system according to the initial pose corresponding to the current frame bird's-eye view preset, to obtain the lane solid line endpoints in the world coordinate system corresponding to the current frame bird's-eye view;

[0027] Convert the lane solid line feature in the vehicle coordinate system corresponding to the previous frame bird's-eye view to the world coordinate system according to the pose corresponding to the previous frame bird's-eye view, to obtain the lane solid line feature in the world coordinate system corresponding to the previous frame bird's-eye view;

[0028] Construct the distance error between the endpoints of the lane solid lines in the world coordinate system corresponding to the current frame bird's-eye view and the lane solid line features in the world coordinate system corresponding to the previous frame bird's-eye view, and use the distance error as the second constraint relationship;

[0029] Iteratively optimize the initial pose corresponding to the preset current frame bird's-eye view according to the first constraint relationship and the second constraint relationship to obtain the pose corresponding to the current frame bird's-eye view.

[0030] Further, the method further includes:

[0031] When the current frame bird's-eye view is not a key frame, or the number of key frames does not reach the preset value, return to execute the step of stitching the current frame image collected by the vehicle-mounted surround-view camera to generate the current frame bird's-eye view of the parking lot.

[0032] Further, a graph optimization method is used to perform matching optimization on the local map to obtain the optimized lane solid line features in the world coordinate system and the optimized poses corresponding to each key frame, including:

[0033] Determine the endpoints of the lane solid lines in the vehicle coordinate system corresponding to each key frame according to the lane solid line features in the vehicle coordinate system corresponding to each key frame;

[0034] Convert the endpoints of the lane solid lines in the vehicle coordinate system corresponding to each key frame to the world coordinate system according to the pose corresponding to each key frame to obtain the endpoints of the lane solid lines in the world coordinate system corresponding to each key frame;

[0035] Convert the lane solid line features in the vehicle coordinate system corresponding to the foremost key frame to the world coordinate system according to the pose corresponding to the foremost key frame to obtain the lane solid line features in the world coordinate system corresponding to the foremost key frame;

[0036] Construct the distance sum and error between the endpoints of the lane solid lines in the world coordinate system corresponding to each key frame and the lane solid line features in the world coordinate system corresponding to the foremost key frame, and use the distance sum and error as the third constraint relationship;

[0037] Iteratively optimize the lane solid line features in the world coordinate system corresponding to the foremost key frame and the poses corresponding to each key frame according to the third constraint relationship to obtain the optimized lane solid line features in the world coordinate system and the optimized poses corresponding to each key frame.

[0038] Further, stitching the current frame image collected by the vehicle-mounted surround-view camera to generate the current frame bird's-eye view of the parking lot includes:

[0039] Obtain the current frame image collected by the vehicle surround-view camera;

[0040] Use the inverse perspective transformation algorithm to stitch the current frame image to obtain the current frame bird's-eye view of the parking lot.

[0041] Furthermore, the categories of the semantic segmentation at least include: parking space lines, curbs, lane solid lines, lane dashed lines, speed bumps, and road arrows.

[0042] In a second aspect, an embodiment of the present invention further provides a device for constructing a semantic map of a parking lot, including:

[0043] A stitching generation unit, configured to stitch and generate the current frame bird's-eye view of the parking lot according to the current frame image collected by the vehicle surround-view camera;

[0044] A semantic segmentation and determination unit, configured to perform semantic segmentation on the current frame bird's-eye view to obtain a corresponding semantic segmentation map, and determine the semantic point cloud and lane solid line features in the vehicle coordinate system based on the semantic segmentation map;

[0045] A matching unit, configured to match the current frame bird's-eye view and the previous frame bird's-eye view based on the semantic point cloud in the vehicle coordinate system corresponding to the current frame bird's-eye view, the lane solid line features in the vehicle coordinate system corresponding to the current frame bird's-eye view, the semantic point cloud in the vehicle coordinate system corresponding to the previous frame bird's-eye view, the pose corresponding to the previous frame bird's-eye view, and the lane solid line features in the vehicle coordinate system corresponding to the previous frame bird's-eye view, and calculate the pose corresponding to the current frame bird's-eye view;

[0046] A setting unit, configured to, when the current frame bird's-eye view is a key frame and the number of key frames reaches a preset value, use the current frame bird's-eye view as the current key frame, and use the current key frame and the first preset number of previous key frames as a local map;

[0047] A matching optimization unit, configured to perform matching optimization on the local map by using a graph optimization method to obtain the optimized lane solid line features in the world coordinate system and the optimized poses corresponding to each key frame;

[0048] A map construction unit, configured to construct a semantic map of the parking lot according to the optimized lane solid line features in the world coordinate system, the semantic point cloud in the vehicle coordinate system corresponding to each key frame, and the optimized poses corresponding to each key frame.

[0049] In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the method according to any one of the first aspects when executing the computer program.

[0050] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing machine-executable instructions, which, when called and executed by a processor, cause the processor to execute the method according to any one of the above first aspects.

[0051] In an embodiment of the present invention, a method for constructing a semantic map of a parking lot is provided, including: stitching current frame images collected by an on-vehicle surround-view camera to generate a current frame bird's-eye view of the parking lot; performing semantic segmentation on the current frame bird's-eye view to obtain a corresponding semantic segmentation map, and determining a semantic point cloud and a lane solid line feature in the vehicle coordinate system based on the semantic segmentation map; matching the current frame bird's-eye view and the previous frame bird's-eye view based on the semantic point cloud in the vehicle coordinate system corresponding to the current frame bird's-eye view, the lane solid line feature in the vehicle coordinate system corresponding to the current frame bird's-eye view, the semantic point cloud in the vehicle coordinate system corresponding to the previous frame bird's-eye view, the pose corresponding to the previous frame bird's-eye view, and the lane solid line feature in the vehicle coordinate system corresponding to the previous frame bird's-eye view, and calculating the pose corresponding to the current frame bird's-eye view; when the current frame bird's-eye view is a key frame and the number of key frames reaches a preset value, using the current frame bird's-eye view as the current key frame, and using the current key frame and the previous first preset number of key frames as a local map; using a graph optimization method to perform matching optimization on the local map to obtain optimized lane solid line features in the world coordinate system and optimized poses corresponding to each key frame; constructing a semantic map of the parking lot according to the optimized lane solid line features in the world coordinate system, the semantic point clouds in the vehicle coordinate system corresponding to each key frame, and the optimized poses corresponding to each key frame. It can be seen from the above description that in the method for constructing a semantic map of a parking lot according to the present invention, the extracted semantic point cloud and lane solid line feature of the parking lot are used for matching and mapping, and finally, the semantic point cloud and lane solid line feature can be retained in the obtained semantic map of the parking lot, improving the accuracy of subsequent positioning. Therefore, in the method for constructing a semantic map of a parking lot according to the present invention, the semantic point cloud and lane solid line feature are more reasonable than the parking space features in the traditional solution, and can adapt to various parking lot scenarios, with good universality, alleviating the technical problems of unreasonable feature selection and poor universality in the existing method for constructing a parking lot environment map. Description of the Drawings

[0052] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0053] Figure 1Flow chart of a method for constructing a semantic map of a parking lot provided by an embodiment of the present invention;

[0054] Figure 2 Schematic diagram of a semantic segmentation map provided by an embodiment of the present invention;

[0055] Figure 3 Flow chart of determining semantic point cloud and lane solid line features in a vehicle coordinate system based on a semantic segmentation map provided by an embodiment of the present invention;

[0056] Figure 4 Schematic diagram of a binary image provided by an embodiment of the present invention;

[0057] Figure 5 Flow chart of matching the current frame bird's-eye view and the previous frame bird's-eye view provided by an embodiment of the present invention;

[0058] Figure 6 Flow chart of using graph optimization method to perform matching optimization of local map provided by an embodiment of the present invention;

[0059] Figure 7 Schematic diagram of a device for constructing a semantic map of a parking lot provided by an embodiment of the present invention;

[0060] Figure 8 Schematic diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0061] Next, the technical solutions of the present invention will be clearly and completely described in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0062] Existing methods for constructing a parking lot environment map have technical problems such as unreasonable feature selection and poor universality.

[0063] Based on this, in the method for constructing a semantic map of a parking lot of the present invention, the semantic point cloud and lane solid line features of the parking lot are extracted, and then matching mapping is performed according to the semantic point cloud and lane solid line features. Finally, in the obtained semantic map of the parking lot, the semantic point cloud and lane solid line features can be retained, improving the accuracy of subsequent positioning. It can be seen that in the method for constructing a semantic map of a parking lot of the present invention, the semantic point cloud and lane solid line features are more reasonable than the parking space features of the traditional scheme, and can adapt to various parking lot scenarios, with good universality.

[0064] To facilitate the understanding of this embodiment, first, a method for constructing a semantic map of a parking lot disclosed in the embodiments of the present invention will be introduced in detail.

[0065] Embodiment 1:

[0066] According to an embodiment of the present invention, an embodiment of a method for constructing a semantic map of a parking lot is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0067] Figure 1 is a flowchart of a method for constructing a semantic map of a parking lot according to an embodiment of the present invention, as Figure 1 shown, the method includes the following steps:

[0068] Step S102, generate a current-frame bird's-eye view of the parking lot by stitching the current-frame images collected by the in-vehicle surround-view camera;

[0069] In an embodiment of the present invention, the in-vehicle surround-view camera can be an in-vehicle surround-view fisheye camera, which can be installed around the vehicle, specifically under the left and right rearview mirrors and in the front and rear bumpers. The resolution of the above-mentioned current-frame bird's-eye view can be 400*400, and the actual physical size corresponding to each pixel point can be 3 cm.

[0070] Step S104, perform semantic segmentation on the current-frame bird's-eye view to obtain a corresponding semantic segmentation map, and determine the semantic point cloud and lane solid line features in the vehicle coordinate system based on the semantic segmentation map;

[0071] Specifically, the categories of semantic segmentation at least include: parking space lines, curbs, lane solid lines, lane dashed lines, speed bumps, and road arrows. As Figure 2 shown, a schematic diagram of the semantic segmentation map is shown therein, and different colors represent different categories.

[0072] After obtaining the semantic segmentation map, determine the semantic point cloud and lane solid line features in the vehicle coordinate system based on the semantic segmentation map. This process will be described in detail later and will not be elaborated here.

[0073] Step S106, match the current-frame bird's-eye view and the previous-frame bird's-eye view based on the semantic point cloud in the vehicle coordinate system corresponding to the current-frame bird's-eye view, the lane solid line features in the vehicle coordinate system corresponding to the current-frame bird's-eye view, the semantic point cloud in the vehicle coordinate system corresponding to the previous-frame bird's-eye view, the pose corresponding to the previous-frame bird's-eye view, and the lane solid line features in the vehicle coordinate system corresponding to the previous-frame bird's-eye view, and calculate the pose corresponding to the current-frame bird's-eye view;

[0074] It should be noted that if the current frame bird's-eye view is the first frame bird's-eye view and there is no previous frame bird's-eye view, then the matching between the current frame bird's-eye view and the previous frame bird's-eye view is not performed, and the pose of the first frame bird's-eye view can be the preset initial pose.

[0075] This process will be described in detail later and will not be elaborated here.

[0076] Step S108, when the current frame bird's-eye view is a key frame and the number of key frames reaches the preset value, take the current frame bird's-eye view as the current key frame, and use the current key frame and the previous first preset number of key frames as the local map;

[0077] Specifically, when determining whether the current frame bird's-eye view is a key frame, it is determined according to the key frame extraction strategy. For example, if the key frame extraction strategy is to extract a key frame every 5 frames, then it can be determined whether the current frame bird's-eye view is a key frame according to this key frame extraction strategy.

[0078] Step S110, use the method of graph optimization to perform matching optimization on the local map to obtain the optimized lane solid line features in the world coordinate system and the optimized poses corresponding to each key frame;

[0079] This process will be specifically introduced later.

[0080] Step S112, construct a parking lot semantic map according to the optimized lane solid line features in the world coordinate system, the semantic point clouds in the vehicle coordinate system corresponding to each key frame, and the optimized poses corresponding to each key frame.

[0081] Specifically, convert the semantic point clouds in the vehicle coordinate system corresponding to each key frame to the world coordinate system according to the optimized poses corresponding to each key frame to obtain the semantic point clouds in the world coordinate system corresponding to each key frame, that is, the coordinates of the points of various categories in each key frame in the world coordinate system. Then, a parking lot semantic map can be constructed according to the semantic point clouds in the world coordinate system corresponding to each key frame and the optimized lane solid line features in the world coordinate system.

[0082] In an embodiment of the present invention, a method for constructing a semantic map of a parking lot is provided, including: stitching the current frame images collected by an in-vehicle panoramic camera to generate a current frame bird's-eye view of the parking lot; performing semantic segmentation on the current frame bird's-eye view to obtain a corresponding semantic segmentation map, and determining the semantic point cloud and lane solid line features in the vehicle coordinate system based on the semantic segmentation map; matching the current frame bird's-eye view and the previous frame bird's-eye view based on the semantic point cloud in the vehicle coordinate system corresponding to the current frame bird's-eye view, the lane solid line features in the vehicle coordinate system corresponding to the current frame bird's-eye view, the semantic point cloud in the vehicle coordinate system corresponding to the previous frame bird's-eye view, the pose corresponding to the previous frame bird's-eye view, and the lane solid line features in the vehicle coordinate system corresponding to the previous frame bird's-eye view, and calculating the pose corresponding to the current frame bird's-eye view; when the current frame bird's-eye view is a key frame and the number of key frames reaches a preset value, taking the current frame bird's-eye view as the current key frame, and using the current key frame and the previous first preset number of key frames as a local map; using a graph optimization method to perform matching optimization on the local map to obtain the optimized lane solid line features in the world coordinate system and the optimized poses corresponding to each key frame; constructing a semantic map of the parking lot according to the optimized lane solid line features in the world coordinate system, the semantic point clouds in the vehicle coordinate system corresponding to each key frame, and the optimized poses corresponding to each key frame. Through the above description, it can be seen that in the method for constructing a semantic map of a parking lot of the present invention, the semantic point cloud and lane solid line features of the parking lot are extracted, and then matching mapping is performed based on the semantic point cloud and lane solid line features. Finally, in the obtained semantic map of the parking lot, the semantic point cloud and lane solid line features can be retained, improving the accuracy of subsequent positioning. It can be seen that in the method for constructing a semantic map of a parking lot of the present invention, the semantic point cloud and lane solid line features are more reasonable than the parking space features of the traditional scheme, and can adapt to various parking lot scenarios, with good universality, alleviating the technical problems of unreasonable feature selection and poor universality in the existing methods for constructing a map of a parking lot environment.

[0083] The above briefly introduces the method for constructing a semantic map of a parking lot of the present invention, and the following will describe the specific content involved in detail.

[0084] In an optional embodiment of the present invention, the method further includes: when the current frame bird's-eye view is not a key frame, or the number of key frames does not reach the preset value, returning to execute the step of stitching the current frame images collected by the in-vehicle panoramic camera to generate the current frame bird's-eye view of the parking lot.

[0085] In an optional embodiment of the present invention, stitching the current frame images collected by the in-vehicle panoramic camera to generate the current frame bird's-eye view of the parking lot specifically includes the following steps:

[0086] (1) Obtain the current frame images collected by the in-vehicle panoramic camera;

[0087] (2) The inverse perspective transformation algorithm is used to stitch the current frame image to obtain the current frame bird's-eye view of the parking lot.

[0088] In an alternative embodiment of the present invention, referring to Figure 3 , the above step S104, determining the semantic point cloud and the lane solid line feature in the vehicle coordinate system based on the semantic segmentation map, specifically includes the following steps:

[0089] Step S301, convert the pixel coordinates of the points in the semantic segmentation map into the coordinates of the points in the vehicle coordinate system, thereby obtaining the semantic point cloud in the vehicle coordinate system;

[0090] Specifically, according to the following formula P b = T bc K -1 P uv Convert the pixel coordinates of the points in the semantic segmentation map into the coordinates of the points in the vehicle coordinate system, where P b represents the coordinates of the points in the vehicle coordinate system, T bc represents the transformation matrix from the camera coordinate system to the vehicle coordinate system, K represents the internal parameter matrix of the virtual top-view camera, and P uv represents the pixel coordinates of the points in the semantic segmentation map.

[0091] Step S302, extract the lane solid line image in the semantic segmentation map, and perform binarization processing on the lane solid line image to obtain the lane solid line binarized image;

[0092] Specifically, the above binarization processing can be that the lane solid line pixels are valid pixels, and the remaining pixels are all invalid pixels. Figure 4 The upper half of shows the lane solid line binarized image.

[0093] In addition, the lane solid line pixels at the outer edge of the bird's-eye view, or at the stitching seam, etc. can also be processed by adding a mask, and the pixels at this part are set as non-lane solid line pixels, that is, invalid pixels.

[0094] Step S303, traverse the lane solid line binarized image row by row. If n adjacent pixels all represent lane solid line pixels, then set the n adjacent pixels as non-lane solid line pixels to obtain the lane solid line binarized image parallel to the vehicle forward direction, where n is a preset integer;

[0095] The purpose of setting the n adjacent pixels as non-lane solid line pixels if the n adjacent pixels all represent lane solid line pixels is to remove the influence of the lane solid line feature perpendicular to the vehicle forward direction.

[0096] Step S304: Traverse the binary image of the lane solid line parallel to the vehicle's forward direction according to a preset width, and determine the number of all lane solid line pixels in each preset width area.

[0097] The above preset width can specifically be a width slightly wider than the width of the lane solid line. Traversing the binary image of the lane solid line parallel to the vehicle's forward direction according to the preset width specifically means dividing the binary image of the lane solid line parallel to the vehicle's forward direction according to the preset width to obtain multiple binary sub-images of the lane solid line of the binary image of the lane solid line (the width of each binary sub-image of the lane solid line is the above preset width). Furthermore, determining the number of all lane solid line pixels in each preset width area, that is, determining the number of all lane solid line pixels in each binary sub-image of the lane solid line.

[0098] Step S305: Determine a second preset number of target preset width areas and the pixel coordinates of the lane solid line pixels in each target preset width area according to the number of all lane solid line pixels in each preset width area, where the number of lane solid line pixels in the target preset width area is the largest.

[0099] The above second preset number can be 2 because the lane solid line consists of two lines. That is, sort each preset width area according to the number of all lane solid line pixels in each preset width area, and determine 2 target preset width areas with the largest number of lane solid line pixels and the pixel coordinates of the lane solid line pixels in each target preset width area.

[0100] Step S306: Convert the pixel coordinates of the lane solid line pixels in each target preset width area into the coordinates of points in the vehicle coordinate system, and then obtain each lane solid line point cloud in the vehicle coordinate system.

[0101] Specifically, perform the conversion according to the formula in Step S301 to obtain each lane solid line point cloud in the vehicle coordinate system.

[0102] Step S307: Solve the unknown parameters in the preset straight line function according to each lane solid line point cloud to obtain the lane solid line feature in the vehicle coordinate system.

[0103] Specifically, the preset straight line function represents the lane solid line feature, that is, model the lane solid line feature to be extracted as a preset straight line function. The preset straight line function can be: ax + by + 1 = 0, where a and b are unknown parameters. Use each lane solid line point cloud (i.e., coordinates) as variables, establish a loss function with minimizing the error of the preset straight line function as the goal, and use the Gauss-Newton method to solve the unknown parameters in the preset straight line function to obtain the corresponding lane solid line feature in the vehicle coordinate system (i.e., the above straight line equation with the unknown parameters solved).

[0104] Figure 4The visualization image corresponding to the lane solid line feature in the lower half of the vehicle coordinate system.

[0105] In an alternative embodiment of the present invention, referring to Figure 5 , the above step S106, based on the semantic point cloud in the vehicle coordinate system corresponding to the current frame bird's-eye view, the lane solid line feature in the vehicle coordinate system corresponding to the current frame bird's-eye view, the semantic point cloud in the vehicle coordinate system corresponding to the previous frame bird's-eye view, the pose corresponding to the previous frame bird's-eye view, and the lane solid line feature in the vehicle coordinate system corresponding to the previous frame bird's-eye view, matches the current frame bird's-eye view and the previous frame bird's-eye view, and calculates the pose corresponding to the current frame bird's-eye view, specifically including the following steps:

[0106] Step S501, convert the semantic point cloud in the vehicle coordinate system corresponding to the previous frame bird's-eye view to the world coordinate system according to the pose corresponding to the previous frame bird's-eye view, and obtain the semantic point cloud in the world coordinate system corresponding to the previous frame bird's-eye view;

[0107] Step S502, convert the semantic point cloud in the vehicle coordinate system corresponding to the current frame bird's-eye view to the world coordinate system according to the initial pose corresponding to the preset current frame bird's-eye view, and obtain the semantic point cloud in the world coordinate system corresponding to the current frame bird's-eye view;

[0108] Step S503, determine the matching relationship between the semantic point cloud in the world coordinate system corresponding to the previous frame bird's-eye view and the semantic point cloud in the world coordinate system corresponding to the current frame bird's-eye view through nearest neighbor search, calculate the nearest neighbor distance error between the matched semantic point clouds, and use the weighted sum of the nearest neighbor distance errors as the first constraint relationship, where the weights of the nearest neighbor distance errors corresponding to different categories of semantic point clouds are different;

[0109] For example, the semantic point cloud A in the world coordinate system corresponding to the previous frame bird's-eye view matches the semantic point cloud A' in the world coordinate system corresponding to the current frame bird's-eye view, and the calculated nearest neighbor distance error between the two is L1. The semantic point cloud B in the world coordinate system corresponding to the previous frame bird's-eye view matches the semantic point cloud B' in the world coordinate system corresponding to the current frame bird's-eye view, and the calculated nearest neighbor distance error between the two is L2. The semantic point cloud C in the world coordinate system corresponding to the previous frame bird's-eye view matches the semantic point cloud C' in the world coordinate system corresponding to the current frame bird's-eye view, and the calculated nearest neighbor distance error between the two is L3. The category of the above A semantic point cloud is a speed bump, and the weight of the corresponding nearest neighbor distance error is 0.2. The category of the B semantic point cloud is a parking space line, and the weight of the corresponding nearest neighbor distance error is 0.6. The category of the C semantic point cloud is a curb, and the weight of the corresponding nearest neighbor distance error is 0.2. Then the final weighted sum of the nearest neighbor distance errors = 0.2 * L1 + 0.6 * L2 + 0.2 * L3.

[0110] Step S504: Determine the lane solid line endpoints in the vehicle coordinate system corresponding to the current frame bird's-eye view based on the lane solid line features in the vehicle coordinate system corresponding to the current frame bird's-eye view.

[0111] Specifically, since the lane solid line features (i.e., the straight line equations) in the vehicle coordinate system corresponding to the current frame bird's-eye view are known, the two lane solid line endpoints of the lane solid line features in the current frame bird's-eye view can be determined, that is, the lane solid line endpoints in the vehicle coordinate system corresponding to the current frame bird's-eye view.

[0112] Step S505: Convert the lane solid line endpoints in the vehicle coordinate system corresponding to the current frame bird's-eye view to the world coordinate system according to the preset initial pose corresponding to the current frame bird's-eye view, and obtain the lane solid line endpoints in the world coordinate system corresponding to the current frame bird's-eye view.

[0113] Step S506: Convert the lane solid line features in the vehicle coordinate system corresponding to the previous frame bird's-eye view to the world coordinate system according to the pose corresponding to the previous frame bird's-eye view, and obtain the lane solid line features in the world coordinate system corresponding to the previous frame bird's-eye view.

[0114] Step S507: Construct the distance error from the lane solid line endpoints in the world coordinate system corresponding to the current frame bird's-eye view to the lane solid line features in the world coordinate system corresponding to the previous frame bird's-eye view, and use the distance error as the second constraint relationship.

[0115] Step S508: Iteratively optimize the preset initial pose corresponding to the current frame bird's-eye view according to the first constraint relationship and the second constraint relationship to obtain the pose corresponding to the current frame bird's-eye view.

[0116] In an alternative embodiment of the present invention, refer to Figure 6 , the above step S110 uses the graph optimization method to perform local map matching optimization to obtain the optimized lane solid line features in the world coordinate system and the optimized poses corresponding to each key frame, which specifically includes the following steps:

[0117] Step S601: Determine the lane solid line endpoints in the vehicle coordinate system corresponding to each key frame based on the lane solid line features in the vehicle coordinate system corresponding to each key frame.

[0118] The specific process can refer to the description of step S504.

[0119] Step S602: Convert the lane solid line endpoints in the vehicle coordinate system corresponding to each key frame to the world coordinate system according to the pose corresponding to each key frame, and obtain the lane solid line endpoints in the world coordinate system corresponding to each key frame.

[0120] Step S603: Convert the lane solid line feature in the vehicle coordinate system corresponding to the foremost key frame to the world coordinate system according to the pose corresponding to the foremost key frame, so as to obtain the lane solid line feature in the world coordinate system corresponding to the foremost key frame;

[0121] Step S604: Construct the distance and error from the lane solid line endpoints in the world coordinate system corresponding to each key frame to the lane solid line feature in the world coordinate system corresponding to the foremost key frame, and use the distance and error as the third constraint relationship;

[0122] Step S605: Iteratively optimize the lane solid line feature in the world coordinate system corresponding to the foremost key frame and the poses corresponding to each key frame according to the third constraint relationship, so as to obtain the optimized lane solid line feature in the world coordinate system and the optimized poses corresponding to each key frame.

[0123] The above process can be simply described as follows: In graph optimization, vertices are used to represent optimization variables, and edges are used to represent error terms. That is, the graph is composed of vertices and edges. The vertices refer to the poses corresponding to the key frames and the lane solid line features in the world coordinate system, and the edges refer to the projection relationship of the lane solid line features (a transformation from the vehicle coordinate system to the world coordinate system).

[0124] According to the relationship between the edges and vertices in the graph optimization model, list the optimization edge error equation (i.e., the distance from the endpoint to the line), and use the Levenberg-Marquardt algorithm for iterative optimization to obtain the optimized lane solid line feature in the world coordinate system and the optimized poses corresponding to each key frame.

[0125] The edge error equation is: where e represents the distance from the endpoint to the line, which is the error function and is a 2*1-dimensional value, x s represents the coordinate of the starting point of the lane solid line in the world coordinate system corresponding to the key frame, x e represents the coordinate of the ending point of the lane solid line in the world coordinate system corresponding to the key frame, l ′ =[a, b, 1] T represents the parameters of the lane solid line feature in the world coordinate system.

[0126] The method for constructing the parking lot semantic map of the present invention has the following advantages:

[0127] 1. By adopting the semantic segmentation result of the bird's-eye view formed by stitching, extracting the semantic point cloud - lane solid line feature for matching mapping, compared with only using point features for matching mapping, the lateral mapping and positioning accuracy of the vehicle in the lane is higher;

[0128] 2. By adopting the semantic segmentation result of the bird's-eye view, extracting the semantic point cloud - lane solid line feature for matching mapping, it saves computing resources and can meet the real-time requirement;

[0129] 3. The extracted semantic point cloud features depend on general ground marking features, and the line features depend on actual long solid lane lines or road edges, which can adapt to above-ground and underground parking lots and have good environmental adaptability.

[0130] 4. Using the semantic segmentation results of the bird's-eye view, the obtained point-line features are more robust to environmental changes such as lighting than traditional ORB features, etc.

[0131] Embodiment 2:

[0132] The embodiment of the present invention also provides a device for constructing a parking lot semantic map. This device for constructing a parking lot semantic map is mainly used to execute the method for constructing a parking lot semantic map provided in Embodiment 1 of the present invention. The following specifically introduces the device for constructing a parking lot semantic map provided by the embodiment of the present invention.

[0133] Figure 7 is a schematic diagram of a device for constructing a parking lot semantic map according to an embodiment of the present invention, as Figure 7 shown. This device mainly includes: a stitching and generating unit 10, a semantic segmentation and determination unit 20, a matching unit 30, a setting unit 40, a matching optimization unit 50, and a map construction unit 60, where:

[0134] The stitching and generating unit is used to stitch and generate the current frame bird's-eye view of the parking lot according to the current frame image collected by the vehicle-mounted surround camera;

[0135] The semantic segmentation and determination unit is used to perform semantic segmentation on the current frame bird's-eye view to obtain the corresponding semantic segmentation map, and determine the semantic point cloud and lane solid line features in the vehicle coordinate system based on the semantic segmentation map;

[0136] The matching unit is used to match the current frame bird's-eye view and the previous frame bird's-eye view based on the semantic point cloud in the vehicle coordinate system corresponding to the current frame bird's-eye view, the lane solid line features in the vehicle coordinate system corresponding to the current frame bird's-eye view, the semantic point cloud in the vehicle coordinate system corresponding to the previous frame bird's-eye view, the pose corresponding to the previous frame bird's-eye view, and the lane solid line features in the vehicle coordinate system corresponding to the previous frame bird's-eye view, and calculate the pose corresponding to the current frame bird's-eye view;

[0137] The setting unit is used to, when the current frame bird's-eye view is a key frame and the number of key frames reaches a preset value, use the current frame bird's-eye view as the current key frame, and use the current key frame and the previous first preset number of key frames as the local map;

[0138] The matching optimization unit is used to perform matching optimization on the local map by using the graph optimization method to obtain the optimized lane solid line features in the world coordinate system and the optimized poses corresponding to each key frame;

[0139] A map construction unit for constructing a parking lot semantic map based on the lane solid line features in the optimized world coordinate system, the semantic point clouds in the vehicle coordinate system corresponding to each key frame, and the optimized poses corresponding to each key frame.

[0140] In an embodiment of the present invention, a device for constructing a parking lot semantic map is provided, including: stitching the current frame image collected by an in-vehicle panoramic camera to generate a current frame bird's-eye view of the parking lot; performing semantic segmentation on the current frame bird's-eye view to obtain a corresponding semantic segmentation map, and determining the semantic point clouds and lane solid line features in the vehicle coordinate system based on the semantic segmentation map; matching the current frame bird's-eye view and the previous frame bird's-eye view based on the semantic point clouds in the vehicle coordinate system corresponding to the current frame bird's-eye view, the lane solid line features in the vehicle coordinate system corresponding to the current frame bird's-eye view, the semantic point clouds in the vehicle coordinate system corresponding to the previous frame bird's-eye view, the pose corresponding to the previous frame bird's-eye view, and the lane solid line features in the vehicle coordinate system corresponding to the previous frame bird's-eye view, and calculating the pose corresponding to the current frame bird's-eye view; when the current frame bird's-eye view is a key frame and the number of key frames reaches a preset value, taking the current frame bird's-eye view as the current key frame, and taking the current key frame and the previous first preset number of key frames as a local map; using a graph optimization method to perform matching optimization on the local map to obtain the lane solid line features in the optimized world coordinate system and the optimized poses corresponding to each key frame; constructing a parking lot semantic map based on the lane solid line features in the optimized world coordinate system, the semantic point clouds in the vehicle coordinate system corresponding to each key frame, and the optimized poses corresponding to each key frame. Through the above description, it can be seen that in the device for constructing a parking lot semantic map of the present invention, the semantic point clouds and lane solid line features of the parking lot are extracted, and then matching mapping is performed based on the semantic point clouds and lane solid line features. Finally, in the obtained parking lot semantic map, the semantic point clouds and lane solid line features can be retained, improving the accuracy of subsequent positioning. It can be seen that in the method for constructing a parking lot semantic map of the present invention, the semantic point clouds and lane solid line features are more reasonable than the parking space features in the traditional scheme, and can adapt to various parking lot scenarios, with good universality, alleviating the technical problems of unreasonable feature selection and poor universality in the existing methods for constructing parking lot environment maps.

[0141] Optionally, the semantic segmentation and determination unit is further configured to: convert the pixel coordinates of the points in the semantic segmentation map into the coordinates of the points in the vehicle coordinate system, so as to obtain the semantic point cloud in the vehicle coordinate system; extract the lane solid line image from the semantic segmentation map, and perform binarization processing on the lane solid line image to obtain the lane solid line binarized image; traverse the lane solid line binarized image row by row, and if n adjacent pixels all represent lane solid line pixels, set the n adjacent pixels as non-lane solid line pixels to obtain the lane solid line binarized image parallel to the vehicle forward direction, where n is a preset integer; traverse the lane solid line binarized image parallel to the vehicle forward direction according to a preset width, and determine the number of all lane solid line pixels in each preset width area; determine a second preset number of target preset width areas and the pixel coordinates of the lane solid line pixels in each target preset width area according to the number of all lane solid line pixels in each preset width area, where the number of lane solid line pixels in the target preset width area is the largest; convert the pixel coordinates of the lane solid line pixels in each target preset width area into the coordinates of the points in the vehicle coordinate system, so as to obtain the lane solid line point clouds in the vehicle coordinate system; solve the unknown parameters in the preset straight line function according to each lane solid line point cloud to obtain the lane solid line feature in the vehicle coordinate system.

[0142] Optionally, the matching unit is further configured to: convert the semantic point cloud in the vehicle coordinate system corresponding to the previous-frame bird's-eye view to the world coordinate system according to the pose corresponding to the previous-frame bird's-eye view, so as to obtain the semantic point cloud in the world coordinate system corresponding to the previous-frame bird's-eye view; convert the semantic point cloud in the vehicle coordinate system corresponding to the current-frame bird's-eye view to the world coordinate system according to the preset initial pose corresponding to the current-frame bird's-eye view, so as to obtain the semantic point cloud in the world coordinate system corresponding to the current-frame bird's-eye view; determine the matching relationship between the semantic point cloud in the world coordinate system corresponding to the previous-frame bird's-eye view and the semantic point cloud in the world coordinate system corresponding to the current-frame bird's-eye view through nearest neighbor search, calculate the nearest neighbor distance error between the matched semantic point clouds, and use the weighted sum of the nearest neighbor distance errors as the first constraint relationship, where the weights of the nearest neighbor distance errors corresponding to different categories of semantic point clouds are different; determine the lane solid line endpoints in the vehicle coordinate system corresponding to the current-frame bird's-eye view according to the lane solid line features in the vehicle coordinate system corresponding to the current-frame bird's-eye view; convert the lane solid line endpoints in the vehicle coordinate system corresponding to the current-frame bird's-eye view to the world coordinate system according to the preset initial pose corresponding to the current-frame bird's-eye view, so as to obtain the lane solid line endpoints in the world coordinate system corresponding to the current-frame bird's-eye view; convert the lane solid line features in the vehicle coordinate system corresponding to the previous-frame bird's-eye view to the world coordinate system according to the pose corresponding to the previous-frame bird's-eye view, so as to obtain the lane solid line features in the world coordinate system corresponding to the previous-frame bird's-eye view; construct the distance error from the lane solid line endpoints in the world coordinate system corresponding to the current-frame bird's-eye view to the lane solid line features in the world coordinate system corresponding to the previous-frame bird's-eye view, and use the distance error as the second constraint relationship; and iteratively optimize the preset initial pose corresponding to the current-frame bird's-eye view according to the first constraint relationship and the second constraint relationship to obtain the pose corresponding to the current-frame bird's-eye view.

[0143] Optionally, the apparatus is further configured to: when the current-frame bird's-eye view is not a key frame or the number of key frames does not reach a preset value, return to execute the step of stitching the current-frame image collected by the vehicle-mounted surround-view camera to generate the current-frame bird's-eye view of the parking lot.

[0144] Optionally, the matching optimization unit is further configured to: determine the lane solid line endpoints in the vehicle coordinate system corresponding to each key frame according to the lane solid line features in the vehicle coordinate system corresponding to each key frame; convert the lane solid line endpoints in the vehicle coordinate system corresponding to each key frame to the world coordinate system according to the poses corresponding to each key frame to obtain the lane solid line endpoints in the world coordinate system corresponding to each key frame; convert the lane solid line features in the vehicle coordinate system corresponding to the earliest key frame to the world coordinate system according to the pose corresponding to the earliest key frame to obtain the lane solid line features in the world coordinate system corresponding to the earliest key frame; construct the distance and error between the lane solid line endpoints in the world coordinate system corresponding to each key frame and the lane solid line features in the world coordinate system corresponding to the earliest key frame, and use the distance and error as the third constraint relationship; iteratively optimize the lane solid line features in the world coordinate system corresponding to the earliest key frame and the poses corresponding to each key frame according to the third constraint relationship to obtain the optimized lane solid line features in the world coordinate system and the optimized poses corresponding to each key frame.

[0145] Optionally, the stitching generation unit is further configured to: obtain the current frame image collected by the vehicle-mounted surround view camera; perform stitching on the current frame image by using the inverse perspective transformation algorithm to obtain the current frame bird's-eye view of the parking lot.

[0146] Optionally, the categories of semantic segmentation at least include: parking space lines, curbs, lane solid lines, lane dotted lines, speed bumps, and road arrows.

[0147] The device provided by the embodiments of the present invention has the same implementation principle and the same technical effects as those of the foregoing method embodiments. For a brief description, for the parts not mentioned in the device embodiments, reference may be made to the corresponding contents in the foregoing method embodiments.

[0148] As Figure 8 shown, an electronic device 600 provided by an embodiment of the present application includes: a processor 601, a memory 602, and a bus. The memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device runs, the processor 601 communicates with the memory 602 through the bus, and the processor 601 executes the machine-readable instructions to execute the steps of the method for constructing a parking lot semantic map as described above.

[0149] Specifically, the foregoing memory 602 and processor 601 can be general-purpose memory and processor, and no specific limitation is made here. When the processor 601 runs the computer program stored in the memory 602, it can execute the method for constructing a parking lot semantic map as described above.

[0150] The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method can be completed by the integrated logic circuit in the hardware of the processor 601 or instructions in the form of software. The above-mentioned processor 601 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field-programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and combines its hardware to complete the steps of the above method.

[0151] Corresponding to the above method for constructing a parking lot semantic map, an embodiment of the present application further provides a computer-readable storage medium. The computer-readable storage medium stores machine-executable instructions. When the computer-executable instructions are called and run by a processor, the computer-executable instructions cause the processor to run the steps of the above method for constructing a parking lot semantic map.

[0152] The device for constructing a parking lot semantic map provided in the embodiments of the present application may be specific hardware on the device or software or firmware installed on the device, etc. For the device provided in the embodiments of the present application, its implementation principle and the technical effects produced are the same as those of the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in the device embodiments, reference may be made to the corresponding content in the foregoing method embodiments. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can all refer to the corresponding processes in the above method embodiments, and will not be elaborated herein.

[0153] In the embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some communication interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0154] For another example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the part of the module, program segment, or code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0155] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0156] In addition, the functional units in the embodiments provided in the present application can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0157] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the vehicle marking method described in various embodiments of this application. The foregoing storage medium includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROM for short), random access memories (RAM for short), magnetic disks, or optical discs.

[0158] It should be noted that: similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third", etc. are only used for descriptive distinction and cannot be understood as indicating or implying relative importance.

[0159] Finally, it should be noted that: the above-mentioned embodiments are only specific implementation manners of this application, used to illustrate the technical solution of this application, rather than limiting it. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed in this application can still modify the technical solution recorded in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A method for constructing a semantic map of a parking lot, characterized in that, Including: Generating a current-frame bird's-eye view of a parking lot by stitching current-frame images collected by on-vehicle surround-view cameras; Performing semantic segmentation on the current-frame bird's-eye view to obtain a corresponding semantic segmentation map, and determining a semantic point cloud and a lane solid-line feature in a vehicle coordinate system based on the semantic segmentation map; Determining a matching relationship between the semantic point cloud in a world coordinate system corresponding to the previous-frame bird's-eye view and the semantic point cloud in a world coordinate system corresponding to the current-frame bird's-eye view through nearest-neighbor search, calculating the nearest-neighbor distance error between the matched semantic point clouds, and taking the weighted sum of the nearest-neighbor distance errors as a first constraint relationship, where the semantic point cloud in a world coordinate system corresponding to the previous-frame bird's-eye view is obtained based on the semantic point cloud in the vehicle coordinate system corresponding to the previous-frame bird's-eye view, the semantic point cloud in a world coordinate system corresponding to the current-frame bird's-eye view is obtained based on the semantic point cloud in the vehicle coordinate system corresponding to the current-frame bird's-eye view, and the weights of the nearest-neighbor distance errors corresponding to different categories of semantic point clouds are different; Determining the lane solid-line endpoints in the vehicle coordinate system corresponding to the current-frame bird's-eye view according to the lane solid-line feature in the vehicle coordinate system corresponding to the current-frame bird's-eye view; Constructing a distance error from the lane solid-line endpoints in a world coordinate system corresponding to the current-frame bird's-eye view to the lane solid-line feature in a world coordinate system corresponding to the previous-frame bird's-eye view, and taking the distance error as a second constraint relationship, where the lane solid-line endpoints in a world coordinate system corresponding to the current-frame bird's-eye view are obtained based on the lane solid-line endpoints in the vehicle coordinate system corresponding to the current-frame bird's-eye view, and the lane solid-line feature in a world coordinate system corresponding to the previous-frame bird's-eye view is obtained based on the lane solid-line feature in the vehicle coordinate system corresponding to the previous-frame bird's-eye view; Iteratively optimizing the initial pose corresponding to the current-frame bird's-eye view preset according to the first constraint relationship and the second constraint relationship to obtain the pose corresponding to the current-frame bird's-eye view; When the current-frame bird's-eye view is a key frame and the number of key frames reaches a preset value, taking the current-frame bird's-eye view as the current key frame, and taking the current key frame and the previous first preset number of key frames as a local map; Performing matching optimization on the local map by using a graph optimization method to obtain the optimized lane solid-line feature in a world coordinate system and the optimized poses corresponding to each key frame; Constructing a parking lot semantic map according to the optimized lane solid-line feature in a world coordinate system, the semantic point clouds in the vehicle coordinate system corresponding to each key frame, and the optimized poses corresponding to each key frame.

2. The construction method according to claim 1, characterized in that Determining a semantic point cloud and a lane solid-line feature in a vehicle coordinate system based on the semantic segmentation map, including: Converting the pixel coordinates of the points in the semantic segmentation map into the coordinates of the points in the vehicle coordinate system, so as to obtain the semantic point cloud in the vehicle coordinate system; Extracting a lane solid-line image from the semantic segmentation map, and performing binary processing on the lane solid-line image to obtain a lane solid-line binary image; Traverse the lane solid line binary image line by line. If n adjacent pixels all represent lane solid line pixels, set the n adjacent pixels as non-lane solid line pixels to obtain a lane solid line binary image parallel to the vehicle's forward direction, where n is a preset integer; Traverse the lane solid line binary image parallel to the vehicle's forward direction according to a preset width, and determine the number of all lane solid line pixels in each preset width area; Determine a second preset number of target preset width areas and the pixel coordinates of the lane solid line pixels in each target preset width area according to the number of all lane solid line pixels in each preset width area, where the number of lane solid line pixels in the target preset width area is the largest; Convert the pixel coordinates of the lane solid line pixels in each target preset width area into the coordinates of points in the vehicle coordinate system, and then obtain the lane solid line point clouds in the vehicle coordinate system; Solve the unknown parameters in the preset straight line function according to the lane solid line point clouds to obtain the lane solid line features in the vehicle coordinate system.

3. The construction method according to claim 1, characterized in that The method further includes: When the current frame bird's-eye view is not a key frame, or the number of key frames has not reached the preset value, return to execute the step of stitching the current frame image collected by the on-vehicle surround-view camera to generate the current frame bird's-eye view of the parking lot.

4. The construction method according to claim 1, characterized in that Adopt a graph optimization method to perform matching optimization on the local map to obtain the optimized lane solid line features in the world coordinate system and the optimized poses corresponding to each key frame, including: Determine the lane solid line endpoints in the vehicle coordinate system corresponding to each key frame according to the lane solid line features in the vehicle coordinate system corresponding to each key frame; Convert the lane solid line endpoints in the vehicle coordinate system corresponding to each key frame to the world coordinate system according to the pose corresponding to each key frame to obtain the lane solid line endpoints in the world coordinate system corresponding to each key frame; Convert the lane solid line features in the vehicle coordinate system corresponding to the earliest key frame to the world coordinate system according to the pose corresponding to the earliest key frame to obtain the lane solid line features in the world coordinate system corresponding to the earliest key frame; Construct the distance and error between the lane solid line endpoints in the world coordinate system corresponding to each key frame and the lane solid line features in the world coordinate system corresponding to the earliest key frame, and use the distance and error as the third constraint relationship; Iteratively optimize the lane solid line features in the world coordinate system corresponding to the earliest key frame and the poses corresponding to each key frame according to the third constraint relationship to obtain the optimized lane solid line features in the world coordinate system and the optimized poses corresponding to each key frame.

5. The construction method according to claim 1, characterized in that Stitch the current frame image collected by the on-vehicle surround-view camera to generate the current frame bird's-eye view of the parking lot, including: Obtain the current frame image collected by the on-vehicle surround-view camera; Use the inverse perspective transformation algorithm to stitch the current frame image to obtain the current frame bird's-eye view of the parking lot.

6. The construction method according to claim 1, characterized in that The categories of the semantic segmentation at least include: parking space lines, curbs, lane solid lines, lane dotted lines, speed bumps, and road arrows.

7. An apparatus for constructing a semantic map of a parking lot, characterized in that, Include: The stitching generation unit is used to stitch and generate the current frame bird's-eye view of the parking lot according to the current frame image collected by the vehicle surround camera; The semantic segmentation and determination unit is used to perform semantic segmentation on the current frame bird's-eye view to obtain the corresponding semantic segmentation map, and determine the semantic point cloud and the lane solid line feature in the vehicle coordinate system based on the semantic segmentation map; The matching unit is used to determine the matching relationship between the semantic point cloud in the world coordinate system corresponding to the previous frame bird's-eye view and the semantic point cloud in the world coordinate system corresponding to the current frame bird's-eye view through nearest neighbor search, calculate the nearest neighbor distance error between the matching semantic point clouds, and use the weighted sum of the nearest neighbor distance errors as the first constraint relationship. Among them, the semantic point cloud in the world coordinate system corresponding to the previous frame bird's-eye view is obtained based on the semantic point cloud in the vehicle coordinate system corresponding to the previous frame bird's-eye view, and the semantic point cloud in the world coordinate system corresponding to the current frame bird's-eye view is obtained based on the semantic point cloud in the vehicle coordinate system corresponding to the current frame bird's-eye view. The weights of the nearest neighbor distance errors corresponding to different categories of semantic point clouds are different; determine the lane solid line endpoints in the vehicle coordinate system corresponding to the current frame bird's-eye view according to the lane solid line feature in the vehicle coordinate system corresponding to the current frame bird's-eye view; construct the distance error between the lane solid line endpoints in the world coordinate system corresponding to the current frame bird's-eye view and the lane solid line feature in the world coordinate system corresponding to the previous frame bird's-eye view, and use the distance error as the second constraint relationship. Among them, the lane solid line endpoints in the world coordinate system corresponding to the current frame bird's-eye view are obtained based on the lane solid line endpoints in the vehicle coordinate system corresponding to the current frame bird's-eye view, and the lane solid line feature in the world coordinate system corresponding to the previous frame bird's-eye view is obtained based on the lane solid line feature in the vehicle coordinate system corresponding to the previous frame bird's-eye view; iteratively optimize the initial pose corresponding to the current frame bird's-eye view preset according to the first constraint relationship and the second constraint relationship to obtain the pose corresponding to the current frame bird's-eye view; The setting unit is used to use the current frame bird's-eye view as the current key frame when the current frame bird's-eye view is a key frame and the number of key frames reaches a preset value, and use the current key frame and the previous first preset number of key frames as the local map; The matching and optimization unit is used to perform matching optimization on the local map by using the method of graph optimization to obtain the optimized lane solid line feature in the world coordinate system and the optimized poses corresponding to each key frame; The map construction unit is used to construct the parking lot semantic map according to the optimized lane solid line feature in the world coordinate system, the semantic point clouds in the vehicle coordinate system corresponding to each key frame, and the optimized poses corresponding to each key frame; 8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 6 above.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are called and run by a processor, the machine-executable instructions cause the processor to run the method described in any one of claims 1 to 6 above.

Images