Method and apparatus for constructing a parking space map

Abstract

The application discloses a kind of construction method and device of parking space map, method includes: obtaining the panoramic splicing graph of vehicle around sent by acquisition equipment, identify the first parking space information and second parking space information in the panoramic splicing graph around vehicle;First parking space information is matched and fused with second parking space information, obtain the parking space attribute information in panoramic splicing graph;Real-time acquisition vehicle inertial navigation information, and determine the real-time pose information of parking space position in panoramic splicing graph relative to the starting position of vehicle in combination with parking space attribute information;Real-time pose information is used to convert the parking space position in panoramic splicing graph to the coordinate system corresponding to the starting position of vehicle, for the parking space position in the coordinate system corresponding to the starting position of vehicle is matched, constructs the parking space map suitable for current application scene.By the above method, the construction of parking space map can be realized, and the problem that the parking space map cannot be accurately constructed due to too many feature quantities in the scene description index in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to a method and apparatus for constructing parking space maps. Background Technology

[0002] In recent years, with the increasing number of vehicles in cities, the demand for autonomous parking functions has been rising, and the accuracy requirements for autonomous parking have also become increasingly stringent. To improve the accuracy and safety of the parking process, building a parking space map is an indispensable part of autonomous parking. This parking space map can be embedded in applications such as smart parking lots and electronic navigation systems to assist vehicles in autonomous parking based on the map.

[0003] In related technologies, parking space maps can be constructed based on visual semantic segmentation. This method involves filtering keyframes from images output by the acquisition device and performing semantic segmentation on these keyframes to build a local parking space map. Simultaneously, each acquired frame is given an image description, generating a scene description index library. This scene description index library and the local parking space map are then used to construct a global parking space map. However, because the scene description index library involves too many features, the computational load required by the system increases accordingly, impacting the real-time performance of vehicles during movement. This makes it difficult for the system to accurately construct a parking space map and provide vehicles with accurate parking space information. Summary of the Invention

[0004] This invention provides a method and apparatus for constructing a parking space map. It integrates first and second parking space information obtained from the recognition of a stitched image of the current frame, and utilizes the fused parking space matching information to construct the parking space map. This solves the problem that existing visual semantic segmentation-based methods cannot accurately construct parking space maps due to the excessive number of features in the scene description index. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of the present invention provide a method for constructing a parking space map, the method comprising:

[0006] The system acquires a surround view mosaic image of the vehicle sent by the acquisition device, and identifies the first parking space information and the second parking space information around the vehicle in the surround view mosaic image. The first parking space information is obtained by extracting parking lines from the surround view mosaic image, and the second parking space information is obtained by performing multi-task detection on the surround view mosaic image.

[0007] The first parking space information and the second parking space information are matched and fused to obtain the parking space attribute information in the surround view stitching image;

[0008] The vehicle inertial navigation information is acquired in real time, and the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitching image is determined in combination with the parking space attribute information. The parking space attribute information corresponds to the vehicle inertial navigation information in time.

[0009] By utilizing the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitched image, the parking space position in the surround view stitched image is transformed into the coordinate system corresponding to the vehicle's starting position. Then, the parking space position in the coordinate system corresponding to the vehicle's starting position is matched to construct a parking space map suitable for the current application scenario.

[0010] In one embodiment, identifying the first parking space information and the second parking space information around the vehicle in the surround view stitched image includes:

[0011] For the surround view stitched image, a pre-trained first network model is used to extract parking space line segmentation information from the surround view stitched image, and the parking space line segmentation information is used to perform corner coordinate detection on the surround view stitched image to obtain the first parking space information around the vehicle in the surround view stitched image;

[0012] The image information in the surround view stitched image is rasterized, and a pre-trained second network model is used to perform multi-task detection on the raster units in the surround view stitched image. The output results of the multi-task detection are then summarized to obtain the second parking space information around the vehicle in the surround view stitched image.

[0013] In one embodiment, the first network model includes a feature extraction module and a detection head module. The pre-trained first network model extracts parking space line segmentation information from the surround-view stitched image, and uses the parking space line segmentation information to perform corner coordinate detection on the surround-view stitched image to obtain first parking space information around the vehicle in the surround-view stitched image, including:

[0014] The feature extraction module is used to extract features from the surround view stitched image to obtain the image features contained in the surround view stitched image. The image features contained in the surround view stitched image are then input into the detection head module to generate a binary image mask representing the parking line segmentation information. The ground truth part of the binary image mask represents the segmented parking line information, and the rest represents the background information.

[0015] Using the parking line information segmented from the binary image mask, corner point detection is performed on the binary image mask to obtain the pixel coordinates of the parking space corner points contained in the binary image mask, and the first parking space information around the vehicle in the surround view stitching image is generated based on the pixel coordinates of the parking space corner points.

[0016] In one embodiment, before using the parking line information segmented from the binary image mask to perform corner detection on the binary image mask to obtain the pixel coordinates of the parking space corners contained in the binary image mask, and before generating the first parking space information around the vehicle in the surround view stitched image based on the pixel coordinates of the parking space corners, the method further includes:

[0017] By performing skeleton extraction processing on the parking line information segmented from the binary image mask, the multi-pixel width parking line information is refined into single-pixel width parking line information.

[0018] Accordingly, using the parking line information segmented from the binary image mask, corner detection is performed on the binary image mask to obtain the pixel coordinates of the parking space corners contained in the binary image mask. Then, based on the pixel coordinates of the parking space corners, the first parking space information around the vehicle in the surround-view stitched image is generated, including:

[0019] Using parking space line information with a single pixel width, corner point detection is performed on the binary image mask after skeleton extraction to obtain the pixel coordinates of the parking space corner points contained in the binary image mask, and the first parking space information around the vehicle in the surround view stitching image is generated based on the pixel coordinates of the parking space corner points.

[0020] In one implementation, the second network model includes detection head structures set for multiple tasks, each detection head structure corresponding to detection logic. The pre-trained second network model performs multi-task detection on the grid units in the surround-view stitched image, and summarizes the output results of the multi-task detection to obtain the second parking space information around the vehicle in the surround-view stitched image, including:

[0021] Using the detection head structure set for multiple tasks, information prediction is performed on the grid units in the surround view stitching diagram according to the detection logic corresponding to each detection head structure to obtain the output results of multiple task detections;

[0022] The output results of the multi-task detection are summarized by mutual constraints to obtain the second parking space information around the vehicle in the surround view stitching image.

[0023] In one implementation, the step of utilizing the detection head structure set for multiple tasks, and predicting information for grid units in the surround-view mosaic according to the detection logic corresponding to each detection head structure to obtain the output results of multiple task detections includes:

[0024] The first detection head structure is used to predict whether there is a parking space in the grid unit of the surround view stitching image, and the grid unit containing the parking space in the surround view stitching image is output.

[0025] For the grid units containing parking spaces in the surround view stitching image, the second detection head structure is used to predict whether there are vehicles in the parking spaces within the grid units in the surround view stitching image, and outputs the occupancy status of the parking spaces in the surround view stitching image.

[0026] For the grid units containing parking spaces in the surround view stitching image, a third detection head structure is used to predict whether the parking spaces in the grid units of the surround view stitching image contain parking space corner points, and outputs the grid units containing parking space corner points in the surround view stitching image.

[0027] For the grid units containing parking space corner points in the surround view stitching image, the fourth detection head structure is used to predict the information of the parking space corner points within the grid units in the surround view stitching image, and output the position of the parking space corner points in the surround view stitching image.

[0028] In one embodiment, the step of matching and fusing the first parking space information with the second parking space information to obtain the parking space attribute information in the surround view stitched image includes:

[0029] A fusion circle with a preset threshold radius is drawn with the corner point of the parking space in the second parking space information as the center, and it is determined whether there is a corner point of the parking space in the first parking space information within the range of the fusion circle.

[0030] If it exists, the parking space corner point in the first parking space information is used to correct the coordinates of the parking space corner point in the second parking space information, and the corrected second parking space information is used as the parking space attribute information in the surround view stitching diagram; otherwise, the second parking space information is used as the parking space attribute information in the surround view stitching diagram.

[0031] In one implementation, when the current application scenario is a closed-loop application scenario, for parking spaces within a preset area entering the mapping mode of the closed-loop application scenario, the parking space positions in the surround-view stitched image are transformed to the coordinate system corresponding to the vehicle's starting position using the real-time pose information of the parking space positions relative to the vehicle's starting position, and the parking space positions in the coordinate system corresponding to the vehicle's starting position are matched to construct a parking space map for the current application scenario, including:

[0032] The real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitching image is transformed to convert the parking space position in the surround view stitching image to the coordinate system corresponding to the vehicle's starting position. The parking space positions in the coordinate system corresponding to the vehicle's starting position are matched and fused with each other using Euclidean distance to create a local parking space map, and a serial number is set for the local parking space map.

[0033] Based on the serial number corresponding to the current local parking space map, iterate through and query historically created local parking space maps whose serial numbers are at a preset interval from the serial number corresponding to the current local parking space map. Use the iterative proximity method to match the real-time pose information of the parking space position relative to the vehicle's starting position in the historically created local parking space map and the current local parking space map. If there is a matching local parking space map within a preset distance range, then the parking space position in the current local parking space map is determined as the loop position and the closed-loop mapping has been completed.

[0034] When the current application scenario is a non-closed-loop application scenario, and the vehicle communication device does not pre-store offline map information, the mapping mode for the non-closed-loop application scenario is entered during vehicle movement. The process involves using the real-time pose information of the parking space positions in the surround-view stitched image relative to the vehicle's starting position to transform the parking space positions in the surround-view stitched image to the coordinate system corresponding to the vehicle's starting position, and matching the parking space positions in the coordinate system corresponding to the vehicle's starting position to construct a parking space map for the current application scenario, including:

[0035] Using the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitched image, the parking space positions in two adjacent frames of the surround view stitched image are matched using an iterative nearest-point method. The matched parking space positions are transformed to the coordinate system corresponding to the vehicle's starting position, and the parking space positions in the coordinate system corresponding to the vehicle's starting position are matched to complete the non-closed-loop mapping.

[0036] In one embodiment, after converting the parking space positions in the surround view stitched image to the coordinate system corresponding to the vehicle's starting position using the real-time pose information of the parking space positions relative to the vehicle's starting position, and matching the parking space positions in the coordinate system corresponding to the vehicle's starting position to construct a parking space map for the current application scenario, the method further includes:

[0037] By utilizing the relative pose information between the real-time pose information corresponding to the parking space positions in each surround view stitched image, graph optimization is performed on the parking space map of the current application scenario to update the parking space positions in the parking space map.

[0038] Secondly, embodiments of the present invention provide a parking space map construction apparatus, the apparatus comprising:

[0039] The identification unit is used to acquire the surround view mosaic image of the vehicle sent by the acquisition device, and to identify the first parking space information and the second parking space information of the vehicle in the surround view mosaic image. The first parking space information is obtained by extracting parking lines from the surround view mosaic image, and the second parking space information is obtained by performing multi-task detection on the surround view mosaic image.

[0040] The matching and fusion unit is used to match and fuse the first parking space information with the second parking space information to obtain the parking space attribute information in the surround view stitching image;

[0041] The determining unit is used to acquire vehicle inertial navigation information in real time and determine the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitching image in combination with the parking space attribute information. The parking space attribute information corresponds to the vehicle inertial navigation information in time.

[0042] The construction unit is used to convert the parking space positions in the surround view stitching image to the coordinate system corresponding to the vehicle's starting position using the real-time pose information of the parking space positions relative to the vehicle's starting position, and to match the parking space positions in the coordinate system corresponding to the vehicle's starting position to construct a parking space map for the current application scenario.

[0043] In one embodiment, the identification unit includes:

[0044] The extraction module is used to extract parking space line segmentation information from the surround view stitched image using a pre-trained first network model, and to use the parking space line segmentation information to perform corner coordinate detection on the surround view stitched image to obtain the first parking space information around the vehicle in the surround view stitched image.

[0045] The detection module is used to perform rasterization processing on the image information in the surround view stitching image, use a pre-trained second network model to perform multi-task detection on the raster units in the surround view stitching image, and summarize the output results of the multi-task detection to obtain the second parking space information around the vehicle in the surround view stitching image.

[0046] In one embodiment, the first network model includes a feature extraction module and a detection head module, wherein the extraction module includes:

[0047] A generation submodule is used to extract features from the surround view stitched image using the feature extraction module to obtain the image features contained in the surround view stitched image, and input the image features contained in the surround view stitched image to the detection head module to generate a binary image mask representing the parking line segmentation information. The ground truth part of the binary image mask represents the segmented parking line information, and the rest represents the background information.

[0048] The detection submodule is used to perform corner detection on the binary image mask using the parking line information segmented from the binary image mask, to obtain the pixel coordinates of the parking space corners contained in the binary image mask, and to generate the first parking space information around the vehicle in the surround view stitching image based on the pixel coordinates of the parking space corners.

[0049] In one embodiment, the extraction module further includes:

[0050] The processing submodule is used to perform corner detection on the binary image mask using the parking line information segmented from the binary image mask to obtain the pixel coordinates of the parking space corners contained in the binary image mask, and to generate the first parking space information around the vehicle in the surround view stitching image based on the pixel coordinates of the parking space corners. Before this is done, the submodule performs skeleton extraction processing on the parking line information segmented from the binary image mask to refine the multi-pixel width parking line information into single-pixel width parking line information.

[0051] Correspondingly, the detection submodule is also used to perform corner detection on the binary image mask after skeleton extraction using parking line information with a single pixel width, to obtain the pixel coordinates of the parking space corners contained in the binary image mask, and to generate the first parking space information around the vehicle in the surround view stitching image based on the pixel coordinates of the parking space corners.

[0052] In one embodiment, the second network model includes detection head structures set for multiple tasks, each detection head structure corresponding to detection logic, and the detection module includes:

[0053] The prediction submodule is used to use the detection head structure set for multiple tasks to predict the information of the grid units in the surround view stitching diagram according to the detection logic corresponding to each detection head structure, so as to obtain the output results of multiple task detections.

[0054] The aggregation submodule is used to aggregate the output results of the multi-task detection through mutual constraints to obtain the second parking space information around the vehicle in the surround view stitching image.

[0055] In one implementation, the prediction submodule is specifically configured to: use a first detection head structure to predict whether a grid unit in the surround view stitching image contains a parking space, and output the grid units in the surround view stitching image containing parking spaces; for the grid units in the surround view stitching image containing parking spaces, use a second detection head structure to predict whether the parking spaces in the grid units in the surround view stitching image contain vehicles, and output the occupancy status of the parking spaces in the surround view stitching image; for the grid units in the surround view stitching image containing parking spaces, use a third detection head structure to predict whether the parking spaces in the grid units in the surround view stitching image contain parking space corner points, and output the grid units in the surround view stitching image containing parking space corner points; for the grid units in the surround view stitching image containing parking space corner points, use a fourth detection head structure to predict the parking space corner points in the grid units in the surround view stitching image, and output the position of the parking space corner points in the surround view stitching image.

[0056] In one embodiment, the matching and fusion unit is specifically used to delineate a fusion circle with a preset threshold radius centered on the parking space corner point in the second parking space information, and to determine whether there is a parking space corner point in the first parking space information within the range of the fusion circle; if there is, the parking space corner point in the first parking space information is used to correct the coordinates of the parking space corner point in the second parking space information, and the corrected second parking space information is used as the parking space attribute information in the multi-frame surround view stitching image; otherwise, the second parking space information is used as the parking space attribute information in the multi-frame surround view stitching image.

[0057] In one implementation, when the current application scenario is a closed-loop application scenario, for parking spaces within a preset area entering the mapping mode of the closed-loop application scenario, the construction unit is specifically used to perform pose transformation on the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitched image, so as to transform the parking space position in the surround view stitched image to the coordinate system corresponding to the vehicle's starting position, and to use Euclidean distance to match and fuse the parking space positions in the coordinate system corresponding to the vehicle's starting position to create a local parking space map, and set a sequence number for the local parking space map; according to the sequence number corresponding to the current local parking space map, iterate through and query historically created local parking space maps whose sequence numbers are at a preset interval of a preset value, and use an iterative proximity method to match the real-time pose information of the parking space position relative to the vehicle's starting position in the historically created local parking space maps with the current local parking space map; if there is a matching local parking space map within a preset distance range, then the parking space position in the current local parking space map is determined as the loop position and the closed-loop mapping has been completed.

[0058] When the current application scenario is a non-closed-loop application scenario, and the vehicle communication device does not have pre-stored offline map information, the mapping mode of the non-closed-loop application scenario is entered during vehicle movement. Specifically, the construction unit is used to use the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitched image to match the parking space positions in two adjacent surround view stitched images using an iterative nearest-point method. The matched parking space positions are then converted to the coordinate system corresponding to the vehicle's starting position, and the parking space positions in the coordinate system corresponding to the vehicle's starting position are matched to complete the non-closed-loop mapping.

[0059] In one embodiment, the device further includes: an updating unit, configured to, after converting the parking space positions in the surround view stitching image to the coordinate system corresponding to the vehicle's starting position using real-time pose information of the parking space positions relative to the vehicle's starting position in the surround view stitching image, and matching the parking space positions in the coordinate system corresponding to the vehicle's starting position to construct a parking space map for the current application scenario, optimize the parking space map for the current application scenario using relative pose information between the real-time pose information of the parking space positions in each surround view stitching image, so as to update the parking space positions in the parking space map.

[0060] Thirdly, embodiments of the present invention provide a storage medium storing executable instructions thereon, which, when executed by a processor, cause the processor to implement the method described in the first aspect.

[0061] Fourthly, embodiments of the present invention provide a vehicle, comprising:

[0062] One or more processors;

[0063] Storage device for storing one or more programs.

[0064] Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in the first aspect.

[0065] As can be seen from the above, the parking space map construction method and apparatus provided in this embodiment of the invention can acquire a surround view stitched image of the vehicle sent by the acquisition device, identify the first parking space information and the second parking space information around the vehicle in the surround view stitched image. The surround view stitched image is an image acquired by the vehicle's fisheye camera. The first parking space information and the second parking space information are matched and fused to obtain the parking space attribute information in the surround view stitched image. The vehicle's inertial navigation information is acquired in real time, and the real-time pose information of the parking space position in the surround view stitched image relative to the preset vehicle position is determined in combination with the parking space attribute information. The parking space position in the surround view stitched image is transformed to the coordinate system corresponding to the vehicle's starting position. The parking space position in the coordinate system corresponding to the vehicle's starting position is matched to construct a parking space map for the current application scenario. Therefore, compared with the existing technology based on visual semantic segmentation, which cannot accurately construct parking space maps due to the excessive number of features in the scene description index, the embodiments of the present invention can integrate the first parking space information and the second parking space information to construct parking space maps, eliminating the influence of the environment on vehicles during travel. Furthermore, by combining with vehicle inertial navigation information, the real-time location information of vehicles can be provided, providing accurate location references for the subsequent construction of parking space maps, and thus providing accurate parking space information to vehicles.

[0066] Furthermore, the technical effects that this embodiment can also achieve include:

[0067] By matching and fusing the information of the first and second parking spaces, the attribute information of parking spaces around the vehicle can be accurately identified. This method is superior to a single identification method and can obtain more valuable parking space identification results, thus providing more accurate parking space information for autonomous parking. Furthermore, by combining the parking space attribute information with the vehicle's inertial navigation information, the parking space position is transformed into the coordinate system corresponding to the vehicle's starting position using the real-time pose information of the combined parking space position relative to the vehicle's starting position, thus constructing a parking space map for the current application scenario. For closed-loop scenarios, an offline parking space map is provided for closed-loop detection, and the offline parking space map is then corrected to improve its accuracy. For non-closed-loop scenarios, a real-time parking space map is provided, making it more convenient for users and unaffected by environmental changes.

[0068] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0070] Figure 1 A flowchart illustrating a method for constructing a parking space map according to an embodiment of the present invention;

[0071] Figures 2a-2d This is an example diagram illustrating the process of identifying parking space corner points contained in the first parking space information, provided by an embodiment of the present invention.

[0072] Figure 3 A flowchart illustrating a method for constructing a parking space map according to an embodiment of the present invention;

[0073] Figure 4 This is a block diagram of a parking space map construction device provided in an embodiment of the present invention. Detailed Implementation

[0074] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0075] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0076] This invention provides a method and apparatus for constructing parking space maps. It integrates first and second parking space information identified from a surround-view stitched image and utilizes the resulting parking space matching information to construct the parking space map. This solves the problem that existing visual semantic segmentation-based methods cannot accurately construct parking space maps due to the excessive number of features in the scene description index. In embodiments of this invention, vehicles can be equipped with in-vehicle communication devices to communicate with data acquisition devices, other vehicles, and a central server (including a cloud server, and in specific scenarios, a personal handheld terminal).

[0077] The embodiments of the present invention will be described in detail below.

[0078] Figure 1 This is a flowchart illustrating a method for constructing a parking space map according to an embodiment of the present invention. The method may include the following steps:

[0079] S100: Acquire the surround view mosaic image of the vehicle sent by the acquisition device, and identify the first parking space information and the second parking space information around the vehicle in the surround view mosaic image.

[0080] The acquisition device can be a fisheye camera, camera, or other sensor device installed on the vehicle body. The acquisition device can continuously acquire images of the vehicle's surroundings while the vehicle is in motion, and then stitch the acquired images together to form a surround view stitched image. This image is then sent to the vehicle-mounted communication device installed in the vehicle, which uses the acquired surround view stitched image to identify parking spaces and create parking space maps.

[0081] Specifically, in the process of stitching the acquired images, as one implementation method, a fisheye camera installed on the vehicle body can be used to acquire surround view images around the vehicle. The acquired surround view images are then stitched together in 2D top view using surround view stitching related algorithms to obtain the actual position information of each point in the 2D top view stitched image in cm, which is used as a frame of surround view stitched image.

[0082] The first parking space information here can be obtained by extracting parking lines from the surround view mosaic image, specifically including parking line segmentation information and parking space corner information. In the process of identifying the first parking space information around a vehicle in the surround view mosaic image, a pre-trained first network model is used to extract the parking line segmentation information from the surround view mosaic image, and the corner coordinates of the surround view mosaic image are detected using the parking line segmentation information to obtain the first parking space information around the vehicle in the surround view mosaic image. The first network model here includes a feature extraction module and a detection head module. Specifically, the feature extraction module can extract features from the surround view stitched image to obtain the image features contained in the surround view stitched image. The image features contained in the surround view stitched image are then input into the detection head module to generate a binary image mask representing the parking space line segmentation information. The ground truth part of the binary image mask represents the segmented parking space line information, and the rest represents the background information. Furthermore, the parking space line information segmented in the binary image mask is used to perform corner detection on the binary image mask to obtain the pixel coordinates of the parking space corners contained in the binary image mask. Based on the pixel coordinates of the parking space corners, the first parking space information around the vehicle in the surround view stitched image is generated.

[0083] Furthermore, to improve the recognition accuracy of the first parking space information, after the first network model outputs the binary image mask, the binary image mask can be further refined. Specifically, the parking space line information segmented from the binary image mask can be processed by skeleton extraction to refine the multi-pixel width parking space line information into single-pixel width parking space line information. This avoids the impact of the parking space line width on the recognition results of parking space corners in subsequent algorithms, thereby improving the accuracy of the recognition results.

[0084] In practical applications, the first network model can use a classic neural network model, specifically a structure with a backbone network and a composite detection head, involving a feature extractor and a detection head structure. iseNetv2 can be selected for feature extraction. For example, in the process of identifying parking space corner points contained in the first parking space information, such as... Figures 2a-2d As shown, firstly, the surround view stitched image is input into the first network model, where the feature extractor extracts image features. These extracted features are then input into the detection head structure, which generates a binary image mask of the same size as the input image. This mask serves as the semantic segmentation result for the parking lines. Further refinement of the binary image mask, i.e., skeleton extraction, abstracts parking lines of a certain width into single-pixel parking lines. Harris corner detection is then performed on the refined binary image mask. The pixel coordinates after corner detection are the parking corner coordinates (i.e., parking corner information) in the first parking information. See [link to image features after feature extraction]. Figure 2a As shown, Figure 2a The image features include vehicle features and parking space features. See the semantic segmentation results for parking line information. Figure 2b As shown, Figure 2b In the binary image mask, the ground truth represents the segmented parking space lines, and the rest is the background. See the image for the refined binary image mask. Figure 2c As shown, Figure 2c This is the parking line image after the width of pixels has been reduced to a single pixel. See the coordinates of the parking space corners for details. Figure 2d As shown, Figure 2d This is a parking space line image that includes the coordinates of the corner points of the parking spaces.

[0085] The second parking space information can be obtained by performing multi-task detection on the surround view stitched image, specifically including parking space corner information and parking space occupancy information. In the process of identifying the second parking space information around the vehicle in the surround view stitched image, the image information in the surround view stitched image can be rasterized. A pre-trained second network model is used to perform multi-task detection on the raster units in the surround view stitched image, and the output results of the multi-task detection are summarized to obtain the second parking space information around the vehicle in the surround view stitched image. The second network model includes detection head structures set for multiple tasks, each with corresponding detection logic. Specifically, the detection head structures set for multiple tasks can be used to predict information on the raster units in the surround view stitched image according to the detection logic corresponding to each detection head structure, obtaining the output results of multiple task detections. These multi-task detection output results are then summarized through mutual constraints to obtain the second parking space information around the vehicle in the surround view stitched image.

[0086] The above-mentioned rasterization process for image information in the surround view stitched image can reduce the number of parameters in the second network model. Specifically, this can be achieved by defining Region of Interest (ROI) on both sides of the vehicle body in the surround view stitched image, rasterizing the ROI, and then generating prediction information in units of individual grids, thereby improving the accuracy of the model output results.

[0087] It should be noted that the above rasterization is a downsampling process for image information. For example, if the image information of the surround view stitched image is 400*400, the rasterization process can downsample the image information of the surround view stitched image to 18*18. That is, the input image is still a surround view stitched image, only the image information of the surround view stitched image is modified.

[0088] Specifically, according to the detection logic corresponding to each detection head structure, information prediction is performed on the grid units in the surround view stitching image to obtain the output results of multiple task detections. Up to four detection head structures can be set. First, the first detection head structure is used to predict whether there is a parking space in the grid unit of the surround view stitching image, and outputs the grid units in the surround view stitching image that contain parking spaces. Then, for the grid units in the surround view stitching image that contain parking spaces, the second detection head structure is used to predict whether there are vehicles in the parking spaces in the grid units of the surround view stitching image, and outputs the occupancy status of the parking spaces in the surround view stitching image. Next, for the grid units in the surround view stitching image that contain parking spaces, the third detection head structure is used to predict whether there are parking space corner points in the grid units of the surround view stitching image, and outputs the grid units in the surround view stitching image that contain parking space corner points. Finally, for the grid units in the surround view stitching image that contain parking space corner points, the fourth detection head structure is used to predict the parking space corner points in the grid units of the surround view stitching image, and outputs the position of the parking space corner points in the surround view stitching image.

[0089] In practical applications, the second network model can use a backbone network with an additional multi-detector head structure. The backbone network can be MobileNet, which provides more reliable support for operation in vehicle conditions in terms of parameter quantity and efficiency. The detector heads are multi-task structures, with a total of four. They further perform multi-task detection on each surround view stitched image input into the second network model, and output corresponding prediction information for each detection task. Detector head 1 can predict whether there is a parking space in the grid, detector head 2 can predict whether there is a car in the parking space in the grid (occupied / available), detector head 3 can predict whether there is a parking space corner point in the grid, and detector head 4 can predict the specific location of the parking space corner point in the grid. Furthermore, a corresponding post-processing module is written to interpret the results output by the four detector heads and output the number of parking spaces, parking space corner point information, and parking space occupancy information in the region of interest on the surround view stitched image. The specific execution process of the post-processing module is as follows: First, based on the prediction information of the detection head 1, the grid containing parking spaces is determined. Then, based on the prediction information of the detection head 2, it is determined whether the parking space is available. If the current grid contains parking spaces, based on the prediction information of the detection head 3, it is determined whether the current grid contains corner points. Finally, based on the prediction information of the detection head 4, the position of the corner point of the parking space is determined.

[0090] It should be noted that the design of the above-mentioned detection head structure is a progressive process. The prediction process of each detection head can be executed based on the prediction information output by the previous detection head. By adding information from the detection heads, the parking space detection results are constrained and refined, thereby obtaining the second parking space information that is refined layer by layer.

[0091] Since adjacent frames in the surround view stitching image contain the same parking spaces during the recognition process, there will be duplicate detection of parking space corner information in the first parking space information and the second parking space information. To further improve this, non-maximum suppression calculation can be performed on the detected parking space information to merge the same parking spaces in the parking space information, retain the most likely parking space information, reduce the duplicate detection of parking space corner information, and thus accurately output the parking space information in the surround view stitching image.

[0092] Furthermore, since the model building process is all performed on a computer, the final process of running the model on the vehicle communication device requires transferring the data from the computer model to the vehicle communication device for deployment. In order to flexibly apply parking space information to the vehicle communication device, the model can be converted into a format that can run on the vehicle communication device after the model training operation is completed on the computer. This process requires reference to the information of the vehicle computing device.

[0093] S110: Match and fuse the first parking space information with the second parking space information to obtain the parking space attribute information in the surround view stitching diagram.

[0094] It is understandable that the first parking space information and the second parking space information, as two different methods, will output parking space recognition results separately, and both parking space recognition results contain parking space corner point information. To improve the accuracy of the parking space recognition results, the parking space recognition results obtained by the two methods can be matched and fused. As one implementation method of matching and fusion, the second parking space information can be set as the main one, and the first parking space information can be used to correct the parking space corner points in the second parking space information. Specifically, a fusion circle with a preset threshold radius is drawn with the parking space corner points in the second parking space information as the center. It is determined whether the parking space corner points in the first parking space information exist within the fusion circle. If they exist, the parking space corner points in the first parking space information are used to correct the coordinates of the parking space corner points in the second parking space information, and the corrected second parking space information is used as the parking space attribute information in the surround view stitching image. Otherwise, the second parking space information is used as the parking space attribute information in the surround view stitching image. Correspondingly, as another matching and fusion method, the first parking space information can also be set as the main one, and the second parking space information can be used to correct the parking space corner points in the first parking space information.

[0095] The above-described fusion matching process can be implemented by selecting a corner point of a parking space from one set of parking space information as the center to draw a fusion circle. If a corner point of another parking space from the same set of parking space information exists within the fusion circle, the match is considered successful. The corner point coordinates from the other set of parking space information are then used to correct the selected parking space information, resulting in parking space attribute information. The method of correcting the selected parking space information is not limited. One implementation method is to calculate the average of the corner point coordinates from the two sets of parking space information as the final parking space attribute information. Another implementation method is to move the corner point of the selected parking space information a preset distance along the coordinate direction of the corner point of the other set of parking space information, and use the resulting corner point coordinates as the final parking space attribute information. If no corner point of another parking space from the same set of parking space information is matched within the fusion circle, the match is considered unsuccessful, and the corner point coordinates from the selected set of parking space information are used as the final parking space attribute information. The matching results are then transmitted, or the parking space attribute information is output after a matching failure. By matching and fusing the corner points of the first and second parking spaces, more accurate corner point coordinates can be output for the surround view mosaic, thereby improving the accuracy of subsequent parking space mapping. It should be noted that the parking space corner point coordinates mentioned in this article can be understood as part or all of the parking space attribute information.

[0096] S120: Acquire vehicle inertial navigation information in real time, and combine it with the parking space attribute information to determine the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitching image.

[0097] The parking space attribute information includes the parking space location; the vehicle inertial navigation information includes the vehicle inertial navigation position and velocity; and the real-time pose information of the parking space location relative to the vehicle's starting position includes the real-time positional and angular changes of the parking space location relative to the vehicle's starting position. Specifically, the vehicle inertial navigation information can be acquired in real time, and based on the vehicle inertial navigation position and the parking space location, the real-time positional change of the parking space location relative to the vehicle's starting position in the surround view stitching image can be determined. Similarly, based on the vehicle inertial velocity and the parking space location, the real-time angular change of the parking space location relative to the vehicle's starting position in the surround view stitching image can be determined. It should be noted that the parking space location is determined by the coordinates of the parking space's corner points, and the parking space attribute information may also include other information, such as the parking space occupancy status.

[0098] For the parking space attribute information in the surround view stitching image, the parking space attribute information in the current frame of the surround view stitching image is fused with the vehicle inertial navigation information to obtain the real-time pose information of the parking space position relative to the vehicle's starting position in the current frame of the surround view stitching image. Simultaneously, it can be fused with the pose information of the parking space position relative to the vehicle's starting position in the previous frame of the surround view stitching image to obtain the relative pose information of the parking space position relative to the vehicle's position in the previous frame of the surround view stitching image. This real-time pose information and relative pose information can be represented in the form of rotation and translation matrices. In one embodiment, the pose information Tc of the current vehicle position relative to the vehicle's starting position can be obtained based on the vehicle's inertial navigation information. Simultaneously, the parking space position is transformed to the coordinate system of the current vehicle position based on the parking space attribute information in the surround view stitching image to obtain the pose information Tic of each parking space relative to the actual vehicle position. Further, the pose information Ti = Tc * Tic of each parking space relative to the vehicle's starting position can be obtained.

[0099] In practical applications, due to the high frame rate of the surround view stitched frames acquired by the camera, coupled with the influence of vehicle speed, there will inevitably be overlapping parking spaces in the surround view stitched images between frames. To avoid overlapping calculations, a frame-by-frame sampling method can usually be adopted to perform voxel filtering on the parking space attribute information in the surround view stitched images of consecutive frames, thereby reducing the feature data of parking spaces in the surround view stitched images and improving the speed of subsequent data processing.

[0100] S130: Using the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitching image, the parking space position in the surround view stitching image is converted to the coordinate system corresponding to the vehicle's starting position, and the parking space position in the coordinate system corresponding to the vehicle's starting position is matched to construct a parking space map suitable for the current application scenario.

[0101] The application scenarios can include closed-loop and non-closed-loop application scenarios. In the closed-loop application scenario, a complete map of parking spaces within a preset area is created. This requires vehicles to drive through the preset area completely before parking space recognition and mapping are performed based on the collected surround-view stitched map. The preset area can be a frequently visited area, such as a parking lot or school. The constructed parking space map can be stored as offline map information in the vehicle communication device or sent to a cloud server for sharing with other vehicles and serving as a basis for real-time parking. Specifically, the closed-loop mapping function can be set in the vehicle communication device to create a map for the preset area where vehicles frequently travel. Non-closed-loop application scenarios can perform real-time mapping of parking spaces around the current vehicle. Parking space identification and mapping can be performed based on the currently collected surround view stitched image. The constructed parking space map only represents the parking space situation near the current vehicle. The constructed parking space map can be used as the basis for real-time parking. Specifically, the real-time mapping function in the vehicle communication device can be used to perform real-time mapping of parking spaces around the current vehicle position. Non-closed-loop application scenarios are usually used when the vehicle communication device does not store offline parking space map information.

[0102] When the current application scenario is a closed-loop application scenario, for the mapping mode of parking spaces entering the closed-loop application scenario within a preset area, considering the large number of parking spaces in the area, the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitched image can be transformed to convert the parking space position in the surround view stitched image into the coordinate system corresponding to the vehicle's starting position. The parking space positions in the coordinate system corresponding to the vehicle's starting position are matched and fused using Euclidean distance to create a local parking space map. A sequence number is set for the local parking space map. Further, based on the sequence number corresponding to the current local parking space map, the historically created local parking space maps with a sequence number that is a preset value apart from the current local parking space map's sequence number are traversed and queried. An iterative proximity method is used to match the real-time pose information of the parking space position relative to the vehicle's starting position in the historically created local parking space maps with the current local parking space map. If a matching local parking space map exists within a preset distance range, the parking space position in the current local parking space map is determined as the loop-loop position and the closed-loop mapping is completed.

[0103] For closed-loop application scenarios, for example, when a vehicle is mapping a parking area, the starting position of the vehicle is first determined. After starting the closed-loop mapping function, the parking spaces within the parking area are traversed, and mapping stops when the vehicle returns to its starting position to form a closed loop. Specifically, during the mapping process, frame-by-frame mapping is used to construct local maps based on the parking space attribute information. By aggregating multiple local maps, a parking space map for a preset area is formed. For the construction of local maps, the vehicle's pose information T relative to its starting position can first be obtained from the vehicle's inertial navigation information. iFurthermore, the relative pose information T of the vehicle's starting position in the current frame surround view stitched image relative to the previous frame surround view stitched image is obtained through pose transformation. i,i-1 =T i-1 -1 *T i At this point, the parking space locations are transformed to the same coordinate system, and Euclidean distance is used to match and merge the parking space locations in the same coordinate system to create a local parking space map. A sequence number is set for each local parking space map. Simultaneously, to find the closed-loop frame surround view stitching image, for the current local parking space map, based on the sequence number corresponding to the current local parking space map, historically created local parking space maps with a preset interval of a preset value are searched within a preset distance near the current parking space location. For example, if the sequence number corresponding to the current local parking space map is 10 and the preset value is 3, local parking space maps with sequence numbers 1-7 are searched within 5m to 10m of the current local parking space map, and an iterative proximity method is used to match and determine the loop-loop position. If a match is found, it means the loop-loop position has been found. This loop-loop position is either near the vehicle's starting position or a position on the vehicle's travel segment that meets the closed-loop condition. By using the found loop-loop position as the map's start and end points, and combining multiple local parking space maps, a closed-loop parking space map is constructed.

[0104] When the current application scenario is a non-closed-loop application scenario, and the vehicle communication device does not have pre-stored offline map information, the mapping mode of the non-closed-loop application scenario is entered during vehicle movement. Specifically, the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitched image can be used to match the parking space positions in two adjacent surround view stitched images using an iterative nearest-point method. The matched parking space positions are then transformed to the coordinate system corresponding to the vehicle's starting position, and the parking space positions in the coordinate system corresponding to the vehicle's starting position are matched to complete the non-closed-loop mapping.

[0105] For example, when a vehicle finds that its in-vehicle communication device does not store offline map information in a parking lot area, the non-closed-loop mapping function is activated. At this time, there is no need to pre-set the vehicle's starting and ending positions. The constructed parking space map will be output in real time during the vehicle's movement. The parking space map is constructed by using continuous frame mapping to build the parking space attribute information. For the parking space position in each two frames of surround view stitching, the iterative nearest method is used to match the parking space position, thereby transforming the parking space position to the previous frame of surround view stitching and outputting the parking space position in the same coordinate system. Then, the parking space position is transformed to the coordinate system relative to the vehicle's starting position, and a parking space map is constructed for the vehicle's starting position.

[0106] Furthermore, since vehicle positions constantly change during the construction of the parking space map, and considering the accumulated errors in the surround-view stitched images during parking space identification and map construction, graph optimization can be performed on the parking space map to improve the accuracy of parking space positions. Specifically, after constructing the parking space map, the relative pose information between the real-time pose information corresponding to the parking space positions in each surround-view stitched image can be used to perform graph optimization on the parking space map for the current application scenario, thereby updating the parking space positions in the map. It should be noted that the parking space map requires the accumulation of multiple frames of surround-view stitched images. During the graph optimization process for the parking space map, the parking space position determined by the last frame of the surround-view stitched image can be used as the basis for optimization, and the relative pose information between the real-time poses corresponding to that parking space position can be used to perform graph optimization on the parking space map for the current application scenario. For parking space maps constructed in closed-loop scenarios, the following methods can be used: 1) The pose information of vehicles in the current local parking space map relative to vehicles in other local parking space maps; 2) The pose information of vehicles in all local parking space maps relative to their starting positions; 3) The pose information of the loopback position relative to the vehicle's starting position. 4) Node constraints are added to the constructed parking space map. Using these mutual constraints between vehicle poses, the corner information of parking spaces in the local parking space maps is converted into a map coordinate system with the vehicle's starting position as the origin, thus updating the parking space positions in the map. 5) For parking space maps constructed in non-closed-loop scenarios, the mutual constraints between vehicle poses in adjacent frames of the surround-view stitched image are used. 6) The corner position information of parking spaces in the current frame of the surround-view stitched image is converted into a map coordinate system with the vehicle's starting position as the origin, thus updating the parking space positions in the map.

[0107] In practical applications, parking space construction can be achieved by setting up a mapping system in the vehicle-mounted communication equipment. This mapping system consists of two modules: a parking space recognition module and a parking space mapping module. The parking space recognition module identifies different parking space information around the vehicle and outputs the parking space recognition result after fusing the different parking space information. It mainly involves three sub-modules: a ground marker segmentation sub-module, a parking space detection sub-module, and a fusion matching sub-module. The ground marker segmentation sub-module is used to identify the parking space segmentation information around the vehicle for each frame of the surround view stitched image. This parking space segmentation information is equivalent to the first parking space information mentioned above. The parking space detection sub-module is used to detect vehicles for each frame of the surround view stitched image. The surrounding parking space corner information, equivalent to the second parking space information mentioned above, is used in the fusion matching submodule to fuse and match parking space segmentation information and parking space corner information, outputting the parking space recognition result. The parking space mapping module is used to construct parking space maps suitable for different scenarios based on the parking space information represented by the parking space recognition result. It mainly involves three submodules: fusion submodule, closed-loop mapping submodule, and non-closed-loop mapping submodule. The fusion submodule is used to fuse the parking space information in the parking space recognition result with the vehicle inertial navigation information to obtain relative pose information. The closed-loop mapping submodule is used to map vehicles in closed-loop scenarios, and the non-closed-loop mapping submodule is used to map vehicles in non-closed-loop scenarios. For the specific process of constructing the parking space map, please refer to [link to documentation]. Figure 3 First, for the surround view stitched image, parking space line information is extracted to form landmark line segmentation results, and parking space corner points are detected to obtain parking space detection results. The landmark line segmentation results and parking space detection results are combined to obtain the parking space recognition results for the current frame. Then, the parking space recognition results for the current frame are fused with inertial navigation information, and maps are built for two scenarios. For the map closed scenario, parking spaces in every other frame are fused during the mapping process. The ICP (Iterative Closest Point) matching method is used to determine whether the loop closure position has been reached, and then the parking space map is built. At this time, the accuracy of the parking space map is high for the fixed scenario. For the map open scenario, the inertial navigation information and parking space recognition results are fused to build the parking space map. At this time, the accuracy of the parking space map is lower for the rapidly changing scenario.

[0108] It is understandable that the parking space map constructed above can include the entrance corner of each parking space, parking space type, parking space occupancy status, etc. Specifically, it can be applied to the process of autonomous parking. By locating the vehicle's position in the parking space map, the location of the vehicle in the parking space map and the location of nearby parking spaces can be provided to facilitate the planning of the vehicle's parking route.

[0109] The parking space map construction method provided in this embodiment of the invention can acquire a surround view stitched image of the vehicle sent by the acquisition device, identify the first parking space information and the second parking space information around the vehicle in the surround view stitched image. The surround view stitched image is an image acquired based on the surround view of the vehicle by a fisheye camera. The first parking space information and the second parking space information are matched and fused to obtain the parking space attribute information in the surround view stitched image. Furthermore, the vehicle inertial navigation information is acquired in real time, and the real-time pose information of the parking space position in the surround view stitched image relative to the preset vehicle position is determined in combination with the parking space attribute information. The parking space position in the surround view stitched image is transformed to the coordinate system corresponding to the starting position of the vehicle. The parking space position in the coordinate system corresponding to the starting position of the vehicle is matched to construct a parking space map for the current application scenario. Therefore, compared with the existing technology based on visual semantic segmentation, which cannot accurately construct parking space maps due to the excessive number of features in the scene description index, the embodiments of the present invention can integrate the first parking space information and the second parking space information to construct parking space maps, eliminating the influence of the environment on vehicles during travel. Furthermore, by combining with vehicle inertial navigation information, the real-time location information of vehicles can be provided, providing accurate location references for the subsequent construction of parking space maps, and thus providing accurate parking space information to vehicles.

[0110] Based on the above embodiments, another embodiment of the present invention provides a parking space map construction apparatus, such as... Figure 4 As shown, the device includes:

[0111] The identification unit 20 is used to acquire the surround view stitched image of the vehicle sent by the acquisition device, and to identify the first parking space information and the second parking space information around the vehicle in the surround view stitched image. The first parking space information is obtained by extracting parking lines from the surround view stitched image, and the second parking space information is obtained by performing multi-task detection on the surround view stitched image.

[0112] The matching and fusion unit 22 is used to match and fuse the first parking space information with the second parking space information to obtain the parking space attribute information in the surround view stitching diagram;

[0113] The determining unit 24 is used to acquire vehicle inertial navigation information in real time and determine the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitching image in combination with the parking space attribute information. The parking space attribute information corresponds to the vehicle inertial navigation information in time.

[0114] The construction unit 26 is used to convert the parking space position in the surround view stitching map to the coordinate system corresponding to the vehicle's starting position using the real-time pose information of the parking space position relative to the vehicle's starting position, and to match the parking space position in the coordinate system corresponding to the vehicle's starting position to construct a parking space map for the current application scenario.

[0115] In one embodiment, the identification unit 20 includes:

[0116] The extraction module is used to extract parking space line segmentation information from the surround view stitched image using a pre-trained first network model, and to use the parking space line segmentation information to perform corner coordinate detection on the surround view stitched image to obtain the first parking space information around the vehicle in the surround view stitched image.

[0117] The detection module is used to perform rasterization processing on the image information in the surround view stitching image, use a pre-trained second network model to perform multi-task detection on the raster units in the surround view stitching image, and summarize the output results of the multi-task detection to obtain the second parking space information around the vehicle in the surround view stitching image.

[0118] In one embodiment, the first network model includes a feature extraction module and a detection head module, wherein the extraction module includes:

[0119] A generation submodule is used to extract features from the surround view stitched image using the feature extraction module to obtain the image features contained in the surround view stitched image, and input the image features contained in the surround view stitched image to the detection head module to generate a binary image mask representing the parking line segmentation information. The ground truth part of the binary image mask represents the segmented parking line information, and the rest represents the background information.

[0120] The detection submodule is used to perform corner detection on the binary image mask using the parking line information segmented from the binary image mask, to obtain the pixel coordinates of the parking space corners contained in the binary image mask, and to generate the first parking space information around the vehicle in the surround view stitching image based on the pixel coordinates of the parking space corners.

[0121] In one embodiment, the extraction module further includes:

[0122] The processing submodule is used to perform corner detection on the binary image mask using the parking line information segmented from the binary image mask to obtain the pixel coordinates of the parking space corners contained in the binary image mask, and to generate the first parking space information around the vehicle in the surround view stitching image based on the pixel coordinates of the parking space corners. Before this process, the submodule performs skeleton extraction processing on the parking line information segmented from the binary image mask to refine the multi-pixel width parking line information into single-pixel width parking line information.

[0123] In one embodiment, the second network model includes detection head structures set for multiple tasks, each detection head structure corresponding to detection logic, and the detection module includes:

[0124] The prediction submodule is used to use the detection head structure set for multiple tasks to predict the information of the grid units in the surround view stitching diagram according to the detection logic corresponding to each detection head structure, so as to obtain the output results of multiple task detections.

[0125] The aggregation submodule is used to aggregate the output results of the multi-task detection through mutual constraints to obtain the second parking space information around the vehicle in the surround view stitching image.

[0126] In one implementation, the prediction submodule is specifically configured to: use a first detection head structure to predict whether a grid unit in the surround view stitching image contains a parking space, and output the grid units in the surround view stitching image containing parking spaces; for the grid units in the surround view stitching image containing parking spaces, use a second detection head structure to predict whether the parking spaces in the grid units in the surround view stitching image contain vehicles, and output the occupancy status of the parking spaces in the surround view stitching image; for the grid units in the surround view stitching image containing parking spaces, use a third detection head structure to predict whether the parking spaces in the grid units in the surround view stitching image contain parking space corner points, and output the grid units in the surround view stitching image containing parking space corner points; for the grid units in the surround view stitching image containing parking space corner points, use a fourth detection head structure to predict the parking space corner points in the grid units in the surround view stitching image, and output the position of the parking space corner points in the surround view stitching image.

[0127] In one embodiment, the matching and fusion unit 22 is specifically used to delineate a fusion circle with a preset threshold radius centered on the parking space corner point in the second parking space information, and to determine whether there is a parking space corner point in the first parking space information within the range of the fusion circle; if there is, the parking space corner point in the first parking space information is used to correct the coordinates of the parking space corner point in the second parking space information, and the corrected second parking space information is used as the parking space attribute information in the multi-frame surround view stitching image; otherwise, the second parking space information is used as the parking space attribute information in the multi-frame surround view stitching image.

[0128] In one implementation, when the current application scenario is a closed-loop application scenario, for parking spaces within a preset area entering the mapping mode of the closed-loop application scenario, the construction unit 26 is specifically used to perform pose transformation on the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitching map, so as to transform the parking space position in the surround view stitching map to the coordinate system corresponding to the vehicle's starting position, and to perform mutual matching and fusion of the parking space positions in the coordinate system corresponding to the vehicle's starting position using Euclidean distance to create a local parking space map, and to set a sequence number for the local parking space map; according to the sequence number corresponding to the current local parking space map, to traverse and query historically created local parking space maps whose sequence number is at a preset interval with the sequence number corresponding to the current local parking space map, and to use an iterative proximity method to match the real-time pose information of the parking space position relative to the vehicle's starting position in the historically created local parking space maps with the current local parking space map; if there is a matching local parking space map within a preset distance range, then the parking space position in the current local parking space map is determined as the loop position and the closed-loop mapping has been completed;

[0129] When the current application scenario is a non-closed-loop application scenario, and the vehicle communication device does not have pre-stored offline map information, the construction unit 26 enters the mapping mode of the non-closed-loop application scenario during vehicle operation. Specifically, it is used to use the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitching image to match the parking space positions in two adjacent frames of the surround view stitching image using an iterative nearest-point method. The matched parking space positions are then converted to the coordinate system corresponding to the vehicle's starting position, and the parking space positions in the coordinate system corresponding to the vehicle's starting position are matched to complete the non-closed-loop mapping.

[0130] In one embodiment, the device further includes: an updating unit, configured to, after converting the parking space positions in the surround view stitching image to the coordinate system corresponding to the vehicle's starting position using real-time pose information of the parking space positions relative to the vehicle's starting position in the surround view stitching image, and matching the parking space positions in the coordinate system corresponding to the vehicle's starting position to construct a parking space map for the current application scenario, optimize the parking space map for the current application scenario using relative pose information between the real-time pose information of the parking space positions in each surround view stitching image, so as to update the parking space positions in the parking space map.

[0131] Based on the above method embodiments, another embodiment of the present invention provides a storage medium storing executable instructions thereon, which, when executed by a processor, cause the processor to implement the above method.

[0132] Based on the above embodiments, another embodiment of the present invention provides a vehicle, including:

[0133] One or more processors;

[0134] Storage device for storing one or more programs.

[0135] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described above. The vehicle can be a non-autonomous vehicle or an autonomous vehicle.

[0136] The above system and device embodiments correspond to the method embodiments and have the same technical effects. For detailed descriptions, please refer to the method embodiments. The device embodiments are derived based on the method embodiments; detailed descriptions can be found in the method embodiments section, and will not be repeated here. Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0137] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a parking space map, characterized in that, The method includes: The system acquires a surround view mosaic image of the vehicle sent by the acquisition device, and identifies the first parking space information and the second parking space information around the vehicle in the surround view mosaic image. The first parking space information is obtained by extracting parking lines from the surround view mosaic image, and the second parking space information is obtained by performing multi-task detection on the surround view mosaic image. The first parking space information and the second parking space information are matched and fused to obtain the parking space attribute information in the surround view stitching image; The vehicle inertial navigation information is acquired in real time, and the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitching image is determined in combination with the parking space attribute information. The parking space attribute information corresponds to the vehicle inertial navigation information in time. Using the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitched image, the parking space position in the surround view stitched image is transformed into the coordinate system corresponding to the vehicle's starting position, and the parking space position in the coordinate system corresponding to the vehicle's starting position is matched to construct a parking space map suitable for the current application scenario. The first network model includes a feature extraction module and a detection head module, which identify the first parking space information around the vehicle in the surround-view stitched image, including: The feature extraction module is used to extract features from the surround view stitched image to obtain the image features contained in the surround view stitched image. The image features contained in the surround view stitched image are then input into the detection head module to generate a binary image mask representing the parking line segmentation information. The ground truth part of the binary image mask represents the segmented parking line information, and the rest represents the background information. By performing skeleton extraction processing on the parking line information segmented from the binary image mask, the multi-pixel width parking line information is refined into single-pixel width parking line information. Using parking space line information with a single pixel width, corner point detection is performed on the binary image mask after skeleton extraction to obtain the pixel coordinates of the parking space corner points contained in the binary image mask, and the first parking space information around the vehicle in the surround view stitching image is generated based on the pixel coordinates of the parking space corner points.

2. The method as described in claim 1, characterized in that, The identification of the second parking space information around the vehicle in the surround view stitched image includes: The image information in the surround view stitched image is rasterized, and a pre-trained second network model is used to perform multi-task detection on the raster units in the surround view stitched image. The output results of the multi-task detection are then summarized to obtain the second parking space information around the vehicle in the surround view stitched image.

3. The method as described in claim 2, characterized in that, The second network model includes detection head structures set for multiple tasks, each with corresponding detection logic. The pre-trained second network model performs multi-task detection on the grid units in the surround-view stitched image and summarizes the output results of the multi-task detection to obtain the second parking space information around the vehicle in the surround-view stitched image, including: Using the detection head structure set for multiple tasks, information prediction is performed on the grid units in the surround view stitching diagram according to the detection logic corresponding to each detection head structure to obtain the output results of multiple task detections; The output results of the multi-task detection are summarized by mutual constraints to obtain the second parking space information around the vehicle in the surround view stitching image.

4. The method as described in claim 3, characterized in that, The method utilizes the detection head structure designed for multiple tasks, and predicts information for the grid units in the surround view mosaic according to the detection logic corresponding to each detection head structure to obtain the output results of multiple task detections, including: The first detection head structure is used to predict whether there is a parking space in the grid unit of the surround view stitching image, and the grid unit containing the parking space in the surround view stitching image is output. For the grid units containing parking spaces in the surround view stitching image, the second detection head structure is used to predict whether there are vehicles in the parking spaces within the grid units in the surround view stitching image, and outputs the occupancy status of the parking spaces in the surround view stitching image. For the grid units containing parking spaces in the surround view stitching image, a third detection head structure is used to predict whether the parking spaces in the grid units of the surround view stitching image contain parking space corner points, and outputs the grid units containing parking space corner points in the surround view stitching image. For the grid units containing parking space corner points in the surround view stitching image, the fourth detection head structure is used to predict the information of the parking space corner points within the grid units in the surround view stitching image, and output the position of the parking space corner points in the surround view stitching image.

5. The method according to any one of claims 1-4, characterized in that, The step of matching and fusing the first parking space information with the second parking space information to obtain the parking space attribute information in the surround view stitched image includes: A fusion circle with a preset threshold radius is drawn with the corner point of the parking space in the second parking space information as the center, and it is determined whether there is a corner point of the parking space in the first parking space information within the range of the fusion circle. If it exists, the parking space corner point in the first parking space information is used to correct the coordinates of the parking space corner point in the second parking space information, and the corrected second parking space information is used as the parking space attribute information in the surround view stitching diagram; otherwise, the second parking space information is used as the parking space attribute information in the surround view stitching diagram.

6. The method according to any one of claims 1-4, characterized in that, When the current application scenario is a closed-loop application scenario, for parking spaces within a preset area entering the mapping mode of the closed-loop application scenario, the parking space positions in the surround-view stitched image are transformed to the coordinate system corresponding to the vehicle's starting position using the real-time pose information of the parking space positions relative to the vehicle's starting position in the surround-view stitched image, and the parking space positions in the coordinate system corresponding to the vehicle's starting position are matched to construct a parking space map for the current application scenario, including: The real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitching image is transformed to convert the parking space position in the surround view stitching image to the coordinate system corresponding to the vehicle's starting position. The parking space positions in the coordinate system corresponding to the vehicle's starting position are matched and fused with each other using Euclidean distance to create a local parking space map, and a serial number is set for the local parking space map. Based on the serial number corresponding to the current local parking space map, iterate through and query historically created local parking space maps whose serial numbers are at a preset interval from the serial number corresponding to the current local parking space map. Use the iterative proximity method to match the real-time pose information of the parking space position relative to the vehicle's starting position in the historically created local parking space map and the current local parking space map. If there is a matching local parking space map within a preset distance range, then the parking space position in the current local parking space map is determined as the loop position and the closed-loop mapping has been completed. When the current application scenario is a non-closed-loop application scenario, and the vehicle communication device does not pre-store offline map information, the mapping mode for the non-closed-loop application scenario is entered during vehicle movement. The process involves using the real-time pose information of the parking space positions in the surround-view stitched image relative to the vehicle's starting position to transform the parking space positions in the surround-view stitched image to the coordinate system corresponding to the vehicle's starting position, and matching the parking space positions in the coordinate system corresponding to the vehicle's starting position to construct a parking space map for the current application scenario, including: Using the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitched image, the parking space positions in two adjacent frames of the surround view stitched image are matched using an iterative nearest-point method. The matched parking space positions are transformed to the coordinate system corresponding to the vehicle's starting position, and the parking space positions in the coordinate system corresponding to the vehicle's starting position are matched to complete the non-closed-loop mapping.

7. The method according to any one of claims 1-4, characterized in that, After converting the parking space positions in the surround view stitched image to the coordinate system corresponding to the vehicle's starting position using the real-time pose information of the parking space positions relative to the vehicle's starting position, and matching the parking space positions in the coordinate system corresponding to the vehicle's starting position to construct a parking space map for the current application scenario, the method further includes: By utilizing the relative pose information between the real-time pose information corresponding to the parking space positions in each surround view stitched image, graph optimization is performed on the parking space map of the current application scenario to update the parking space positions in the parking space map.

8. A device for constructing a parking space map, characterized in that, The device includes: The identification unit is used to acquire the surround view mosaic image of the vehicle sent by the acquisition device, and to identify the first parking space information and the second parking space information of the vehicle in the surround view mosaic image. The first parking space information is obtained by extracting parking lines from the surround view mosaic image, and the second parking space information is obtained by performing multi-task detection on the surround view mosaic image. The matching and fusion unit is used to match and fuse the first parking space information with the second parking space information to obtain the parking space attribute information in the surround view stitching image; The determining unit is used to acquire vehicle inertial navigation information in real time and determine the real-time pose information of the parking space position relative to the vehicle's starting position in the surround view stitching image in combination with the parking space attribute information. The parking space attribute information corresponds to the vehicle inertial navigation information in time. The construction unit is used to convert the parking space position in the surround view stitching image to the coordinate system corresponding to the vehicle's starting position using the real-time pose information of the parking space position relative to the vehicle's starting position, and to match the parking space position in the coordinate system corresponding to the vehicle's starting position to construct a parking space map for the current application scenario. The identification unit includes: The extraction module is used to extract parking space line segmentation information from the surround view stitched image using a pre-trained first network model, and to use the parking space line segmentation information to perform corner coordinate detection on the surround view stitched image to obtain the first parking space information around the vehicle in the surround view stitched image. The first network model includes a feature extraction module and a detection head module. The extraction module includes: A generation submodule is used to extract features from the surround view stitched image using the feature extraction module to obtain the image features contained in the surround view stitched image, and input the image features contained in the surround view stitched image to the detection head module to generate a binary image mask representing the parking line segmentation information. The ground truth part of the binary image mask represents the segmented parking line information, and the rest represents the background information. The detection submodule is used to perform corner detection on the binary image mask using the parking line information segmented in the binary image mask, to obtain the pixel coordinates of the parking space corners contained in the binary image mask, and to generate the first parking space information around the vehicle in the surround view stitching image based on the pixel coordinates of the parking space corners. The extraction module further includes: The processing submodule is used to perform corner detection on the binary image mask using the parking line information segmented from the binary image mask to obtain the pixel coordinates of the parking space corners contained in the binary image mask, and to generate the first parking space information around the vehicle in the surround view stitching image based on the pixel coordinates of the parking space corners. Before this is done, the submodule performs skeleton extraction processing on the parking line information segmented from the binary image mask to refine the multi-pixel width parking line information into single-pixel width parking line information. Correspondingly, the detection submodule is also used to perform corner detection on the binary image mask after skeleton extraction using parking line information with a single pixel width, to obtain the pixel coordinates of the parking space corners contained in the binary image mask, and to generate the first parking space information around the vehicle in the surround view stitching image based on the pixel coordinates of the parking space corners.

Images