Semantic mapping method, device, electronic device and storage medium in parking garage environment

Abstract

The present application relates to a semantic mapping method, device, electronic device and storage medium in a parking garage environment, the method comprising: obtaining vehicle posture information according to sensor data; splicing multiple surround images, detecting lane line semantic information and parking space line semantic information through semantic segmentation algorithm, marking corresponding image pixels, extracting lane line centerline and parking space line centerline images; calculating the vehicle posture information of the surround image according to the vehicle posture information and the time of the surround image, obtaining lane line point clouds and parking space line point clouds at different times based on the vehicle posture information, lane line centerline and parking space line centerline images of the surround image, splicing to obtain an initial semantic point cloud map; downsampling to create a three-dimensional voxel grid, the center of gravity of all point clouds in the three-dimensional voxel grid is used as a new semantic point, and a semantic point cloud map in the parking garage environment is obtained according to the new semantic point. The problem of large computational complexity, high processor computing power requirements and high cost in the mapping method in the related art is solved.

Description

Technical Field

[0001] The present application relates to the field of autonomous parking technology, and in particular to a semantic mapping method, device, electronic device and storage medium in a parking garage environment. Background Art

[0002] As part of the assisted driving function, autonomous parking technology is being integrated into more and more cars to help drivers park their cars at the target location, reducing the technical requirements for drivers and improving the driving experience. In parking garages, especially underground parking garages, due to the lack of high-precision map information, drivers are required to drive the car to collect parking garage environment information and create an environment map for subsequent autonomous parking.

[0003] In the related art, the method of creating a map is mainly to directly use a feature point map obtained from an image or a semantic vector map obtained from a semantic vector.

[0004] However, the semantic vector map obtained by directly using the feature point map obtained from the image has a large amount of calculation. At the same time, the point cloud data obtained directly using the semantic image is large in amount, which requires high computing power of the processor. The method of using semantic vectors to obtain the semantic vector map has a high sensor hardware cost and a large amount of measurement. Summary of the invention

[0005] The present application provides a semantic mapping method, device, electronic device and storage medium in a parking garage environment, which solves the problems of large computational complexity, high processor computing power requirements and high cost in the related art mapping methods, improves the processing speed, reduces the occurrence of erroneous semantic detection results and the requirements for deep learning to detect semantic features.

[0006] The first aspect of the present application provides a semantic mapping method in a parking garage environment, comprising the following steps: acquiring sensor data of a vehicle, and calculating the position and posture information of the vehicle relative to a starting point based on the sensor data; acquiring multiple surround-view images around the vehicle, and after splicing the multiple surround-view images, detecting lane line semantic information and parking space line semantic information from the spliced ​​images through a preset semantic segmentation algorithm, and respectively marking the corresponding image pixels of the lane line semantic information and the parking space line semantic information, and then extracting lane line centerline and parking space line centerline images; based on the position and posture information of the vehicle relative to the starting point and the The method further comprises the following steps: calculating the vehicle posture information corresponding to the multiple surround view images at the time of the multiple surround view images, and obtaining the lane line point clouds and the storage space line point clouds at different times based on the vehicle posture information corresponding to the multiple surround view images and the lane line centerline and storage space line centerline images, and splicing the lane line and storage space line point clouds at different times to obtain an initial semantic point cloud map; downsampling the point clouds in the initial semantic point cloud map according to the lane lines and storage space lines, and creating a three-dimensional voxel grid, and taking the centroid of all point clouds in the three-dimensional voxel grid as a new semantic point, and obtaining a semantic point cloud map in a parking garage environment according to the new semantic point.

[0007] According to the above technical means, the problems of large computational complexity, high processor computing power requirements and high cost in related mapping methods are solved, the processing speed is improved, the occurrence of erroneous semantic detection results and the requirements for deep learning to detect semantic features are reduced.

[0008] Furthermore, the lane line point clouds and storage position line point clouds at different times are obtained based on the vehicle posture information corresponding to the multiple surround view pictures and the lane line centerline and storage position line centerline pictures, including: selecting the pixel points of the lane lines and the pixel points of the storage position lines based on the vehicle posture information corresponding to the multiple surround view pictures and the lane line centerline and storage position line centerline pictures; calculating the lane line coordinates and the coordinates of the storage position lines at different times in the vehicle body coordinate system according to the correspondence between the pixel points of the lane lines and the pixel points of the storage position lines and the spatial positions; and obtaining the lane line point clouds and the storage position line point clouds at different times according to the lane line coordinates and the coordinates of the storage position lines.

[0009] According to the above technical means, only the lane line and storage location line information in the surround stitching map is used, which reduces the requirements for deep learning detection semantic features.

[0010] Furthermore, after marking the corresponding image pixels of the lane line semantic information and the storage position line semantic information respectively, extracting the lane line centerline and storage position line centerline image, including: marking the first image pixel of the lane line semantic information, and marking the second image pixel of the storage position line semantic information to obtain the lane line and storage position line semantic image; marking the boundary pixel points of multiple first preset positions and eight pixel points around each pixel point in the semantic pixels of the lane line and storage position line semantic image, deleting the marked pixel points to obtain the initial lane line and storage position line semantic image; marking the boundary pixel points of multiple second preset positions and eight pixel points around each pixel point in the semantic pixels of the initial lane line and storage position line semantic image, deleting the marked pixel points to obtain the final lane line and storage position line semantic image; using the final lane line and storage position line semantic image as a new lane line and storage position line semantic image for iterative processing until the preset iteration condition is met to obtain the lane line centerline and storage position line centerline image.

[0011] According to the above technical means, the semantic pixel points of redundant pixels are deleted, the amount of pixel data that needs to be processed in the image is reduced, and the processing speed is improved.

[0012] Furthermore, the method of calculating the vehicle posture information corresponding to the multiple surround-view pictures based on the posture information of the vehicle relative to the starting point and the time of the multiple surround-view pictures includes: based on a preset interpolation method, calculating the vehicle posture information corresponding to the multiple surround-view pictures based on the posture information of the vehicle relative to the starting point and the time of the multiple surround-view pictures, wherein the preset interpolation method includes linear interpolation of position and / or spherical interpolation of quaternions representing direction.

[0013] According to the above technical means, the interpolation method is used to calculate the vehicle's posture information. The algorithm is simple and the calculation speed is fast, which improves the processing speed.

[0014] The second aspect of the present application provides a semantic mapping device in a parking garage environment, including: a first acquisition module, used to acquire sensor data of a vehicle, and calculate the position information of the vehicle relative to a starting point based on the sensor data; an extraction module, used to acquire multiple surround view images around the vehicle, and after splicing the multiple surround view images, detect lane line semantic information and parking space line semantic information from the spliced ​​images through a preset semantic segmentation algorithm, and respectively mark the corresponding image pixels of the lane line semantic information and the parking space line semantic information, and then extract the lane line centerline and parking space line centerline images; a splicing module, used to obtain the position information of the vehicle relative to the starting point according to the position of the vehicle relative to the starting point, and then calculate the position information of the vehicle relative to the starting point according to the position of the vehicle relative to the starting point. The method comprises the following steps: calculating the vehicle posture information corresponding to the multiple surround view pictures based on the vehicle posture information corresponding to the multiple surround view pictures and the time of the lane line centerline and the storage space line centerline pictures, and obtaining the lane line point cloud and the storage space line point cloud at different times based on the vehicle posture information corresponding to the multiple surround view pictures and the lane line centerline and the storage space line centerline pictures, and splicing the lane line and storage space line point clouds at different times to obtain an initial semantic point cloud map; a second acquisition module is used to downsample the point clouds in the initial semantic point cloud map according to the lane lines and the storage space lines, and create a three-dimensional voxel grid, and use the centroid of all point clouds in the three-dimensional voxel grid as a new semantic point, and obtain a semantic point cloud map in the parking garage environment according to the new semantic point.

[0015] Furthermore, the stitching module is specifically used to: select the pixel points of the lane line and the pixel points of the storage line based on the vehicle posture information corresponding to the multiple surround images and the lane line centerline and storage line centerline images; calculate the lane line coordinates and the coordinates of the storage line at different times in the vehicle body coordinate system according to the correspondence between the lane line pixel points and the storage line pixel points and the spatial position; obtain the lane line point cloud and the storage line point cloud at different times according to the lane line coordinates and the storage line coordinates.

[0016] Furthermore, the extraction module is specifically used to: mark the first image pixels of the lane line semantic information, and mark the second image pixels of the storage position line semantic information to obtain a lane line and storage position line semantic image; mark the boundary pixel points of multiple first preset positions in the semantic pixels of the lane line and storage position line semantic image and eight pixel points around each pixel point, delete the marked pixel points to obtain an initial lane line and storage position line semantic image; mark the boundary pixel points of multiple second preset positions in the semantic pixels of the initial lane line and storage position line semantic image and eight pixel points around each pixel point, delete the marked pixel points to obtain a final lane line and storage position line semantic image; use the final lane line and storage position line semantic image as a new lane line and storage position line semantic image for iterative processing until the preset iteration condition is met to obtain the lane line centerline and storage position line centerline image.

[0017] Furthermore, the stitching module is also used to: based on a preset interpolation method, calculate the vehicle posture information corresponding to the multiple surround-view pictures according to the posture information of the vehicle relative to the starting point and the time of the multiple surround-view pictures, wherein the preset interpolation method includes linear interpolation of position and / or spherical interpolation of quaternions representing direction.

[0018] A third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the semantic mapping method in a parking garage environment as described in the above embodiment.

[0019] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the semantic mapping method in a parking garage environment as described in the above embodiment.

[0020] Therefore, the present application obtains vehicle posture information according to sensor data, splices multiple surround images, detects lane line semantic information and storage line semantic information through semantic segmentation algorithm, marks corresponding image pixels, extracts lane line centerline and storage line centerline images, calculates vehicle posture information of surround image according to vehicle posture information and the time of surround image, obtains lane line point cloud and storage line point cloud at different times based on vehicle posture information, lane line centerline and storage line centerline images of surround image, splices to obtain initial semantic point cloud map, downsamples point cloud in initial semantic point cloud map according to lane line and storage line, creates three-dimensional voxel grid, takes the center of gravity of all point clouds in three-dimensional voxel grid as new semantic point, and obtains semantic point cloud map in parking garage environment according to new semantic point. Therefore, the problem of large computational amount, high processor computing power requirement and high cost in related technologies is solved, processing speed is improved, and erroneous semantic detection results and requirements for deep learning to detect semantic features are reduced.

[0021] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0023] Figure 1 A flowchart of a semantic mapping method in a parking garage environment provided according to an embodiment of the present application;

[0024] Figure 2A flowchart of a semantic mapping method in a parking garage environment according to an embodiment of the present application;

[0025] Figure 3 A schematic diagram of an image after semantic segmentation of a surround view image mosaic according to an embodiment of the present application;

[0026] Figure 4 is a schematic diagram of a semantic segmentation image after centerline extraction according to an embodiment of the present application;

[0027] Figure 5 A schematic diagram of a semantic segmentation point cloud map created according to an embodiment of the present application;

[0028] Figure 6 It is a block diagram of a semantic mapping device in a parking garage environment according to an embodiment of the present application;

[0029] Figure 7 Schematic diagram of the structure of an electronic device according to an embodiment of the present application.

[0030] Explanation of the accompanying reference numerals: 10 - semantic mapping device in a parking garage environment, 100 - first acquisition module, 200 - extraction module, 300 - splicing module, 400 - second acquisition module, 703 - communication interface, 701 - memory, 702 - processor. DETAILED DESCRIPTION

[0031] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0032] The following describes the semantic mapping method, device, electronic device and storage medium in the parking garage environment of the embodiment of the present application with reference to the accompanying drawings. In response to the problems mentioned in the above background technology that the mapping methods in the related technologies have large computational complexity, high processor computing power requirements and high costs, the present application provides a semantic mapping method in a parking garage environment. In this method, the sensor data of the vehicle is obtained, and the position information of the vehicle relative to the starting point is calculated based on the sensor data; multiple surround view pictures around the vehicle are obtained, and after the multiple surround view pictures are spliced, the lane line semantic information and the storage position line semantic information are detected from the spliced ​​pictures through a preset semantic segmentation algorithm, and after marking the corresponding picture pixels of the lane line semantic information and the storage position line semantic information respectively, the lane line centerline and the storage position line centerline map are extracted. The vehicle posture information corresponding to the multiple surround view images is calculated based on the posture information of the vehicle relative to the starting point and the time of the multiple surround view images, and the lane line point cloud and the storage line point cloud at different times are obtained based on the vehicle posture information corresponding to the multiple surround view images, the lane line centerline and the storage line centerline images, and the lane line and storage line point cloud at different times are spliced ​​to obtain the initial semantic point cloud map; the point cloud in the initial semantic point cloud map is downsampled according to the lane line and the storage line, and a three-dimensional voxel grid is created, and the center of gravity of all point clouds in the three-dimensional voxel grid is used as a new semantic point, and a semantic point cloud map in the parking garage environment is obtained according to the new semantic point. In this way, the problems of large calculation amount, high computing power requirements and high cost of the mapping method in the related technology are solved, the processing speed is improved, and the erroneous semantic detection results and the requirements for deep learning to detect semantic features are reduced.

[0033] Specifically, Figure 1 A flowchart of a semantic mapping method in a parking garage environment provided in an embodiment of the present application.

[0034] like Figure 1 As shown, the semantic mapping method in the parking garage environment includes the following steps:

[0035] In step S101, sensor data of the vehicle is acquired, and the position information of the vehicle relative to the starting point is calculated based on the sensor data.

[0036] The sensors may include but are not limited to: wheel speed sensors, inertial measurement units (IMUs), and navigation devices. Figure 2 As shown, the embodiment of the present application measures the wheel speed data of the vehicle through a wheel speed sensor, and detects the angular velocity and acceleration data of the vehicle according to the inertial measurement unit IMU of the vehicle.

[0037] The vehicle's posture information includes the vehicle's position information and the vehicle's attitude information. The embodiment of the present application establishes a vehicle coordinate system to obtain the vehicle's position information on the X-axis, Y-axis, and Z-axis, and obtains the vehicle's attitude information such as the vehicle's pitch angle, heading angle, and roll angle through sensors. Relevant technicians in this field can calculate the vehicle's posture at the starting point through an integral algorithm.

[0038] In step S102, multiple surround view pictures around the vehicle are obtained and stitched together, and then the lane line semantic information and the storage line semantic information are detected from the stitched pictures through a preset semantic segmentation algorithm. After marking the corresponding picture pixels of the lane line semantic information and the storage line semantic information respectively, the lane line centerline and the storage line centerline pictures are extracted.

[0039] Specifically, the embodiment of the present application uses a surround view camera installed on the vehicle to capture surround view images around the vehicle, splices the surround view images according to the internal and external parameters after the surround view camera is calibrated, detects semantic information such as lane lines and storage lines through a preset semantic segmentation algorithm, marks the corresponding image pixels according to the detection results, and sets the values ​​of pixels without detected semantic information to background values. Among them, the surround view image splicing map and semantic segmentation map are as follows: Figure 3 shown.

[0040] Further, in some embodiments, after marking the corresponding image pixels of the lane line semantic information and the storage position line semantic information respectively, the lane line centerline and the storage position line centerline image are extracted, including: marking the first image pixel of the lane line semantic information, and marking the second image pixel of the storage position line semantic information to obtain the lane line and storage position line semantic image; marking the boundary pixel points of multiple first preset positions and eight pixel points around each pixel point in the semantic pixels of the lane line and storage position line semantic image, deleting the marked pixel points to obtain the initial lane line and storage position line semantic image; marking the boundary pixel points of multiple second preset positions and eight pixel points around each pixel point in the semantic pixels of the initial lane line and storage position line semantic image, deleting the marked pixel points to obtain the final lane line and storage position line semantic image; using the final lane line and storage position line semantic image as the new lane line and storage position line semantic image for iterative processing until the preset iteration condition is met to obtain the lane line centerline and storage position line centerline image.

[0041] Among them, the boundary pixel points of the multiple first preset positions are the boundary pixel points at the upper left corner and the right and bottom sides, and the boundary pixel points of the multiple second preset positions are the boundary pixel points at the lower right corner and the left and top sides.

[0042] Specifically, the image pixels corresponding to the lane line semantic information and the storage position line semantic information are marked, the first image pixels are marked for the lane line semantic information, and the second image pixels are marked for the storage position line semantic information. The center line extraction algorithm is used, including marking the upper left corner, right side and bottom boundary pixels in the semantic pixels of the lane line and storage position line semantic images, and marking the lower right corner, left side and top boundary pixels in the semantic pixels of the lane line and storage position line semantic images, as well as eight pixels around each pixel, deleting the marked pixels to obtain the final lane line and storage position line semantic images, and judging whether the values ​​of these eight pixels meet the preset iteration conditions. When the preset iteration conditions are met, the pixel points to be processed are added to the deletion sequence. When there are no pixel points to be added to the deletion sequence, it means that the redundant pixel points have been processed, and a line with a thickness of only one pixel is obtained, and the lane line center line and storage position line center line images are obtained, such as Figure 4 shown.

[0043] In step S103, the vehicle posture information corresponding to the multiple surround view images is calculated according to the posture information of the vehicle relative to the starting point and the time of the multiple surround view images, and the lane line point cloud and storage location line point cloud at different times are obtained based on the vehicle posture information corresponding to the multiple surround view images, the lane line center line and the storage location line center line images, and the lane line and storage location line point clouds at different times are spliced ​​to obtain the initial semantic point cloud map.

[0044] Optionally, in some embodiments, the vehicle posture information corresponding to the multiple surround view pictures is calculated based on the posture information of the vehicle relative to the starting point and the time of the multiple surround view pictures, including: based on a preset interpolation method, the vehicle posture information corresponding to the multiple surround view pictures is calculated based on the posture information of the vehicle relative to the starting point and the time of the multiple surround view pictures, wherein the preset interpolation method includes linear interpolation of position and / or spherical interpolation of quaternions representing direction.

[0045] Specifically, the embodiment of the present application obtains the time when the surround-view camera takes multiple surround-view pictures, and based on the calculated posture information of the vehicle relative to the starting point, uses the linear interpolation method of the position or the spherical interpolation method of the quaternion representing the direction, or the linear interpolation method of the position and the spherical interpolation method of the quaternion representing the direction, calculates the vehicle posture information corresponding to the multiple surround-view pictures.

[0046] Furthermore, in some embodiments, lane line point clouds and storage location line point clouds at different times are obtained based on the vehicle posture information, lane line centerline and storage location line centerline images corresponding to multiple surround view images, including: selecting pixel points of lane lines and pixel points of storage location lines based on the vehicle posture information, lane line centerline and storage location line centerline images corresponding to multiple surround view images; calculating the coordinates of lane lines and coordinates of storage location lines at different times in the vehicle body coordinate system according to the correspondence between the pixel points of lane lines and the pixel points of storage location lines and the spatial positions; obtaining lane line point clouds and storage location line point clouds at different times according to the lane line coordinates and the coordinates of storage location lines.

[0047] Specifically, based on the vehicle posture information corresponding to multiple surround-view pictures obtained by a preset interpolation method and the pictures of the lane center lines and the storage space line center lines, the lane line pixel points and the storage space line pixel points are selected respectively, and according to the correspondence between the pixels and the spatial positions, the pixels are converted to the spatial positions in the vehicle body coordinate system, and the coordinates of the lane lines and the storage space lines in the vehicle body coordinate system are calculated to obtain the lane line and storage space line point clouds respectively.

[0048] Furthermore, the surround images collected at different times are processed to obtain the lane line and storage location line point clouds at the same time, and are spliced ​​together to obtain the initial semantic point cloud map.

[0049] In step S104, the point clouds in the initial semantic point cloud map are downsampled according to the lane lines and the parking space lines, and a three-dimensional voxel grid is created. The centroids of all point clouds in the three-dimensional voxel grid are used as new semantic points, and a semantic point cloud map in the parking garage environment is obtained based on the new semantic points.

[0050] It should be understood that the acquired lane line point cloud and parking space line point cloud are downsampled according to the lane line and parking space line respectively, and the VoxelGrid method of PCL (Point Cloud Library) is used to create a three-dimensional voxel grid. Within the set grid size, the center of gravity of all points in the point cloud in the grid is calculated, and the center of gravity is used as a new semantic point. According to the new semantic point, a semantic point cloud map in the parking garage environment is obtained, such as Figure 5 shown.

[0051] The beneficial effects of the embodiments of the present application are:

[0052] (1) The embodiment of the present application uses the pixel values ​​after semantic segmentation of the semantic surround camera mosaic image to avoid the operation of calculating feature points using the original image. The step of extracting the center line greatly reduces the number of pixels that need to be processed, thereby improving the processing speed;

[0053] (2) The embodiment of the present application directly stitches the point clouds obtained from the images collected at different times, and then uses the downsampling method, thereby avoiding the tracking and matching steps required for directly using the semantic vector. The downsampling step also reduces the occasional erroneous semantic detection results.

[0054] (3) The embodiment of the present application only uses the lane line and storage location line information in the surround view stitching map, which reduces the requirements for deep learning to detect semantic features.

[0055] According to the semantic mapping method in the parking garage environment proposed in the embodiment of the present application, the vehicle posture information is obtained according to the sensor data, multiple surround images are spliced, the lane line semantic information and the storage line semantic information are detected by the semantic segmentation algorithm, the corresponding image pixels are marked, and the lane line centerline and the storage line centerline image are extracted. The vehicle posture information of the surround image is calculated according to the vehicle posture information and the time of the surround image, and the lane line point cloud and the storage line point cloud at different times are obtained based on the vehicle posture information, the lane line centerline and the storage line centerline image of the surround image. The initial semantic point cloud map is spliced, and the point cloud in the initial semantic point cloud map is downsampled according to the lane line and the storage line, respectively, to create a three-dimensional voxel grid, and the center of gravity of all point clouds in the three-dimensional voxel grid is used as a new semantic point. According to the new semantic point, the semantic point cloud map in the parking garage environment is obtained. Thus, the problem of large calculation amount, high processor computing power requirement and high cost in the mapping method in the related art is solved, the processing speed is improved, and the error semantic detection results and the requirements for deep learning to detect semantic features are reduced.

[0056] Next, a semantic mapping device in a parking garage environment proposed according to an embodiment of the present application is described with reference to the accompanying drawings.

[0057] Figure 6 Schematic block diagram of a semantic mapping device in a parking garage environment according to an embodiment of the present application.

[0058] As shown in the figure, the semantic mapping device 10 in the parking garage environment includes: a first acquisition module 100, an extraction module 200, a splicing module 300 and a second acquisition module 400.

[0059] Among them, the first acquisition module 100 is used to acquire the sensor data of the vehicle and calculate the posture information of the vehicle relative to the starting point based on the sensor data; the extraction module 200 is used to acquire multiple surround view pictures around the vehicle, and after splicing the multiple surround view pictures, detect the lane line semantic information and the storage line semantic information from the spliced ​​pictures through a preset semantic segmentation algorithm, and respectively mark the corresponding picture pixels of the lane line semantic information and the storage line semantic information, and then extract the lane line centerline and the storage line centerline picture; the splicing module 300 is used to calculate the position information of the vehicle relative to the starting point and the multiple surround view pictures. time to calculate the vehicle posture information corresponding to multiple surround view images, and obtain the lane line point cloud and storage location line point cloud at different times based on the vehicle posture information corresponding to the multiple surround view images, the lane line centerline and the storage location line centerline images, and splice the lane line and storage location line point clouds at different times to obtain an initial semantic point cloud map; the second acquisition module 400 is used to downsample the point clouds in the initial semantic point cloud map according to the lane lines and storage location lines, respectively, and create a three-dimensional voxel grid, and use the center of gravity of all point clouds in the three-dimensional voxel grid as a new semantic point, and obtain a semantic point cloud map in the parking garage environment according to the new semantic point.

[0060] Optionally, in some embodiments, the stitching module 300 is specifically used to: select pixel points of lane lines and pixel points of storage lines based on vehicle posture information corresponding to multiple surround view images, lane line centerline images and storage line centerline images; calculate the coordinates of lane lines and storage line coordinates at different times in the vehicle body coordinate system according to the correspondence between the pixel points of lane lines and the pixel points of storage line and the spatial positions; obtain lane line point clouds and storage line point clouds at different times according to the lane line coordinates and the storage line coordinates.

[0061] Optionally, in some embodiments, the extraction module 200 is specifically used to: mark the first image pixels for lane line semantic information, and mark the second image pixels for storage position line semantic information to obtain a lane line and storage position line semantic image; mark the boundary pixel points of multiple first preset positions in the semantic pixels of the lane line and storage position line semantic image and eight pixel points around each pixel point, and delete the marked pixel points to obtain an initial lane line and storage position line semantic image; mark the boundary pixel points of multiple second preset positions in the semantic pixels of the initial lane line and storage position line semantic image and eight pixel points around each pixel point, and delete the marked pixel points to obtain a final lane line and storage position line semantic image; use the final lane line and storage position line semantic image as a new lane line and storage position line semantic image for iterative processing until the preset iteration conditions are met to obtain a lane line centerline and storage position line centerline image.

[0062] Optionally, in some embodiments, the stitching module 300 is further used to: calculate the vehicle posture information corresponding to the multiple surround view pictures based on a preset interpolation method, according to the posture information of the vehicle relative to the starting point and the time of the multiple surround view pictures, wherein the preset interpolation method includes linear interpolation of position and / or spherical interpolation of quaternions representing direction.

[0063] It should be noted that the above explanation of the embodiment of the semantic mapping method in a parking garage environment is also applicable to the semantic mapping device in the parking garage environment of this embodiment, and will not be repeated here.

[0064] According to the semantic mapping device in the parking garage environment proposed in the embodiment of the present application, the vehicle posture information is obtained according to the sensor data, multiple surround images are spliced, the lane line semantic information and the storage position line semantic information are detected by the semantic segmentation algorithm, the corresponding image pixels are marked, and the lane line centerline and the storage position line centerline images are extracted. The vehicle posture information of the surround image is calculated according to the vehicle posture information and the time of the surround image, and the lane line point cloud and the storage position line point cloud at different times are obtained based on the vehicle posture information, the lane line centerline and the storage position line centerline images of the surround image. The initial semantic point cloud map is spliced, and the point cloud in the initial semantic point cloud map is downsampled according to the lane line and the storage position line, respectively, to create a three-dimensional voxel grid, and the center of gravity of all point clouds in the three-dimensional voxel grid is used as a new semantic point. According to the new semantic point, the semantic point cloud map in the parking garage environment is obtained. Thus, the problem of large calculation amount, high processor computing power requirement and high cost in the mapping method in the related art is solved, the processing speed is improved, and the error semantic detection results and the requirements for deep learning to detect semantic features are reduced.

[0065] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0066] A memory 701 , a processor 702 , and a computer program stored in the memory 701 and executable on the processor 702 .

[0067] When the processor 702 executes the program, the semantic mapping method in the parking garage environment provided in the above embodiment is implemented.

[0068] Furthermore, the electronic device further comprises:

[0069] The communication interface 703 is used for communication between the memory 701 and the processor 702 .

[0070] The memory 701 is used to store computer programs that can be executed on the processor 702 .

[0071] The memory 701 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0072] If the memory 701, the processor 702 and the communication interface 703 are implemented independently, the communication interface 703, the memory 701 and the processor 702 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0073] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0074] The processor 702 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0075] The embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned semantic mapping method in a parking garage environment.

[0076] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0077] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0078] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0079] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0080] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0081] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A semantic mapping method in a parking garage environment, characterized in that: The following steps are involved: Acquire sensor data of the vehicle, and calculate the position information of the vehicle relative to the starting point according to the sensor data; Acquire multiple surround view images around the vehicle, and after stitching the multiple surround view images, detect lane line semantic information and storage position line semantic information from the stitched images by using a preset semantic segmentation algorithm, and respectively mark the corresponding image pixels of the lane line semantic information and the storage position line semantic information, and then extract the lane line centerline and storage position line centerline images; Calculate the vehicle posture information corresponding to the multiple surround view images according to the posture information of the vehicle relative to the starting point and the time of the multiple surround view images, and obtain the lane line point cloud and the storage position line point cloud at different times based on the vehicle posture information corresponding to the multiple surround view images, the lane line centerline and the storage position line centerline images, and splice the lane line and storage position line point clouds at different times to obtain an initial semantic point cloud map; as well as The point clouds in the initial semantic point cloud map are downsampled according to lane lines and parking space lines, and a three-dimensional voxel grid is created. The centroids of all point clouds in the three-dimensional voxel grid are used as new semantic points, and a semantic point cloud map in a parking garage environment is obtained based on the new semantic points.

2. The method according to claim 1, characterized in that: The method of obtaining lane point clouds and storage location line point clouds at different times based on the vehicle posture information corresponding to the multiple surround view images, the lane center line images and the storage location line center line images includes: Selecting the pixel points of the lane line and the pixel points of the storage line based on the vehicle posture information corresponding to the multiple surround view images and the lane line centerline and storage line centerline images; Calculate the coordinates of the lane line and the coordinates of the storage line at different times in the vehicle body coordinate system according to the correspondence between the pixel points of the lane line and the pixel points of the storage line and the spatial positions; The lane line point clouds and the storage location line point clouds at different times are obtained according to the lane line coordinates and the storage location line coordinates.

3. The method according to claim 1, characterized in that After marking the image pixels corresponding to the lane line semantic information and the storage position line semantic information respectively, extracting the lane line center line and the storage position line center line images, including: Marking the first image pixels with the lane line semantic information, and marking the second image pixels with the storage location line semantic information to obtain a lane line and storage location line semantic image; Marking a plurality of boundary pixels at first preset positions and eight pixels around each pixel in the semantic pixels of the lane line and storage position line semantic image, and deleting the marked pixels to obtain an initial lane line and storage position line semantic image; Marking a plurality of boundary pixels at second preset positions and eight pixels around each pixel in the semantic pixels of the initial lane line and storage position line semantic image, and deleting the marked pixels to obtain a final lane line and storage position line semantic image; The final lane line and storage location line semantic images are used as new lane line and storage location line semantic images for iterative processing until a preset iteration condition is met, thereby obtaining the lane line centerline and storage location line centerline images.

4. The method according to claim 1, characterized in that The calculating the vehicle posture information corresponding to the multiple surround view pictures according to the posture information of the vehicle relative to the starting point and the time of the multiple surround view pictures includes: Based on a preset interpolation method, the vehicle posture information corresponding to the multiple surround-view pictures is calculated according to the posture information of the vehicle relative to the starting point and the time of the multiple surround-view pictures, wherein the preset interpolation method includes linear interpolation of position and / or spherical interpolation of quaternions representing direction.

5. A semantic mapping device in a parking garage environment, characterized in that: include: A first acquisition module is used to acquire sensor data of the vehicle and calculate the position information of the vehicle relative to the starting point according to the sensor data; An extraction module, used for acquiring a plurality of surround view images around the vehicle, and after splicing the plurality of surround view images, detecting the lane line semantic information and the storage position line semantic information from the spliced ​​images by a preset semantic segmentation algorithm, and respectively marking the corresponding image pixels of the lane line semantic information and the storage position line semantic information, and then extracting the lane line centerline and the storage position line centerline images; A splicing module, for calculating the vehicle posture information corresponding to the multiple surround view pictures according to the posture information of the vehicle relative to the starting point and the time of the multiple surround view pictures, and obtaining the lane line point cloud and the storage position line point cloud at different times based on the vehicle posture information corresponding to the multiple surround view pictures, the lane line centerline and the storage position line centerline pictures, and splicing the lane line and storage position line point clouds at different times to obtain an initial semantic point cloud map; as well as The second acquisition module is used to downsample the point clouds in the initial semantic point cloud map according to lane lines and parking space lines, and create a three-dimensional voxel grid, and use the centroids of all point clouds in the three-dimensional voxel grid as new semantic points, and obtain a semantic point cloud map in the parking garage environment according to the new semantic points.

6. The device according to claim 5, characterized in that The splicing module is specifically used for: Selecting the pixel points of the lane line and the pixel points of the storage line based on the vehicle posture information corresponding to the multiple surround view images and the lane line centerline and storage line centerline images; Calculate the coordinates of the lane line and the coordinates of the storage line at different times in the vehicle body coordinate system according to the correspondence between the pixel points of the lane line and the pixel points of the storage line and the spatial positions; The lane line point clouds and the storage location line point clouds at different times are obtained according to the lane line coordinates and the storage location line coordinates.

7. The device according to claim 5, characterized in that The extraction module is specifically used for: Marking the first image pixels with the lane line semantic information, and marking the second image pixels with the storage location line semantic information to obtain a lane line and storage location line semantic image; Marking a plurality of boundary pixels at first preset positions and eight pixels around each pixel in the semantic pixels of the lane line and storage position line semantic image, and deleting the marked pixels to obtain an initial lane line and storage position line semantic image; Marking a plurality of boundary pixels at second preset positions and eight pixels around each pixel in the semantic pixels of the initial lane line and storage position line semantic image, and deleting the marked pixels to obtain a final lane line and storage position line semantic image; The final lane line and storage location line semantic images are used as new lane line and storage location line semantic images for iterative processing until a preset iteration condition is met, thereby obtaining the lane line centerline and storage location line centerline images.

8. The device according to claim 5, characterized in that The splicing module is also used for: Based on a preset interpolation method, the vehicle posture information corresponding to the multiple surround-view pictures is calculated according to the posture information of the vehicle relative to the starting point and the time of the multiple surround-view pictures, wherein the preset interpolation method includes linear interpolation of position and / or spherical interpolation of quaternions representing direction.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the semantic mapping method in a parking garage environment as described in any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the semantic mapping method in a parking garage environment as described in any one of claims 1 to 4.

Images