Parking lot mapping methods, equipment, vehicles and storage media

Abstract

This application relates to a method, apparatus, vehicle, and storage medium for mapping parking lots. The method includes: acquiring parking lot images around a vehicle and performing distortion correction; performing inverse projection transformation on the distorted parking lot images around the vehicle to generate a ground bird's-eye view, and segmenting the image according to preset segmentation requirements to obtain a ground marker image segmentation map; transforming the ground marker image segmentation map based on a preset inverse perspective projection transformation method to generate a ground marker point cloud, performing frame matching to obtain a surround-view camera odometer, and fusing it with a preset IMU odometer to obtain the vehicle's pose data; and then mapping and locating the parking lot to obtain a final global map of the parking lot. This solves the problems of inaccurate camera perspective transformation, unstable marker segmentation, low mapping and positioning accuracy, and high cost. By introducing a new algorithm, the accuracy is improved while reducing the cost of mapping and locating vehicles in parking lots.

Description

Technical Field

[0001] This application relates to the field of vehicle fusion positioning technology, and in particular to a parking lot mapping method, device, vehicle and storage medium. Background Technology

[0002] Intelligent driving technology is a technology that enables vehicles to automatically perform fusion localization, environmental perception, path planning, and automatic control to achieve autonomous driving without human intervention. This technology relies on sensors such as cameras, radar, sound waves, and inertial navigation systems, as well as numerous algorithms for localization, recognition, perception, planning, and control, to achieve functions such as physical localization, environmental perception, obstacle recognition, path planning, and driving control, ultimately completing autonomous driving.

[0003] Parking lot mapping and positioning technology is a sub-technology of intelligent driving and one of the fundamental technologies for realizing automatic parking in parking lots. The vehicle acquires data such as images of the surrounding environment, point clouds, its own angular velocity, and acceleration through body sensors, and processes the sensor data through algorithms to estimate its own pose and construct a point cloud map of the surrounding environment, providing a basis for subsequent relocalization and automatic parking path planning.

[0004] In related technologies, most methods employ fusion methods based on sensors such as LiDAR, deep learning methods based on multiple camera images, or setting up sign codes in parking lots to achieve parking lot fusion mapping and positioning.

[0005] However, the above methods often suffer from limitations such as inaccurate camera perspective transformation, unstable segmentation of parking lot landmarks, low mapping and positioning accuracy, and high costs in the process of achieving integrated mapping and positioning of parking lots, which urgently need to be addressed. Summary of the Invention

[0006] This application provides a parking lot mapping method, apparatus, vehicle, and storage medium to solve problems in related technologies such as inaccurate inverse perspective transformation of surround-view cameras, unstable segmentation of parking lot signs based on bird's-eye view, and low accuracy of mapping and positioning based solely on sign point clouds. By introducing an adaptive inverse perspective transformation algorithm, the accuracy of point cloud generation for ground parking signs is optimized; by introducing a deep learning image segmentation algorithm, the segmentation effect of ground parking signs is optimized; and by fusing IMU (Inertial Measurement Unit) and GNSS (Global Navigation Satellite System), the accuracy of mapping and positioning is improved.

[0007] The first aspect of this application provides a method for mapping a parking lot, comprising the following steps:

[0008] Acquire images of the parking lot surrounding the vehicle, and perform distortion correction on the images of the parking lot surrounding the vehicle to obtain a distortion-corrected image of the parking lot surrounding the vehicle.

[0009] The distortion-corrected parking lot image around the vehicle is subjected to inverse projection transformation to generate a ground bird's-eye view. This ground bird's-eye view is then segmented according to preset segmentation requirements to obtain a ground marker image segmentation map.

[0010] Based on a preset inverse perspective projection transformation method, the ground marker image segmentation map is transformed to generate a ground marker point cloud. Frame matching is then performed on the ground marker point cloud to obtain a surround-view camera odometer. The surround-view camera odometer and a preset IMU odometer are fused to obtain the vehicle's pose data. The parking lot is then mapped and located based on the pose data to obtain the final global map of the parking lot.

[0011] According to one embodiment of this application, after obtaining the distortion-corrected parking lot image around the vehicle, the method further includes:

[0012] The preset IMU odometer is obtained by integrating the IMU data using the inertial navigation integral model.

[0013] According to one embodiment of this application, the distortion correction of the parking lot image surrounding the vehicle includes:

[0014] Based on a preset fourth-order polynomial parameter model, distortion correction is performed on the parking lot image surrounding the vehicle, wherein the preset fourth-order polynomial parameter model is:

[0015] ρ(θ)=k1*θ+k2*θ 2 +k3*θ 3 +k4*θ 4 ;

[0016] Where θ is the incident angle relative to the optical axis, ρ is the distance between the image center and the projection point, and k1, k2, k3 and k4 are constants given in the calibration file.

[0017] According to one embodiment of this application, segmenting the ground bird's-eye view according to preset segmentation requirements to obtain a ground marker image segmentation map includes:

[0018] The ground bird's-eye view is segmented into parking space marking lines, driving guide lines, and stop lines to obtain the ground marking image segmentation map.

[0019] The parking lot mapping method according to the embodiments of this application acquires parking lot images around the vehicle and performs distortion correction. The distorted parking lot images around the vehicle are then subjected to inverse projection transformation to generate a ground bird's-eye view. This view is then segmented according to preset segmentation requirements to obtain a ground marker image segmentation map. Based on a preset inverse perspective projection transformation method, the ground marker image segmentation map is transformed to generate a ground marker point cloud. Frame matching is then performed to obtain the surround-view camera odometer, which is fused with a preset IMU odometer to obtain the vehicle's pose data. This data is then used for parking lot mapping and localization to obtain the final global map of the parking lot. This solves the problems of inaccurate camera perspective transformation, unstable marker segmentation, low mapping and localization accuracy, and high cost. By introducing a new algorithm, the accuracy is improved while reducing the cost of vehicle mapping and localization in the parking lot.

[0020] A second aspect of this application provides a parking lot mapping device, comprising:

[0021] The correction module is used to acquire images of the parking lot around the vehicle and to correct the distortion of the images of the parking lot around the vehicle to obtain a distortion-corrected image of the parking lot around the vehicle.

[0022] A segmentation module is used to perform inverse projection transformation on the distortion-corrected parking lot image around the vehicle to generate a ground bird's-eye view, and to segment the ground bird's-eye view according to preset segmentation requirements to obtain a ground sign image segmentation map; and

[0023] The mapping module is used to transform the ground marker image segmentation map based on a preset inverse perspective projection transformation method to generate a ground marker point cloud, and to perform frame matching on the ground marker point cloud to obtain the surround-view camera odometer. The surround-view camera odometer and the preset IMU odometer are fused to obtain the vehicle's pose data, and the parking lot is mapped and located based on the pose data to obtain the final global map of the parking lot.

[0024] According to one embodiment of this application, after obtaining the distortion-corrected parking lot image around the vehicle, the correction module is further configured to:

[0025] The preset IMU odometer is obtained by integrating the IMU data using the inertial navigation integral model.

[0026] According to one embodiment of this application, the correction module is specifically used for:

[0027] The correction unit is used to correct the distortion of the parking lot image around the vehicle based on a preset fourth-order polynomial parameter model, wherein the preset fourth-order polynomial parameter model is:

[0028] ρ(θ)=k1*θ+k2*θ2 +k3*θ 3 +k4*θ 4 ;

[0029] Where θ is the incident angle relative to the optical axis, ρ is the distance between the image center and the projection point, and k1, k2, k3 and k4 are constants given in the calibration file.

[0030] According to one embodiment of this application, the segmentation module is specifically used for:

[0031] The ground bird's-eye view is segmented into parking space marking lines, driving guide lines, and stop lines to obtain the ground marking image segmentation map.

[0032] The parking lot mapping device according to an embodiment of this application acquires parking lot images around a vehicle, performs distortion correction, and then performs inverse projection transformation on the distorted parking lot images around the vehicle to generate a ground bird's-eye view. This image is then segmented according to preset segmentation requirements to obtain a ground marker image segmentation map. Based on a preset inverse perspective projection transformation method, the ground marker image segmentation map is transformed to generate a ground marker point cloud. Frame matching is then performed to obtain a surround-view camera odometer, which is fused with a preset IMU odometer to obtain the vehicle's pose data. This data is then used for parking lot mapping and localization to obtain the final global map of the parking lot. This solves the problems of inaccurate camera perspective transformation, unstable marker segmentation, low mapping and localization accuracy, and high cost limitations. By introducing a new algorithm, the accuracy is improved while reducing the cost of vehicle mapping and localization in the parking lot.

[0033] A third aspect of this application provides a vehicle, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the parking lot mapping method as described in the above embodiments.

[0034] A fourth aspect of this application provides a computer-readable storage medium storing computer instructions for causing the computer to perform the parking lot mapping method as described in the above embodiments.

[0035] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0036] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0037] Figure 1This is a flowchart of a parking lot mapping method according to an embodiment of this application;

[0038] Figure 2 This is a schematic diagram of the overall process according to one embodiment of this application;

[0039] Figure 3 This is a schematic diagram illustrating the configuration of a vehicle surround view camera according to an embodiment of this application;

[0040] Figure 4 This is a schematic diagram of a raw fisheye image acquired by a vehicle surround view camera according to an embodiment of this application;

[0041] Figure 5 This is a schematic diagram of a fisheye image after distortion correction according to an embodiment of this application;

[0042] Figure 6 This is a bird's-eye view diagram generated by stitching together the inverse perspective projection transformation according to an embodiment of this application;

[0043] Figure 7 This is a schematic diagram of the segmentation result of a bird's-eye view image by a deep learning network according to an embodiment of this application;

[0044] Figure 8 This is a schematic diagram of a ground marker point cloud generated based on a segmented bird's-eye view according to an embodiment of this application;

[0045] Figure 9 This is a partial point cloud diagram of a parking lot according to an embodiment of this application;

[0046] Figure 10 This is a schematic diagram of the overall point cloud of a parking lot according to an embodiment of this application;

[0047] Figure 11 This is an example diagram of a parking lot mapping device according to an embodiment of this application;

[0048] Figure 12 This is a structural schematic diagram of a vehicle provided according to an embodiment of this application. Detailed Implementation

[0049] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0050] The following description, with reference to the accompanying drawings, describes a parking lot mapping method, apparatus, vehicle, and storage medium according to embodiments of this application. Addressing the problems mentioned in the background art regarding parking lot mapping methods, such as inaccurate inverse perspective transformation of surround-view cameras, unstable segmentation of parking lot landmarks based on bird's-eye views, and low accuracy of mapping and positioning solely based on landmark point clouds, this application provides a parking lot mapping method. In this method, images of the parking lot surrounding a vehicle are acquired and distortion corrected. The distorted images are then subjected to inverse projection transformation to generate a ground bird's-eye view, which is segmented according to preset segmentation requirements to obtain a ground landmark image segmentation map. Based on a preset inverse perspective projection transformation method, the ground landmark image segmentation map is transformed to generate a ground landmark point cloud, which is then frame-matched to obtain a surround-view camera odometer. This odometer is fused with a preset IMU odometer to obtain the vehicle's pose data, thereby enabling parking lot mapping and positioning to obtain the final global map of the parking lot. This solves the problems of inaccurate camera perspective transformation, unstable marker segmentation, low mapping and positioning accuracy, and high cost. By introducing a new algorithm, the accuracy is improved while the cost of mapping and positioning vehicles in parking lots is reduced.

[0051] Specifically, Figure 1 This is a schematic flowchart illustrating a parking lot mapping method provided in an embodiment of this application.

[0052] like Figure 1 As shown, the mapping method for this parking lot includes the following steps:

[0053] In step S101, an image of the parking lot around the vehicle is acquired, and the image of the parking lot around the vehicle is distorted to obtain a distorted image of the parking lot around the vehicle.

[0054] Furthermore, in some embodiments, distortion correction of the parking lot image around the vehicle includes: distorting the parking lot image around the vehicle based on a preset fourth-order polynomial parameter model, wherein the preset fourth-order polynomial parameter model is:

[0055] ρ(θ)=k1*θ+k2*θ 2 +k3*θ 3 +k4*θ 4 ;

[0056] Where θ is the incident angle relative to the optical axis, ρ is the distance between the image center and the projection point, and k1, k2, k3 and k4 are constants given in the calibration file.

[0057] Specifically, such as Figure 2As shown in the illustration, this embodiment of the application acquires parking lot images around the vehicle using a vehicle-mounted surround-view camera. The images should ideally include fixed ground markers such as parking space markings, driving guide lines, lane lines, and stop lines. Since the vehicle-mounted surround-view camera is typically a fisheye camera, the acquired images are mostly fisheye images of the area around the vehicle containing ground traffic features. The acquired images are as follows: Figure 3 As shown.

[0058] Furthermore, in this embodiment of the application, after acquiring parking information around the vehicle using a fisheye camera, distortion correction needs to be performed on the acquired fisheye image. Specifically, the extrinsic and extrinsic parameters of the surround-view camera need to be calibrated in advance. This includes calibrating the polynomial model parameters of the fisheye camera used for distortion correction to obtain the initial extrinsic parameters of the surround-view camera.

[0059] Specifically, the calibration parameters in this application embodiment can be used to correct the distortion of fisheye images based on a radial fourth-order polynomial parameter model. For example, the image comparison before and after correction is shown below. Figure 4 and Figure 5 As shown.

[0060] Furthermore, the preset fourth-order polynomial parameter model used in the embodiments of this application is as follows:

[0061] ρ(θ)=k1*θ+k2*θ 2 +k3*θ 3 +k4*θ 4 ;

[0062] Where θ is the incident angle relative to the optical axis, ρ is the distance between the image center and the projection point, and the coefficients k1, k2, k3, and k4 are given in the calibration file. It should be noted that the image width and height of the principal point, as well as the offset (cx, cy), are in pixels.

[0063] For example, taking the 3D point given by camera coordinates and the image coordinates as an example, the projection of the 3D point (X, Y, Z) given by camera coordinates to the image coordinates (u, v) is as follows:

[0064]

[0065] θ=arctan2(χ,Z)=pi / 2-arctan2(Z,χ)

[0066] ρ=ρ(θ)

[0067] u′=ρ*X / χ if χ≠0 else 0

[0068] v′=ρ*Y / χ if χ≠0 else 0

[0069] u = u′ + cx + w / 2 - 0.5

[0070] v=v′*aspect.ratio+cy+h / 2-0.5

[0071] Where cx is the offset of the principal point in the u direction, cy is the offset of the principal point in the v direction, w is the image width, h is the image height, aspect.ratio is the aspect ratio of the image, χ, u', and v' are intermediate values ​​in the calculation, and the last two lines show the final transformation of the image coordinate system. Assuming that the origin of the image coordinate system is located at the top left corner, the top left corner pixel is (0, 0).

[0072] Furthermore, in some embodiments, after obtaining the distortion-corrected parking lot image around the vehicle, the method further includes: integrating the IMU data according to the inertial navigation integral model to obtain a preset IMU odometer.

[0073] Specifically, in this embodiment of the application, after obtaining the distorted parking lot image around the vehicle, it is necessary to integrate the IMU data according to the inertial navigation integral model to obtain the vehicle's predicted pose, i.e., the IMU odometry. The predicted pose includes the vehicle's position, velocity, attitude, etc. The main integration process is shown in the following formula:

[0074]

[0075] v←v+(R(a m -a b )+g)Δt

[0076]

[0077] a b ←a b

[0078] w b ←ω b

[0079] g←g,

[0080] Where p is the IMU position vector, v is the IMU velocity vector, q is the IMU attitude quaternion, and a m For IMU acceleration, ω m For IMU angular velocity, a b For IMU acceleration bias, ω b R is the IMU angular velocity offset, R is the IMU attitude matrix, Δt is the time interval between two IMU frames, and g is the gravitational acceleration vector.

[0081] In step S102, the image of the parking lot around the vehicle after distortion correction is subjected to inverse projection transformation to generate a ground bird's-eye view. The ground bird's-eye view is then segmented according to preset segmentation requirements to obtain a ground sign image segmentation map.

[0082] Furthermore, in some embodiments, the ground bird's-eye view is segmented according to preset segmentation requirements to obtain a ground sign image segmentation map, including: segmenting the ground bird's-eye view into parking space marking lines, driving guide lines and stop lines to obtain a ground sign image segmentation map.

[0083] Specifically, such as Figure 6 As shown, this embodiment first performs inverse projection transformation on the distortion-corrected vehicle's surrounding environment using an adaptive IPM algorithm to generate a single ground bird's-eye view. This view is then stitched together using extrinsic parameters from each camera to form a complete bird's-eye view. The adaptive IPM (Inverse Perspective Mapping) uses the vehicle pose prediction from the IMU odometry to obtain the camera pose, thus determining the camera's dynamic pitch and yaw angles in the current environment. Pitch angle corrections are added between adjacent frames. The specific process is as follows:

[0084]

[0085]

[0086] Where, θ o θ is the initial pitch angle of the camera, which is the angle between the optical axis and the horizontal plane. p α is the dynamic incremental pitch angle of the camera. r α is half the vertical field of view. c Half of the horizontal field of view, where m is the image height and n is the image width.

[0087] Furthermore, in this embodiment, after obtaining the overall ground bird's-eye view, the ground markings such as parking space lines, driving guide lines, and stop lines are segmented based on a UNET (Network). During the segmentation process, the network needs to be trained beforehand using a bird's-eye view containing various types and situations of ground markings in the parking lot to obtain weight parameters to initialize the network for online segmentation. The segmentation result is as follows: Figure 7 As shown.

[0088] Specifically, the UNET network is a typical segmentation network that is fast, accurate, and robust for monotonous ground environments. Taking into account the features of parking lot bird's-eye views, such as lane lines, parking lines, guide signs, speed bumps, obstacles, and free space, the network is pre-trained using a parking lot dataset to obtain weights for recognition. Then, based on these trained weights, the network is configured to segment the overall bird's-eye view.

[0089] In step S103, based on a preset inverse perspective projection transformation method, the ground marker image segmentation map is transformed to generate a ground marker point cloud, and frame matching is performed on the ground marker point cloud to obtain the surround-view camera odometer. The surround-view camera odometer and the preset IMU odometer are fused to obtain the vehicle's pose data, and the parking lot is mapped and located based on the pose data to obtain the final global map of the parking lot.

[0090] Specifically, such as Figure 8 As shown, this embodiment of the application extracts and segments the corresponding lane lines, parking lines, guide signs, speed bumps and obstacles in the bird's-eye view, and generates a ground marker point cloud based on the pose relationship between the virtual camera in the bird's-eye view and the vehicle coordinate system by inverse transformation of inverse perspective projection.

[0091] Furthermore, the ground marker point cloud is first analyzed using the NDT (Normal Distributions Transform) frame matching algorithm to obtain the odometry from the surrounding camera. The frame matching process employs the NDT algorithm. Secondly, during frame matching, the local map is extracted using a distance-based sliding window principle to ensure a balance between accuracy and efficiency. The predicted pose during matching utilizes the integrated results of the IMU, and is aligned and interpolated based on time.

[0092] It should be noted that since the odometry obtained solely based on ground feature point cloud frame matching has a large error, this embodiment uses the ESKF (Error State Kalman Filter) algorithm to fuse the IMU odometry and the odometry from the ground marker point cloud of the surround-view camera, thereby obtaining high-precision pose data of the vehicle body for mapping and localization. For example, Figure 9 As shown, the mapping in this embodiment can be constructed using the SLAM (Simultaneous Localization and Mapping) method, which uses a local map to construct the mapping based on the fused global pose.

[0093] Furthermore, ESKF is divided into two parts: prediction and observation. The error state system equation in the prediction is:

[0094] δx←f(x, δx, u) m ,i)=F x (x, u) m )·δx+F i ·i;

[0095] Where δx represents the error state, x represents the nominal state, and u m For IMU measurement results, i represents the disturbance, and F represents the value of the disturbance.x Let f() be the Jacobian matrix of the error state, F i Let f() be the Jacobian matrix of the perturbation.

[0096] The error state equation in the prediction is:

[0097]

[0098] in, To predict the error state, F x Let f() be the Jacobian matrix of the error state.

[0099] The error state covariance matrix in the prediction is:

[0100]

[0101] Where P is the error state covariance matrix, Q i Let be the covariance matrix of the perturbation.

[0102] The observation equation in the observation is:

[0103] y = h(x) t )+v;

[0104] Where y represents the observation state, x t is the true state, () is a general nonlinear function, and v is Gaussian noise.

[0105] Kalman gain update in observation

[0106]

[0107] Where H is the Jacobian matrix of () with respect to the error state, and V is the observation Gaussian noise covariance matrix.

[0108] Error status update during observation:

[0109]

[0110] Where K is the Kalman gain, Predict the true state.

[0111] Update of the error state covariance matrix in observations:

[0112] P←(I-KH)P;

[0113] Where I is the identity matrix.

[0114] Specifically, due to the cumulative error of point cloud odometry, pose drift is inevitable when the map is large and there are many turns. Therefore, this embodiment of the application needs to detect potential loop closures based on distance and time constraints. The point cloud ICP (Iterative Closest Point) algorithm is used to match the ground marker point cloud and the loop closure local map of the current frame to further confirm whether a loop closure has occurred. If the loop closure is successful, the ICP matching result of the current frame is the loop closure constraint.

[0115] Furthermore, such as Figure 10 As shown, the optimization of the global pose is based on the GTSAM graph optimization library. When a closure constraint occurs, the current pose constraint is added to the optimization graph. After the update operation, a new global pose can be obtained, which in turn updates the global map, thus obtaining the final global map.

[0116] In summary, the embodiments of this application have the following technical effects:

[0117] (1) Using low-cost vehicle surround view cameras and inertial navigation equipment, relatively stable and high-precision positioning and mapping can be provided in parking lot environments with variable visual characteristics, weak GNSS signals, and no need to set up visual markers or make other modifications in advance.

[0118] (2) Using low-cost surround-view cameras, inertial navigation equipment and GNSS positioning equipment, no additional sensors are required and no other modifications such as pre-setting visual markers in the parking lot are needed. It is low-cost and highly practical.

[0119] (3) Introduce algorithms with appropriate accuracy and efficiency in each step to improve the accuracy of the processing results in each step, and ultimately ensure the accuracy and real-time performance in the mapping and positioning process. At the same time, the map can be continuously iterated and optimized during use.

[0120] The parking lot mapping method according to the embodiments of this application acquires parking lot images around the vehicle and performs distortion correction. The distorted parking lot images around the vehicle are then subjected to inverse projection transformation to generate a ground bird's-eye view. This view is then segmented according to preset segmentation requirements to obtain a ground marker image segmentation map. Based on a preset inverse perspective projection transformation method, the ground marker image segmentation map is transformed to generate a ground marker point cloud. Frame matching is then performed to obtain the surround-view camera odometer, which is fused with a preset IMU odometer to obtain the vehicle's pose data. This data is then used for parking lot mapping and localization to obtain the final global map of the parking lot. This solves the problems of inaccurate camera perspective transformation, unstable marker segmentation, low mapping and localization accuracy, and high cost. By introducing a new algorithm, the accuracy is improved while reducing the cost of vehicle mapping and localization in the parking lot.

[0121] Next, referring to the accompanying drawings, a parking lot mapping device according to an embodiment of this application is described.

[0122] Figure 11 This is a block diagram of a parking lot mapping device according to an embodiment of this application.

[0123] like Figure 11 As shown, the mapping device 10 for the parking lot includes: a correction module 100, a segmentation module 200, and a mapping module 300.

[0124] The correction module 100 is used to acquire images of the parking lot around the vehicle and to correct the distortion of the parking lot images around the vehicle to obtain a distorted parking lot image around the vehicle.

[0125] The segmentation module 200 is used to perform inverse projection transformation on the distortion-corrected parking lot image around the vehicle to generate a ground bird's-eye view, and to segment the ground bird's-eye view according to preset segmentation requirements to obtain a ground sign image segmentation map; and

[0126] The mapping module 300 is used to transform the ground marker image segmentation map based on a preset inverse perspective projection transformation method to generate a ground marker point cloud, and to perform frame matching on the ground marker point cloud to obtain the surround-view camera odometer. The surround-view camera odometer and the preset IMU odometer are fused to obtain the vehicle pose data, and the parking lot is mapped and located based on the pose data to obtain the final global map of the parking lot.

[0127] Furthermore, in some embodiments, after obtaining the distortion-corrected parking lot image around the vehicle, the correction module 100 is also used to:

[0128] The IMU data is integrated using the inertial navigation integral model to obtain the preset IMU odometry.

[0129] Furthermore, in some embodiments, the correction module 100 is specifically used for:

[0130] The correction unit is used to correct distortion in parking lot images surrounding vehicles based on a preset fourth-order polynomial parameter model, wherein the preset fourth-order polynomial parameter model is:

[0131] ρ(θ)=k1*θ+k2*θ 2 +k3*θ 3 +k4*θ 4 ;

[0132] Where θ is the incident angle relative to the optical axis, ρ is the distance between the image center and the projection point, and k1, k2, k3 and k4 are constants given in the calibration file.

[0133] Furthermore, in some embodiments, the segmentation module 200 is specifically used for:

[0134] The ground bird's-eye view is segmented into parking space marking lines, driving guide lines, and stop lines to obtain a ground marking image segmentation map.

[0135] The parking lot mapping device according to an embodiment of this application acquires parking lot images around a vehicle, performs distortion correction, and then performs inverse projection transformation on the distorted parking lot images around the vehicle to generate a ground bird's-eye view. This image is then segmented according to preset segmentation requirements to obtain a ground marker image segmentation map. Based on a preset inverse perspective projection transformation method, the ground marker image segmentation map is transformed to generate a ground marker point cloud. Frame matching is then performed to obtain a surround-view camera odometer, which is fused with a preset IMU odometer to obtain the vehicle's pose data. This data is then used for parking lot mapping and localization to obtain the final global map of the parking lot. This solves the problems of inaccurate camera perspective transformation, unstable marker segmentation, low mapping and localization accuracy, and high cost limitations. By introducing a new algorithm, the accuracy is improved while reducing the cost of vehicle mapping and localization in the parking lot.

[0136] Figure 12 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include:

[0137] The memory 1201, the processor 1202, and the computer program stored on the memory 1201 and executable on the processor 1202.

[0138] When the processor 1202 executes the program, it implements the parking lot mapping method provided in the above embodiments.

[0139] Furthermore, the vehicle also includes:

[0140] Communication interface 1203 is used for communication between memory 1201 and processor 1202.

[0141] The memory 1201 is used to store computer programs that can run on the processor 1202.

[0142] The memory 1201 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0143] If the memory 1201, processor 1202, and communication interface 1203 are implemented independently, then the communication interface 1203, memory 1201, and processor 1202 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 12 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0144] Optionally, in a specific implementation, if the memory 1201, processor 1202, and communication interface 1203 are integrated on a single chip, then the memory 1201, processor 1202, and communication interface 1203 can communicate with each other through an internal interface.

[0145] The processor 1202 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0146] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described parking lot mapping method.

[0147] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0148] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0149] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0150] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0151] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0152] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.

[0153] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0154] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method of mapping a parking lot, characterized by, The method comprises the following steps: obtaining a parking lot image around a vehicle and distortion correcting the parking lot image around the vehicle to obtain a distortion-corrected parking lot image around the vehicle; performing inverse projection transformation on the distortion-corrected parking lot image around the vehicle to generate a ground bird's-eye view, and segmenting the ground bird's-eye view according to a preset segmentation requirement to obtain a ground sign image segmentation map; and based on a preset inverse perspective projection transformation method, transforming the ground sign image segmentation map to generate a ground sign point cloud, performing frame map matching on the ground sign point cloud to obtain a surround-view camera odometry, fusing the surround-view camera odometry and a preset IMU odometry to obtain pose data of the vehicle, and building a map and positioning the parking lot according to the pose data to obtain a final global map of the parking lot; wherein the distortion correction of the parking lot image around the vehicle comprises distortion correcting the parking lot image around the vehicle based on a preset fourth-order polynomial parameter model, wherein the preset fourth-order polynomial parameter model is: ； wherein θ is an angle of incidence relative to an optical axis, ρ is a distance between an image center and a projection point, and k1, k2, k3 and k4 are all constants given in a calibration file; inverse projection transformation of the distortion-corrected environment around the vehicle is performed by an adaptive IPM algorithm to generate a single ground bird's-eye view, and the overall bird's-eye view is spliced by the extrinsic parameters between the cameras, wherein the adaptive IPM algorithm uses the prediction of the vehicle pose in the IMU odometry to obtain the dynamic pitch angle and yaw angle of the camera in the current environment, and adds a pitch angle correction in adjacent frames; the surround-view camera odometry of the ground sign point cloud is obtained by an NDT frame map matching algorithm, wherein the frame map matching adopts the NDT algorithm, and the local map is extracted based on the distance sliding window principle when performing frame map matching, the prediction pose in the matching adopts the integral result of the IMU, and the alignment and interpolation are performed based on time.

2. The method of claim 1, wherein, After obtaining the distortion-corrected parking lot image around the vehicle, the following steps are further included: integrating the IMU data according to an inertial navigation integration model to obtain the preset IMU odometry.

3. The method of claim 1, wherein, The segmentation of the ground bird's-eye view according to the preset segmentation requirement to obtain the ground sign image segmentation map comprises: segmenting the ground bird's-eye view into parking space sign lines, driving guide lines and stop lines to obtain the ground sign image segmentation map.

4. A parking lot mapping device, characterized by, It comprises: a correction module for obtaining a parking lot image around a vehicle and distortion correcting the parking lot image around the vehicle to obtain a distortion-corrected parking lot image around the vehicle; a segmentation module for performing inverse projection transformation on the distortion-corrected parking lot image around the vehicle to generate a ground bird's-eye view, and segmenting the ground bird's-eye view according to a preset segmentation requirement to obtain a ground sign image segmentation map; and The mapping module is configured to transform the ground mark image segmentation map based on a preset inverse perspective projection transformation method, generate a ground mark point cloud, and perform frame map matching on the ground mark point cloud to obtain a surround-view camera odometry, fuse the surround-view camera odometry and a preset IMU odometry to obtain pose data of the vehicle, and map and locate the parking lot based on the pose data to obtain a final global map of the parking lot. The correction module is specifically configured to: a correction unit configured to correct the parking lot image around the vehicle based on a preset fourth-order polynomial parameter model, wherein the preset fourth-order polynomial parameter model is: ； where θ is the angle of incidence relative to the optical axis, ρ is the distance between the image center and the projection point, and k1, k2, k3, and k4 are constants given in the calibration file. The surround-view camera odometry of the ground bird's-eye view is obtained by performing inverse projection transformation on the corrected image around the vehicle using an adaptive IPM algorithm, and the overall bird's-eye view is obtained by splicing the poses of the cameras based on the extrinsic parameters of the cameras, wherein the adaptive IPM algorithm uses the prediction of the vehicle pose in the IMU odometry to obtain the dynamic pitch angle and yaw angle of the camera in the current environment, and adds a correction of the pitch angle in adjacent frames. The surround-view camera odometry of the ground bird's-eye view is obtained by performing inverse projection transformation on the corrected image around the vehicle using an adaptive IPM algorithm, and the overall bird's-eye view is obtained by splicing the poses of the cameras based on the extrinsic parameters of the cameras, wherein the adaptive IPM algorithm uses the prediction of the vehicle pose in the IMU odometry to obtain the dynamic pitch angle and yaw angle of the camera in the current environment, and adds a correction of the pitch angle in adjacent frames.

5. The apparatus of claim 4, wherein, After obtaining the corrected parking lot image around the vehicle, the correction module is further configured to: Integrate the IMU data based on a strapdown inertial navigation integration model to obtain the preset IMU odometry.

6. The apparatus of claim 4, wherein, The segmentation module is specifically configured to: Segment the ground bird's-eye view into parking space mark lines, driving guide lines, and stop lines to obtain the ground mark image segmentation map.

7. A vehicle characterized by comprising: It includes: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the mapping method of the parking lot according to any one of claims 1-3.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the mapping method of the parking lot according to any one of claims 1-3.

Images