A parking lot map construction method, apparatus, and computer storage medium

Abstract

This application relates to the field of computer technology, and in particular to a parking lot map construction method, apparatus, and computer storage medium. The method includes: performing data calibration on platform motion data collected by a mobile platform at a corresponding time to obtain platform data corresponding to a target time; performing data constraint processing based on the platform data corresponding to the previous time of the target time and the platform data corresponding to the target time to obtain a first constraint relationship between the target image frame and the previous image frame; updating the sub-map corresponding to the previous image frame to obtain a sub-map corresponding to the target image frame; determining the next time of the target time as the target time; repeating the above steps until the next time of the target time is determined as the target time, until there is no target image frame corresponding to the target time; updating the parking lot map using motion data collected by the mobile platform avoids over-reliance on positioning signals, thereby improving the construction accuracy of the parking lot map.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a parking lot map construction method, apparatus and computer storage medium. Background Technology

[0002] With the development of autonomous driving technology, automatic parking has gradually become an indispensable function. In the process of automatic parking in a parking lot, map data of the parking lot is required. In the current technology, the construction of parking lot maps is often carried out by manual surveying, and the accuracy of manually surveyed parking lot maps is low. Moreover, the construction of the map depends on GPS positioning data collection. In places with weak GPS positioning signals, such as underground parking lots, the accuracy of the surveyed parking lot maps will be low, which is not conducive to automatic parking. Summary of the Invention

[0003] To address the aforementioned problems in the prior art, the purpose of this application is to use a mobile platform to collect data and build maps of parking lots, and to use the motion data of the mobile platform to update the sub-maps of parking lots, thereby avoiding over-reliance on positioning signals and improving the accuracy of parking lot map construction.

[0004] To address the aforementioned problems, this application provides a parking lot map construction method, the method comprising:

[0005] The system acquires a target image frame captured at a target time and platform motion data captured at a related time. The target image frame is obtained by the image acquisition device of the mobile platform capturing images of the parking lot. The platform motion data is obtained by the motion sensor of the mobile platform during its movement. The related time is a time that is a preset time distance from the target time.

[0006] Based on the target time and the associated time, the platform motion data collected at the associated time is calibrated to obtain platform data corresponding to the target time;

[0007] Based on the platform data corresponding to the previous time of the target time, and the platform data corresponding to the target time, data constraint processing is performed to obtain the first constraint relationship between the target image frame and the previous image frame; the previous image frame is the image frame acquired at the previous time of the target time.

[0008] Based on the first constraint relationship and the target image frame, the sub-map corresponding to the previous image frame is updated to obtain the sub-map corresponding to the target image frame.

[0009] The next moment after the target moment is determined as the target moment;

[0010] Repeat the following steps: acquire the target image frame collected at the target time, and the platform motion data collected at the associated time, until the next time of the target time is determined as the target time, until there is no target image frame corresponding to the target time.

[0011] In this embodiment of the application, updating the sub-map corresponding to the previous image frame based on the first constraint relationship and the target image frame to obtain the sub-map corresponding to the target image frame includes:

[0012] If the difference between the platform data corresponding to the previous time and the platform data corresponding to the target time is less than a preset threshold, based on the second constraint relationship between the previous image frame and the historical image frame of the target image frame, the previous image frame is separated from the sub-map corresponding to the previous image frame to obtain the sub-map corresponding to the historical image frame; the historical image frame is an image frame that is adjacent to the previous image frame and whose time sequence is before the previous image frame.

[0013] Determine a third constraint relationship between the historical image frame and the target image frame;

[0014] Based on the third constraint relationship, the target image frame is stitched to the sub-map corresponding to the historical image frame to obtain the sub-map corresponding to the target image frame.

[0015] In this embodiment of the application, updating the sub-map corresponding to the previous image frame based on the first constraint relationship and the target image frame to obtain the sub-map corresponding to the target image frame includes:

[0016] If the difference between the platform data corresponding to the previous time and the platform data corresponding to the target time is greater than or equal to a preset threshold, the target image frame is stitched to the sub-map corresponding to the previous image frame based on the first constraint relationship to obtain the sub-map corresponding to the target image frame.

[0017] In this embodiment of the application, the method includes:

[0018] Multiple target sub-maps are acquired; each target sub-map is constructed based on a preset number of temporally adjacent image frames.

[0019] If duplicate frames exist in the multiple target sub-maps, image frame deduplication processing is performed on the multiple target sub-maps to obtain multiple deduplicated target sub-maps;

[0020] Based on the temporal relationship of the multiple deduplicated target sub-maps, the multiple deduplicated target sub-maps are stitched together to obtain a parking lot map.

[0021] In this embodiment of the application, the platform data includes inertial measurement data and vehicle pulse data; the data constraint processing based on the platform data corresponding to the previous time of the target time and the platform data corresponding to the target time to obtain the first constraint relationship between the target image frame and the previous image frame includes:

[0022] Based on the inertial measurement data corresponding to the previous time of the target time and the inertial measurement data corresponding to the target time, pre-integration is performed to obtain the inertial pose constraint and the inertial position constraint.

[0023] Based on the vehicle pulse data corresponding to the previous time of the target time and the vehicle pulse data corresponding to the target time, the pulse pose constraint and pulse position constraint are obtained by summing and integrating.

[0024] The inertial pose constraint and the pulse pose constraint are fused to obtain the target pose constraint.

[0025] The inertial position constraint and the pulse position constraint are fused to obtain the target position constraint;

[0026] Based on the target pose constraint and the target position constraint, a first constraint relationship between the target image frame and the previous image frame is determined.

[0027] In this embodiment of the application, the target image frame is point cloud data; the data constraint processing based on the platform data corresponding to the previous time of the target time and the platform data corresponding to the target time to obtain the first constraint relationship between the target image frame and the previous image frame includes:

[0028] Based on the inertial measurement data corresponding to the previous time of the target time and the inertial measurement data corresponding to the target time, the point cloud data corresponding to the target image frame is corrected to obtain the corrected point cloud data corresponding to the target image frame.

[0029] Point cloud registration is performed based on the corrected point cloud data corresponding to the target image frame and the sub-map corresponding to the previous image frame to obtain a first constraint relationship between the target image frame and the previous image frame.

[0030] In this embodiment of the application, the step of performing point cloud registration based on the corrected point cloud data corresponding to the target image frame and the sub-map corresponding to the previous image frame to obtain the first constraint relationship between the target image frame and the previous image frame includes:

[0031] Based on the corrected point cloud data corresponding to the target image frame and the sub-map corresponding to the previous image frame, point-line registration is performed to obtain the first residual constraint.

[0032] Based on the corrected point cloud data corresponding to the target image frame and the sub-map corresponding to the previous image frame, point-area registration is performed to obtain the second residual constraint.

[0033] Based on the first residual constraint and the second residual constraint, a first constraint relationship between the target image frame and the previous image frame is determined.

[0034] On the other hand, this application also provides a parking lot map construction device, the device comprising:

[0035] The acquisition module is used to acquire target image frames collected at a target time and platform motion data collected at a related time; the target image frames are obtained by image acquisition devices of the mobile platform on the parking lot; the platform motion data are obtained by motion sensors of the mobile platform during the movement of the mobile platform; the related time is a time that is a preset time length away from the target time.

[0036] The data calibration module is used to perform data calibration on the platform motion data collected at the associated time based on the target time and the associated time, so as to obtain platform data corresponding to the target time.

[0037] The constraint processing module is used to perform data constraint processing based on the platform data corresponding to the previous time of the target time and the platform data corresponding to the target time, to obtain a first constraint relationship between the target image frame and the previous image frame; the previous image frame is the image frame acquired at the previous time of the target time.

[0038] The image update module is used to update the sub-map corresponding to the previous image frame based on the first constraint relationship and the target image frame, so as to obtain the sub-map corresponding to the target image frame.

[0039] A target time determination module is used to determine the next time after the target time as the target time;

[0040] The loop module is used to repeat the following steps: acquiring the target image frame collected at the target time and the platform motion data collected at the associated time, until the next time of the target time is determined as the target time, until there is no target image frame corresponding to the target time.

[0041] On the other hand, this application also provides an electronic device, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the parking map construction method described above.

[0042] On the other hand, this application also provides a computer storage medium storing at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the parking map construction method described above.

[0043] Due to the above technical solutions, the parking lot map construction method described in this application has the following beneficial effects:

[0044] By employing a mobile platform and installing image acquisition devices and motion sensors on it, data collection and map construction of parking lots can be performed based on the movement of the mobile platform, thereby achieving automatic parking lot map construction and improving the accuracy of parking lot construction. During the map construction process, by associating the platform motion data collected at different times, data constraints are applied to the platform data corresponding to the previous time and the platform data corresponding to the target time. The parking lot sub-map can be updated based on the constraint relationship, improving the correlation between image frames in the sub-map, and thus improving the map accuracy of the parking lot sub-map and the overall parking lot map accuracy. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of this application, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0046] Figure 1 This is a schematic diagram of a parking lot map construction method provided in an embodiment of this application;

[0047] Figure 2 This is a schematic diagram of the sub-map acquisition process in a parking lot map construction method provided in an embodiment of this application;

[0048] Figure 3 This is a schematic diagram of the parking map acquisition process in a parking map construction method provided in this application embodiment;

[0049] Figure 4 These are two target sub-maps in adjacent time sequences in a parking lot map construction method provided in this application embodiment;

[0050] Figure 5 This is a sub-map construction example of a parking lot map construction method provided in this application embodiment;

[0051] Figure 6 This is an example of parking map construction using a parking map construction method provided in this application.

[0052] Figure 7 This is a schematic diagram of various constraint processing steps in a parking lot map construction method provided in an embodiment of this application;

[0053] Figure 8 This is a schematic diagram of the point cloud data processing flow in a parking lot map construction method provided in an embodiment of this application;

[0054] Figure 9 This is a schematic diagram of the point cloud registration process in a parking lot map construction method provided in this application embodiment;

[0055] Figure 10 This is a schematic diagram of the structure of a parking lot map construction device provided in an embodiment of this application;

[0056] Figure 11 This is a hardware structure block diagram of a parking lot map construction method provided in an embodiment of this application. Detailed Implementation

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0058] The term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of this application. In the description of this application, it should be understood that the terms "upper," "lower," "left," "right," "top," "bottom," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. Moreover, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein.

[0059] Combination Figure 1 This application introduces a parking lot map construction method provided by an embodiment, the method comprising:

[0060] S1001. Acquire the target image frame acquired at the target time and the platform motion data acquired at the associated time; the target image frame is obtained by the image acquisition device of the mobile platform acquiring images of the parking lot; the platform motion data is obtained by the motion sensor of the mobile platform acquiring images during the movement of the mobile platform; the associated time is the time that is a preset time length away from the target time.

[0061] In the embodiments of this application, the image acquisition device can be an optical ranging sensor, specifically, it can be a lidar or a camera device; the motion sensor can be one or more of the following measurement sensors: IMU (inertial measurement unit), vehicle pulse measurement unit, GPS (Global Positioning System).

[0062] S1002. Based on the target time and the associated time, perform data calibration on the platform motion data collected at the associated time to obtain platform data corresponding to the target time. Data calibration refers to mapping the platform data corresponding to the associated time to the target time using the target time as a reference. Specifically, data integration or data interpolation can be used. Data integration means determining the platform data corresponding to the associated time as the platform data corresponding to the target time when the associated time and the target time are the same. Data interpolation means establishing a linear relationship between two associated times adjacent to the target time when there is no associated time that is the same as the target time, to obtain a linear relationship between platform data and time. Input the target time into the above linear relationship to obtain the platform data corresponding to the target time.

[0063] S1003. Based on the platform data corresponding to the previous time of the target time, and the platform data corresponding to the target time, perform data constraint processing to obtain the first constraint relationship between the target image frame and the previous image frame; the previous image frame is the image frame acquired at the previous time of the target time; data constraint processing refers to solving the correlation relationship between the previous time and the target time in the platform data, which can characterize the platform motion, so that the correlation relationship can characterize the constraint relationship between the target image frame and the previous image frame.

[0064] S1004. Based on the first constraint relationship and the target image frame, update the sub-map corresponding to the previous image frame to obtain the sub-map corresponding to the target image frame; wherein, the latest frame in the sub-map corresponding to the target image frame is the target image frame, and the latest frame in the sub-map corresponding to the previous image frame is the previous image frame.

[0065] S1005. Determine the next moment after the target moment as the target moment;

[0066] S1006. Repeat the following steps: acquire the target image frame acquired at the target time and the platform motion data acquired at the associated time, until the next time after the target time is determined as the target time, until there is no target image frame corresponding to the target time; if there is no target image frame corresponding to the target time, it indicates that the mobile platform has stopped acquiring images, and the sub-map obtained from all the previously acquired image frames is the sub-map of the parking lot map; preferably, the sub-maps are stitched together to obtain the complete parking lot map.

[0067] In this embodiment, a mobile platform is used, and an image acquisition device and motion sensor are set on the mobile platform. This enables data acquisition and map construction of the parking lot based on the movement of the mobile platform, thereby achieving automatic construction of the parking lot map and improving the accuracy of the parking lot construction. During the map construction process, by associating the platform motion data collected at the same time, data constraints are imposed on the platform data corresponding to the previous time and the platform data corresponding to the target time. The parking lot sub-map can be updated based on the constraint relationship, which improves the correlation between image frames in the sub-map, thereby improving the map accuracy of the parking lot sub-map and the overall accuracy of the parking lot map.

[0068] refer to Figure 2 In this embodiment of the application, S1004 includes:

[0069] S2001. If the difference between the platform data corresponding to the previous time step and the platform data corresponding to the target time step is less than a preset threshold, based on the second constraint relationship between the previous image frame and the historical image frame of the target image frame, the previous image frame is separated from the sub-map corresponding to the previous image frame to obtain the sub-map corresponding to the historical image frame; the historical image frame is an image frame that is adjacent to the previous image frame and whose time sequence is before the previous image frame; if the difference between the platform data corresponding to the previous time step and the platform data corresponding to the target time step is less than the preset threshold, it indicates that the mobile platform is stationary or its movement is small; that is, the information content in the previous image frame and the target image frame is similar; the preset threshold is determined based on the actual situation and the order of difference, and this application does not limit it; preferably, the preset threshold includes a rotation matrix threshold and a translation threshold, the rotation matrix threshold can be 0.017°, and the translation threshold can be 0.3m.

[0070] S2002. Determine the third constraint relationship between the historical image frame and the target image frame; preferably, perform data constraint processing based on the platform data corresponding to the target time and the platform data corresponding to the historical image frame to obtain the third constraint relationship between the historical image frame and the target image frame.

[0071] S2003. Based on the third constraint relationship, stitch the target image frame to the sub-map corresponding to the historical image frame to obtain the sub-map corresponding to the target image frame.

[0072] In a specific embodiment of this application, a sliding window can be used to stitch the target image frame with historical image frames, thereby improving the real-time performance of parking lot map construction. Specifically, the process of splitting a certain image frame within the sliding window can employ an edge-mapping process, and the specific edge-mapping model is as follows:

[0073]

[0074] (a×db×c)×x2=a×fe×c (2)

[0075] Where a, b, c, d, e, and f refer to the constraint data in the second constraint relationship; x1 refers to the image frame that needs to be split from the submap; and x2 refers to the submap after splitting the image frame.

[0076] In a specific embodiment of this application, by adopting the above-mentioned edge-division model, the residual constraints in the historical image frame sub-map can be solved; by using the edge-division process for image splitting, anchor points can be preserved for the stitching of target image frames, thereby improving the construction accuracy of the parking lot map.

[0077] In this embodiment, by using the platform data corresponding to the previous moment, the difference between the data and the platform data corresponding to the target moment is less than a preset threshold, indicating that the mobile platform is stationary or has a small amount of movement. This allows the previous image frame to be directly removed when there is too much duplicate information in the previous image frame and the target image frame. The sub-maps corresponding to the target image frame and the historical image frames are then stitched together, thereby reducing the amount of data in the sub-maps corresponding to the target image frame and improving the data accuracy of the sub-maps corresponding to the target image frame.

[0078] In this embodiment of the application, S1004 further includes:

[0079] If the difference between the platform data at the previous time step and the platform data at the target time step is greater than or equal to a preset threshold, based on the first constraint relationship, the target image frame is stitched to the sub-map corresponding to the previous image frame to obtain the sub-map corresponding to the target image frame. If the difference between the platform data at the previous time step and the platform data at the target time step is greater than or equal to the preset threshold, it indicates that the mobile platform is in normal operation, meaning that the information content in the previous image frame of the target image is not completely consistent with the information content in the target image frame.

[0080] In a specific embodiment of this application, a sliding window can be used to stitch the target image frame with the previous image frame, thereby improving the real-time performance of parking lot map construction.

[0081] refer to Figure 3 In this embodiment of the application, the parking lot map construction method further includes:

[0082] S3001. Obtain multiple target sub-maps; each target sub-map is constructed based on a preset number of temporally adjacent image frames; preferably, two adjacent target sub-maps contain duplicate frames; a duplicate frame refers to an image frame existing in two different target sub-maps; a duplicate frame can be 1 frame or 2 frames; each target sub-map can include 25 image frames or 50 image frames, for example... Figure 4 Each target sub-map includes 6 image frames, and the repeated frames of two target sub-maps in adjacent time sequences are three frames.

[0083] S3002. In the case of duplicate frames in multiple target sub-maps, perform image frame deduplication processing on multiple target sub-maps to obtain multiple deduplicated target sub-maps; the deduplication operation includes, but is not limited to, overlapping duplicate frames and deleting duplicate frames in any target sub-map.

[0084] S3003. Based on the temporal relationship of the deduplicated target sub-maps, the deduplicated target sub-maps are stitched together to obtain the parking lot map. Since there are duplicate frames in the target sub-maps, meaning that after deleting duplicate frames, two target sub-maps with adjacent temporal sequences are on the same plane, the parking lot map can be obtained by stitching together the multiple target sub-maps. See details in the reference section. Figure 5-6 , Figure 5 It is a sub-map instance constructed using the parking lot map construction method provided in this application. Figure 6 It is a parking map instance constructed using the parking map construction method provided in this application.

[0085] In this embodiment, by using multiple target sub-maps for stitching, the output of each map construction can be reduced, thereby reducing the amount of data computation and improving the efficiency of parking lot map construction. In addition, when there are duplicate frames in the target sub-maps, they can be directly stitched based on deduplication operations without having to solve the constraint relationships between multiple target sub-maps, thereby reducing the amount of data computation and improving the efficiency of parking lot map construction.

[0086] refer to Figure 7 In this embodiment of the application, the platform data includes inertial measurement data and vehicle pulse data, and S1003 includes:

[0087] S7001. Based on the inertial measurement data corresponding to the previous moment before the target moment and the inertial measurement data corresponding to the target moment, pre-integration is performed to obtain inertial pose constraints and inertial position constraints. The inertial measurement data is obtained by the inertial measurement unit, specifically including the linear acceleration and angular velocity of the moving platform. The vehicle pulse data is obtained by the vehicle pulse measurement unit, specifically including the linear velocity and angular velocity of the moving platform. The inertial pose constraint refers to the pose relationship of the moving platform between two moments obtained by solving the inertial measurement data, specifically the target moment and the previous moment before the target moment. The inertial position constraint refers to the position relationship of the moving platform between two moments obtained by solving the inertial measurement data, specifically the target moment and the previous moment before the target moment.

[0088] In a specific embodiment of this application, the observation model for inertial measurement data is as follows:

[0089]

[0090] in, It refers to the angular velocity at time t in the inertial measurement data. ω refers to the acceleration at time t in inertial measurement data. b (t) refers to the actual angular velocity of the mobile platform at time t, a w (t) refers to the actual acceleration of the mobile platform at time t, b g (t) refers to the angular velocity offset at time t, b a (t) refers to the acceleration bias at time t, η g (t) refers to the additional noise at time t, η a (t) refers to the additional noise at time t. It refers to the rotation matrix at time t, g w The acceleration due to gravity is denoted as 'x'. The rotation matrix refers to the rotation matrix of the world coordinate system in the coordinate system corresponding to the inertial measurement data.

[0091] By performing pre-integration based on the above observation model formula, the inertial pose constraints and inertial position constraints between the target time and the previous time can be obtained. The specific pre-integration method is a conventional pre-integration method, which will not be elaborated here.

[0092] S7002. Based on the vehicle pulse data corresponding to the previous moment of the target time and the vehicle pulse data corresponding to the target time, the pulse pose constraint and pulse position constraint are obtained by summing and integrating. The vehicle pulse data is obtained by the vehicle pulse measurement unit. Specifically, the vehicle pulse data includes the data acquisition time and the number of rotations of the wheels of the mobile platform. The pulse pose constraint refers to the pose relationship of the mobile platform between two moments, specifically the target time and the previous moment of the target time, obtained by solving the vehicle pulse data. The pulse position constraint refers to the position relationship of the mobile platform between two moments, specifically the target time and the previous moment of the target time, obtained by solving the vehicle pulse data.

[0093] In a specific embodiment of this application, based on the structural parameters of the mobile platform, the vehicle pulse data is converted to obtain the linear velocity and angular velocity corresponding to the vehicle pulse data; by integrating the linear velocity and the linear velocity corresponding to the vehicle pulse data between the previous time and the target time, the pulse pose constraint and the pulse position constraint can be obtained.

[0094] S7003. Constraint fusion is performed on inertial pose constraints and pulse pose constraints to obtain target pose constraints. Constraint fusion refers to the process of integrating two constraints into one constraint. Specifically, constraint fusion can be a simple weighted average process or a fusion process with other preset relationships.

[0095] S7004. The inertial position constraint and the pulse position constraint are fused to obtain the target position constraint;

[0096] S7005. Based on the target pose constraint and the target position constraint, determine the first constraint relationship between the target image frame and the previous image frame; specifically, the first constraint relationship may include the target pose constraint and the target position constraint; or it may be the result of fusing the target pose constraint and the target position constraint with other constraints of the same type, such as the position constraint provided by GPS, to obtain the first constraint relationship.

[0097] In this embodiment of the application, by fusing the constraint relationships obtained from data collected by different motion sensors, multiple constraint relationships are integrated into one constraint relationship, thereby improving the accuracy of the first constraint relationship between the target image frame and the previous image frame, and thus improving the accuracy of parking lot map construction.

[0098] refer to Figure 8 In this embodiment of the application, the target image frame is point cloud data; S1003 includes:

[0099] S8001. Based on the inertial measurement data corresponding to the previous time of the target time and the inertial measurement data corresponding to the target time, perform data correction on the point cloud data corresponding to the target image frame to obtain the corrected point cloud data corresponding to the target image frame; the target image frame is acquired by an image acquisition device. When the target image frame is point cloud data, the image acquisition device is a lidar; the lidar acquires data at a preset time interval to generate the corresponding target image frame (laser frame); the preset time interval can be 0.1s or 0.2s, which is set based on the performance of the lidar and is not limited here.

[0100] In a specific embodiment of this application, pre-integration is performed based on the inertial measurement data corresponding to the previous time of the target time and the inertial measurement data corresponding to the target time to obtain the rotation matrix and translation vector of the acquisition pose of the nth laser and the acquisition pose of the initial laser; based on the rotation matrix and translation vector of the acquisition pose of the nth laser and the acquisition pose of the initial laser, data correction is performed on the point cloud data corresponding to the target image frame to obtain the corrected point cloud data corresponding to the target image frame; n is any non-zero natural number.

[0101] S8002. Based on the corrected point cloud data corresponding to the target image frame and the sub-map corresponding to the previous image frame, point cloud registration is performed to obtain the first constraint relationship between the target image frame and the previous image frame. Point cloud registration can be one or more of point-area registration, point-line registration, and other conventional registration methods, which are not limited here.

[0102] In a specific embodiment of this application, since the lidar collects data at preset time intervals, the mobile platform may be in motion during these intervals. Therefore, the data collected by the lidar may have data deviations, which require data correction to reduce or eliminate the deviations. The specific correction formula is as follows:

[0103]

[0104] Where, p i It refers to the i-th point in the corrected point cloud data; q i It refers to the i-th point in the point cloud data before correction; It refers to the rotation matrix between the acquisition pose of the i-th laser point and the acquisition pose of the initial laser point; It refers to the translation vector between the acquisition pose of the i-th laser point and the acquisition pose of the initial laser point.

[0105] In this embodiment of the application, the point cloud data accuracy is improved by performing data correction, thereby improving the point cloud registration accuracy and the parking lot map construction accuracy.

[0106] refer to Figure 9In this embodiment of the application, S5002 includes:

[0107] S9001. Based on the corrected point cloud data corresponding to the target image frame and the sub-map corresponding to the previous image frame, point-line registration is performed to obtain the first residual constraint.

[0108] In a specific embodiment of this application, the formula for point-line registration is as follows:

[0109]

[0110] Among them, residual edge This refers to the first residual constraint; This refers to the rotation matrix between the target image frame and the corresponding sub-map of the previous image frame; p i It refers to any point in the target image frame; t i It refers to the translation vector between the target image frame and the sub-map corresponding to the previous image frame; and It refers to any two points in the sub-map corresponding to the previous image frame.

[0111] S9002. Based on the corrected point cloud data corresponding to the target image frame and the sub-map corresponding to the previous image frame, point-area registration is performed to obtain the second residual constraint.

[0112] In a specific embodiment of this application, the formula for point-area registration is as follows:

[0113]

[0114] Among them, residual plane This refers to the second residual constraint; n T It refers to the planar normal vector of the sub-map corresponding to the previous image frame; This refers to the rotation matrix between the target image frame and the corresponding sub-map of the previous image frame; p i It refers to any point in the target image frame; t i It refers to the translation vector between the target image frame and the sub-map corresponding to the previous image frame; It refers to any point in the sub-map corresponding to the previous image frame.

[0115] S9003. Based on the first residual constraint and the second residual constraint, determine the first constraint relationship between the target image frame and the previous image frame.

[0116] In this embodiment of the application, the first constraint relationship is optimized based on the first residual constraint and the second residual constraint to determine the first constraint relationship between the target image frame and the previous image frame.

[0117] In a specific embodiment of this application, the specific optimization formula is as follows:

[0118] residual=∑residual edge +∑residual plane (7)

[0119] Among them, residual edge For the first residual constraint; residual plane The first residual is the second residual constraint; the second residual is the sum of all first and second residual constraints; the optimal condition is achieved when the sum of the residual constraints is minimized. With t i .

[0120] In this embodiment, the accuracy of the first constraint relationship is improved by using point-line registration and point-area registration, thereby improving the accuracy of parking lot map construction.

[0121] refer to Figure 10 This application also provides a parking lot map construction device, which includes:

[0122] The acquisition module 101 is used to acquire the target image frame collected at the target time and the platform motion data collected at the associated time; the target image frame is obtained by the image acquisition device of the mobile platform to acquire images of the parking lot; the platform motion data is obtained by the motion sensor of the mobile platform during the movement of the mobile platform; the associated time is the time that is a preset time length away from the target time.

[0123] The data calibration module 102 is used to perform data calibration on the platform motion data collected at the associated time based on the target time and the associated time, so as to obtain the platform data corresponding to the target time.

[0124] The constraint processing module 103 is used to perform data constraint processing based on the platform data corresponding to the previous time of the target time and the platform data corresponding to the target time, to obtain the first constraint relationship between the target image frame and the previous image frame; the previous image frame is the image frame acquired at the previous time of the target time.

[0125] The image update module 104 is used to update the sub-map corresponding to the previous image frame based on the first constraint relationship and the target image frame, so as to obtain the sub-map corresponding to the target image frame.

[0126] The target time determination module 105 is used to determine the next time after the target time as the target time;

[0127] The loop module 106 is used to repeat the steps: acquiring the target image frame collected at the target time and the platform motion data collected at the associated time, until the next time of the target time is determined as the target time, until there is no target image frame corresponding to the target time.

[0128] The image update module includes:

[0129] The splitting unit is used to split the previous image frame from the sub-map corresponding to the previous image frame based on the second constraint relationship between the previous image frame and the historical image frame when the data difference between the platform data corresponding to the previous time and the platform data corresponding to the target time is less than a preset threshold, so as to obtain the sub-map corresponding to the historical image frame; the historical image frame is the image frame that is adjacent to the previous image frame and whose time sequence is before the previous image frame.

[0130] The first constraint determination unit is used to determine the third constraint relationship between the historical image frame and the target image frame;

[0131] The first stitching unit is used to stitch the target image frame to the sub-map corresponding to the historical image frame based on the third constraint relationship, so as to obtain the sub-map corresponding to the target image frame.

[0132] The second stitching unit is used to stitch the target image frame to the sub-map corresponding to the previous image frame based on the first constraint relationship, when the difference between the platform data corresponding to the previous time and the platform data corresponding to the target time is greater than or equal to a preset threshold, so as to obtain the sub-map corresponding to the target image frame.

[0133] The parking lot map building device also includes:

[0134] The sub-map acquisition module is used to acquire multiple target sub-maps; each target sub-map is constructed based on a preset number of temporally adjacent image frames.

[0135] The deduplication module is used to perform image frame deduplication on multiple target sub-maps when there are duplicate frames in multiple target sub-maps, and obtain multiple target sub-maps after deduplication.

[0136] The global map stitching module is used to stitch together multiple deduplicated target sub-maps based on their temporal relationship to obtain a parking lot map.

[0137] The platform data includes inertial measurement data and vehicle pulse data, and the constraint processing module includes:

[0138] The pre-integration unit is used to perform pre-integration based on the inertial measurement data corresponding to the previous time step of the target time and the inertial measurement data corresponding to the target time, so as to obtain the inertial pose constraint and the inertial position constraint.

[0139] The sum-integration unit is used to perform sum-integration based on the vehicle pulse data corresponding to the previous time step of the target time and the vehicle pulse data corresponding to the target time, to obtain the pulse pose constraint and the pulse position constraint.

[0140] The first constraint fusion unit is used to fuse inertial pose constraints and pulse pose constraints to obtain target pose constraints.

[0141] The second constraint fusion unit is used to fuse the inertial position constraint and the pulse position constraint to obtain the target position constraint.

[0142] The second constraint determination unit is used to determine the first constraint relationship between the target image frame and the previous image frame based on the target pose constraint and the target position constraint.

[0143] The data correction unit is used to correct the point cloud data corresponding to the target image frame based on the inertial measurement data corresponding to the previous time of the target time and the inertial measurement data corresponding to the target time, so as to obtain the corrected point cloud data corresponding to the target image frame.

[0144] The point cloud registration unit is used to perform point cloud registration based on the corrected point cloud data corresponding to the target image frame and the sub-map corresponding to the previous image frame, to obtain the first constraint relationship between the target image frame and the previous image frame.

[0145] The point cloud registration unit includes:

[0146] The point-line registration unit is used to perform point-line registration based on the corrected point cloud data corresponding to the target image frame and the sub-map corresponding to the previous image frame to obtain the first residual constraint.

[0147] The point-area registration unit is used to perform point-area registration based on the corrected point cloud data corresponding to the target image frame and the sub-map corresponding to the previous image frame to obtain the second residual constraint.

[0148] The first constraint determination unit is used to determine the first constraint relationship between the target image frame and the previous image frame based on the first residual constraint and the second residual constraint.

[0149] This application also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction or at least one program. The processor loads and executes the at least one instruction or at least one program to implement the parking map construction method described above.

[0150] Memory is used to store software programs and modules. The processor executes these stored software programs and modules to perform various functional applications and data processing. Memory can primarily consist of a program storage area and a data storage area. The program storage area stores the operating system, application programs required for functionality, etc.; the data storage area stores data created based on device usage, etc. Furthermore, memory can include high-speed random access memory (RAM) and non-volatile memory, such as at least one hard disk drive, flash memory, or other volatile solid-state storage devices. Correspondingly, memory can also include a memory controller to provide the processor with access to the memory.

[0151] The methods and embodiments provided in this application can be executed in electronic devices such as mobile terminals, computer terminals, servers, or similar computing devices. Figure 11 This is the electronic device provided in the embodiments of this application. For example... Figure 11 As shown, the electronic device 900 can vary significantly due to differences in configuration or performance. It may include one or more central processing units (CPUs) 910 (CPUs 910 may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory 930 for storing data, and one or more storage media 920 (e.g., one or more mass storage devices) for storing application programs 923 or data 922. The memory 930 and storage media 920 may be temporary or persistent storage. The program stored in the storage media 920 may include one or more modules, each module may include a series of instruction operations on the electronic device. Furthermore, the CPU 910 may be configured to communicate with the storage media 920 and execute the series of instruction operations in the storage media 920 on the electronic device 900. Electronic device 900 may also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input / output interfaces 940, and / or one or more operating systems 921, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0152] The input / output interface 940 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device 900. In one example, the input / output interface 940 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 940 may be a radio frequency (RF) module for wireless communication with the Internet.

[0153] Those skilled in the art will understand that Figure 11 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device 900 may also include... Figure 11 The more or fewer components shown, or having the same Figure 11 The different configurations shown.

[0154] Embodiments of this application also provide a storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the parking map construction method as described above.

[0155] The foregoing description has fully disclosed the specific embodiments of this application. It should be noted that any modifications made by those skilled in the art to the specific embodiments of this application do not depart from the scope of the claims. Accordingly, the scope of the claims of this application is not limited to the foregoing specific embodiments.

Claims

1. A method for constructing a parking lot map, characterized in that, include: Acquire the target image frame captured at the target time, and the platform motion data captured at the associated time; The target image frame is obtained by image acquisition of the parking lot using an image acquisition device on a mobile platform. The platform motion data is collected by the motion sensors of the mobile platform during its movement. The associated time is the time that is a preset time length away from the target time; Based on the target time and the associated time, the platform motion data collected at the associated time is calibrated to obtain platform data corresponding to the target time; Based on the platform data corresponding to the previous time of the target time and the platform data corresponding to the target time, data constraint processing is performed to obtain the first constraint relationship between the target image frame and the previous image frame. The previous image frame is the image frame acquired at the time preceding the target time. Based on the first constraint relationship and the target image frame, the sub-map corresponding to the previous image frame is updated to obtain the sub-map corresponding to the target image frame. The next moment after the target moment is determined as the target moment; Repeat the following steps: acquire the target image frame collected at the target time, and the platform motion data collected at the associated time, until the next time of the target time is determined as the target time, until there is no target image frame corresponding to the target time.

2. The method according to claim 1, characterized in that, The step of updating the sub-map corresponding to the previous image frame based on the first constraint relationship and the target image frame to obtain the sub-map corresponding to the target image frame includes: If the difference between the platform data corresponding to the previous time and the platform data corresponding to the target time is less than a preset threshold, based on the second constraint relationship between the previous image frame and the historical image frame of the target image frame, the previous image frame is separated from the sub-map corresponding to the previous image frame to obtain the sub-map corresponding to the historical image frame; the historical image frame is an image frame that is adjacent to the previous image frame and whose time sequence is before the previous image frame. Determine a third constraint relationship between the historical image frame and the target image frame; Based on the third constraint relationship, the target image frame is stitched to the sub-map corresponding to the historical image frame to obtain the sub-map corresponding to the target image frame.

3. The method according to claim 2, characterized in that, The step of updating the sub-map corresponding to the previous image frame based on the first constraint relationship and the target image frame to obtain the sub-map corresponding to the target image frame includes: If the difference between the platform data corresponding to the previous time and the platform data corresponding to the target time is greater than or equal to a preset threshold, the target image frame is stitched to the sub-map corresponding to the previous image frame based on the first constraint relationship to obtain the sub-map corresponding to the target image frame.

4. The method according to claim 1, characterized in that, The method includes: Multiple target sub-maps are acquired; each target sub-map is constructed based on a preset number of temporally adjacent image frames. If duplicate frames exist in the multiple target sub-maps, image frame deduplication processing is performed on the multiple target sub-maps to obtain multiple deduplicated target sub-maps; Based on the temporal relationship of the multiple deduplicated target sub-maps, the multiple deduplicated target sub-maps are stitched together to obtain a parking lot map.

5. The method according to claim 1, characterized in that, The platform data includes inertial measurement data and vehicle pulse data; the data constraint processing based on the platform data corresponding to the previous time of the target time and the platform data corresponding to the target time to obtain the first constraint relationship between the target image frame and the previous image frame includes: Based on the inertial measurement data corresponding to the previous time of the target time and the inertial measurement data corresponding to the target time, pre-integration is performed to obtain the inertial pose constraint and the inertial position constraint. Based on the vehicle pulse data corresponding to the previous time of the target time and the vehicle pulse data corresponding to the target time, the pulse pose constraint and pulse position constraint are obtained by summing and integrating. The inertial pose constraint and the pulse pose constraint are fused to obtain the target pose constraint. The inertial position constraint and the pulse position constraint are fused to obtain the target position constraint; Based on the target pose constraint and the target position constraint, a first constraint relationship between the target image frame and the previous image frame is determined.

6. The method according to claim 5, characterized in that, The target image frame is point cloud data; the data constraint processing based on the platform data corresponding to the previous time of the target time and the platform data corresponding to the target time to obtain the first constraint relationship between the target image frame and the previous image frame includes: Based on the inertial measurement data corresponding to the previous time of the target time and the inertial measurement data corresponding to the target time, the point cloud data corresponding to the target image frame is corrected to obtain the corrected point cloud data corresponding to the target image frame. Point cloud registration is performed based on the corrected point cloud data corresponding to the target image frame and the sub-map corresponding to the previous image frame to obtain a first constraint relationship between the target image frame and the previous image frame.

7. The method according to claim 6, characterized in that, The step of performing point cloud registration based on the corrected point cloud data corresponding to the target image frame and the sub-map corresponding to the previous image frame to obtain the first constraint relationship between the target image frame and the previous image frame includes: Based on the corrected point cloud data corresponding to the target image frame and the sub-map corresponding to the previous image frame, point-line registration is performed to obtain the first residual constraint. Based on the corrected point cloud data corresponding to the target image frame and the sub-map corresponding to the previous image frame, point-area registration is performed to obtain the second residual constraint. Based on the first residual constraint and the second residual constraint, a first constraint relationship between the target image frame and the previous image frame is determined.

8. A parking lot map construction device, characterized in that, include: The acquisition module is used to acquire the target image frame collected at the target time, as well as the platform motion data collected at the associated time. The target image frame is obtained by the image acquisition device of the mobile platform from the parking lot; the platform motion data is obtained by the motion sensor of the mobile platform during the movement of the mobile platform. The associated time is the time that is a preset time length away from the target time; The data calibration module is used to perform data calibration on the platform motion data collected at the associated time based on the target time and the associated time, so as to obtain platform data corresponding to the target time. The constraint processing module is used to perform data constraint processing based on the platform data corresponding to the previous time of the target time and the platform data corresponding to the target time, so as to obtain the first constraint relationship between the target image frame and the previous image frame. The previous image frame is the image frame acquired at the time preceding the target time. The image update module is used to update the sub-map corresponding to the previous image frame based on the first constraint relationship and the target image frame, so as to obtain the sub-map corresponding to the target image frame. A target time determination module is used to determine the next time after the target time as the target time; The loop module is used to repeat the following steps: acquiring the target image frame collected at the target time and the platform motion data collected at the associated time, until the next time of the target time is determined as the target time, until there is no target image frame corresponding to the target time.

9. A computer storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, which is loaded and executed by a processor to implement the parking map construction method as described in any one of claims 1-8.

10. An electronic device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the parking map construction method as described in any one of claims 1-7.

Images