Autonomous driving map generation method, device, electronic device and storage medium

Abstract

The present application provides an autonomous driving map generation method, device, electronic device and storage medium. The method obtains environmental image information and vehicle position information of a vehicle, performs inverse depth projection processing on the environmental image information to obtain a processed image, performs image segmentation on the processed image to obtain a plurality of sub-images, obtains trajectory data of the image acquisition device in a sub-image from a start moment to an end moment based on the position of the image acquisition device at different moments in the sub-image, determines a projection plane based on the pose difference between the first frame image and the last frame image in the trajectory data, and makes the projection plane pass through the first frame image and the last frame image, and uses the last frame image as the first frame image of the next sub-image, so that all sub-images are connected through the projection plane to form a driving map. This method achieves low cost, does not require additional equipment, is short in time, and is conducive to the generation of autonomous driving maps for assisting vehicles in performing high-precision positioning.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving, and specifically to an autonomous driving map generation method, device, electronic device, and storage medium. Background Art

[0002] Acquiring a map of the environment in which autonomous driving operates is the first step in achieving autonomous driving. With a known map, positioning and path planning are performed, and then autonomous driving is achieved through vehicle-side control. Therefore, acquiring a map for autonomous driving, or a high-precision map, is the first step in achieving autonomous driving capabilities.

[0003] Currently, high-precision maps are primarily produced manually, using millions of fiber-optic inertial navigation systems. For example, a map-building method that integrates vision and lasers can be used. However, this approach requires the deployment of additional relay devices, which increases costs. Alternatively, a SLAM-oriented outdoor positioning method based on consumer-grade GPS and 2.5D building models can be used. Feature point extraction and triangulation are used to obtain feature point depths, and subsequently construct a 2.5D map. However, feature point extraction is relatively time-consuming, and existing methods typically store 2D information, making it difficult to assist vehicles in high-precision positioning. Summary of the Invention

[0004] In view of the above-mentioned shortcomings of the prior art, the present invention provides an autonomous driving map generation method, device, electronic device and storage medium to solve the above-mentioned technical problems.

[0005] The autonomous driving map generation method provided in this application includes:

[0006] Acquire environmental image information and vehicle position information of the vehicle, wherein the environmental image information is acquired based on an onboard image acquisition device;

[0007] Performing inverse depth projection processing on the environmental image information to obtain a processed image, and performing image segmentation on the processed image to obtain a plurality of sub-image images;

[0008] Obtaining trajectory data of the image acquisition device from a start time to an end time in a sub-image according to the position of the image acquisition device at different times in the sub-image;

[0009] Determine a projection plane according to a pose difference between a first frame image and a last frame image in the trajectory data, and make the projection plane pass through the first frame image and the last frame image;

[0010] The last frame image is used as the first frame image of the next sub-image, so that all sub-images are connected through the projection plane to form a driving map.

[0011] In one embodiment of the present application, obtaining trajectory data of the image acquisition device from a start time to an end time within a sub-image according to the position of the image acquisition device at different times within the sub-image includes:

[0012] If the vehicle position information is obtained normally, position constraints are applied to the vehicle position information to obtain the positions of the image acquisition device at different moments in the sub-image;

[0013] If an abnormality occurs in obtaining the vehicle position information, the vehicle's posture difference when obtaining the environmental image information from the start time to the end time is calculated based on the position information and state parameters, and the position of the image acquisition device at different times in the sub-image is obtained based on the posture difference. The state parameters include the vehicle's linear velocity, angular velocity and acceleration.

[0014] In one embodiment of the present application, obtaining the position of the image acquisition device at different moments in the sub-image based on the posture difference includes:

[0015] Recording the vehicle's posture at the start time and the vehicle's posture at the end time to obtain the posture difference;

[0016] The state parameters between adjacent frame images in the sub-image are used as posture compensation to perform dead reckoning to obtain the dead reckoning result;

[0017] An optimization model is constructed based on the pose difference and dead reckoning results to obtain the position of the image acquisition device at different moments in the sub-image.

[0018] In one embodiment of the present application, constructing an optimization model based on the posture difference and dead reckoning results includes:

[0019] determining a total error based on the pose difference and the dead reckoning result;

[0020] The optimization model is determined based on the goal that the total error gradually approaches zero.

[0021] In one embodiment of the present application, before performing image segmentation on the processed image to obtain a plurality of sub-images, the method further includes:

[0022] If the vehicle position information is normally acquired, the processed image is segmented according to a preset distance threshold, the processed image is segmented according to a preset horizontal rotation angle threshold, and the processed image is segmented according to a preset elevation angle threshold;

[0023] If an abnormality occurs in obtaining the vehicle position information, image segmentation will not be performed until the vehicle position information is obtained normally.

[0024] In one embodiment of the present application, obtaining trajectory data of the image acquisition device from a start time to an end time within a sub-image according to the position of the image acquisition device at different times within the sub-image includes:

[0025] The vehicle position information is interpolated using the following formula:

[0026]

[0027] Among them, t c is the acquisition time of the environmental image information, t1 and t2 are the acquisition times of the adjacent position information, p1 and p2 are the acquired adjacent position information, and p c t c The location information collected at all times, and t1<t c <t2;

[0028] Based on the position information after position interpolation processing, the trajectory data of the image acquisition device in the sub-image from the starting time to the ending time is obtained.

[0029] In one embodiment of the present application, determining a projection plane according to a pose difference between a first frame image and a last frame image in the trajectory data, and making the projection plane pass through the first frame image and the last frame image includes:

[0030] The position of the first frame image in the trajectory data is used as the initial reference position, and the posture difference between the first frame image and the last frame image is calculated;

[0031] Pitch angle compensation is performed according to the posture difference to obtain a reference posture, and projection is performed based on the reference posture so that the projection plane passes through the first frame image and the last frame image.

[0032] In one embodiment of the present application, performing projection based on the reference pose includes:

[0033] Obtaining a transformation relationship between the projection plane and the world coordinate system;

[0034] A three-dimensional point cloud is generated in a vehicle coordinate system based on the environmental image information and vehicle position information of the vehicle, and the three-dimensional point cloud is projected to a world coordinate system. A secondary projection is performed based on the transformation relationship, and the three-dimensional point cloud projected to the world coordinate system is converted into a two-dimensional point cloud by compressing the occupied grid map.

[0035] In one embodiment of the present application, performing pitch angle compensation according to the attitude difference to obtain a reference posture includes:

[0036] Decomposing the attitude difference to obtain pitch angle, yaw angle and roll angle;

[0037] Mapping the pitch angle, the yaw angle, and the roll angle to each other through a preset rotation matrix;

[0038] Setting the yaw angle and the roll angle to zero to determine a compensation matrix;

[0039] The attitude difference is compensated for a pitch angle according to the compensation matrix to obtain a reference attitude.

[0040] According to one aspect of an embodiment of the present application, there is provided an apparatus for generating an autonomous driving map, comprising:

[0041] An image acquisition module is used to obtain environmental image information of the vehicle;

[0042] Position information module, used to obtain vehicle position information;

[0043] An image processing module is used to perform inverse depth projection processing on the environmental image information to obtain a processed image, and perform image segmentation on the processed image to obtain a plurality of sub-images;

[0044] A trajectory module, configured to obtain trajectory data of the image acquisition device from a start time to an end time within a sub-image according to the position of the image acquisition device at different times within the sub-image;

[0045] A projection plane module, configured to determine a projection plane according to a posture difference between a first frame image and a last frame image in the trajectory data, and make the projection plane pass through the first frame image and the last frame image;

[0046] The output module is configured to use the last frame image as the first frame image of the next sub-image, so that all sub-images are connected through the projection plane to form a driving map. According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the electronic device implements the autonomous driving map generation method described above.

[0047] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a computer, the computer executes the autonomous driving map generation method as described above.

[0048] In the technical solutions provided in some embodiments of the present application, the autonomous driving map generation method, device, electronic device, and storage medium of the present application obtain environmental image information and vehicle position information of the vehicle, perform inverse depth projection processing on the environmental image information to obtain a processed image, perform image segmentation on the processed image to obtain a plurality of sub-images, obtain trajectory data of the image acquisition device within the sub-image from the start time to the end time based on the position of the image acquisition device at different times within the sub-image, determine the projection plane based on the pose difference between the first frame image and the last frame image in the trajectory data, and make the projection plane pass through the first frame image and the last frame image, and use the last frame image as the first frame image of the next sub-image, so that all sub-images are connected through the projection plane to form a driving map. The autonomous driving map generation method is low-cost, does not require additional equipment, is short in time, and is conducive to assisting vehicles in high-precision positioning. In this application, a map between 2D and 3D is used, which is called a 2.5D map. This application splits the map into multiple local sub-images, with the sub-images using a 2D format and the connections between the sub-images using a 3D format. Doing so can, on the one hand, reduce the amount of data stored. At the same time, the ultimate goal of autonomous driving is actually to control the vehicle's throttle, brakes, and steering. This type of control algorithm is essentially completed in a planar 2D coordinate system, so providing a local 2D map is more beneficial for controlling the vehicle. Finally, a local 2D map can help with multi-temporal data fusion. That is, the local maps collected by different vehicles at different times will have certain errors in 3D positions due to the limited cost of sensors. This patent uses a 2D plane to fuse them, which has more mature operability than 3D.

[0049] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are incorporated into and constitute a part of the specification, illustrating embodiments consistent with the present application and, together with the specification, serving to explain the principles of the present application. It is obvious that the drawings described below are merely some embodiments of the present application, and a person of ordinary skill in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0051] Figure 1 is a schematic diagram of the system architecture of an autonomous driving map generation method shown in an exemplary embodiment of the present application;

[0052] Figure 2 is a flowchart of a method for generating an autonomous driving map according to an exemplary embodiment of the present application;

[0053] Figure 3is a projection diagram of an autonomous driving map generation method shown in an exemplary embodiment of the present application;

[0054] Figure 4 1 is a schematic diagram of a subgraph data structure of an autonomous driving map generation method according to an exemplary embodiment of the present application;

[0055] Figure 5 This is a schematic diagram of a judgment flow of a method for generating an autonomous driving map according to an exemplary embodiment of the present application;

[0056] Figure 6 1 is a schematic diagram of sub-graph camera poses of an autonomous driving map generation method shown in an exemplary embodiment;

[0057] Figure 7 is a schematic diagram of pose graph optimization of an autonomous driving map generation method shown in an exemplary embodiment of the present application;

[0058] Figure 8 is a schematic structural diagram of an autonomous driving map generation device shown in an exemplary embodiment of the present application;

[0059] Figure 9 A schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0060] The following describes the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art will readily appreciate the other advantages and benefits of the present invention from the disclosure herein. The present invention may also be implemented or applied through various other specific embodiments, and the various details in this specification may be modified or altered based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are intended only to illustrate the present invention and are not intended to limit the scope of protection of the present invention.

[0061] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0062] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention.

[0063] In this application, "plurality" refers to two or more. "And / or" describes the relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0064] Figure 1 It is a schematic diagram of an exemplary system architecture shown in an exemplary embodiment of the present application.

[0065] Reference Figure 1 As shown, the system architecture may include a vehicle 101, a cloud 102, and a computer device 103. The computer device 103 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, a neural network computer, and the like. Relevant technicians can use the computer device 103 to generate an autonomous driving map. The computer device 103 can communicate with the vehicle 101 through the cloud 102 to obtain corresponding information and generate an autonomous driving map. For example, by storing vehicle location information such as GPS data in the cloud 102, and also storing environmental image information collected by the vehicle 101 in the cloud 102, the vehicle location information is transmitted to the computer device 103 to generate an autonomous driving map.

[0066] Illustratively, after obtaining vehicle location information and vehicle environment image information from cloud 102, computer device 103 executes the autonomous driving map generation method provided in this embodiment to generate an autonomous driving map. In one embodiment, the computer device can also be directly installed in the vehicle to enable the vehicle to generate the autonomous driving map locally. In one embodiment, the computer device can also be a server or other device known to those skilled in the art that is capable of executing the method provided in this embodiment.

[0067] It should be noted that the autonomous driving map generation method provided in the embodiment of the present application is generally executed by the autonomous driving map generation device 101.

[0068] The following is a detailed description of the implementation details of the technical solution of the embodiment of the present application:

[0069] Figure 2 is a flowchart of an exemplary embodiment of the present invention showing a method for generating an autonomous driving map. The method for generating an autonomous driving map can be executed by an autonomous driving map generating device, which can be Figure 1 The computer device 103 shown in FIG. Figure 2 As shown, the method includes at least steps S210 to S250, which are described in detail as follows:

[0070] Step S210: Acquire environmental image information and vehicle position information of the vehicle, wherein the environmental image information is acquired based on the vehicle-mounted image acquisition device.

[0071] In one embodiment of the present application, the vehicle-mounted image acquisition device may be one or more vehicle-mounted cameras. The vehicle-mounted image acquisition device may capture 360-degree video of the vehicle, as well as images of the front, left, and right sides of the vehicle, or images of areas specified by those skilled in the art as needed. The images captured by the vehicle-mounted image acquisition device may be two-dimensional images, or the environment image information may be captured using binocular cameras or multiple cameras, and the images captured by each camera may be integrated to obtain a three-dimensional image.

[0072] In an optional embodiment, the environmental image information may be a plurality of video frames, each of which is marked with a timestamp. In another embodiment, the environmental image information may be a video stream within a certain time period.

[0073] The vehicle location information may be determined by combining GPS (Global Positioning System), BeiDou and other devices, or by other methods known to those skilled in the art.

[0074] An inertial navigation system (INS), also known as an inertial reference system, is an autonomous navigation system that does not rely on external information or radiate energy (like radio navigation). Its operating environment includes not only air and ground conditions, but also underwater. The basic operating principle of inertial navigation is based on Newton's laws of mechanics. By measuring the acceleration of a vehicle in an inertial reference frame, integrating it over time, and transforming it into a navigation coordinate system, information such as velocity, yaw angle, and position in the navigation coordinate system can be obtained.

[0075] In one embodiment, vehicle position information can be collected by a vehicle-mounted combined inertial navigation device. For example, GPS and Beidou signals can be acquired through an antenna to calculate the longitude and latitude of the vehicle. RTK (Real-time kinematic) differential compensation can improve accuracy from 10m to 20cm (the above accuracy improvement is for example and does not limit the present invention). The inertial device in the device can measure acceleration and angular velocity information. The device also reads the vehicle's wheel speed pulses to estimate the vehicle's accurate speed. The vehicle speed, angular velocity, and acceleration can be used to compensate for the inter-frame pose of the image.

[0076] In one embodiment, the vehicle position information includes the vehicle posture, and it can be considered that the environmental image information collected at the time of positioning the vehicle posture has the same posture as the vehicle posture.

[0077] It should be noted that the image data (environmental image information) and GPS data (vehicle position information) need to be timed based on the same clock source to ensure time synchronization, or the vehicle position information and the environmental image information are not timed based on the same clock source, but the time conversion relationship between their clock sources is known, and the two clock sources can be calibrated in advance to obtain the time conversion matrix before subsequent autonomous driving map generation.

[0078] It should be noted that the implementation of the method in this embodiment is based on placing the vehicle and the viewing range of the vehicle-mounted image acquisition device in the same plane.

[0079] Step S220 , performing inverse depth projection processing on the environment image information to obtain a processed image, and performing image segmentation on the processed image to obtain a plurality of sub-images.

[0080] In one embodiment, before depth inversion processing is performed on the environmental image information, semantic segmentation preprocessing is also performed on the environmental image information. Assuming that the viewing range of the vehicle-mounted image acquisition device is a plane, the 2D pixel points of the semantically segmented environmental image information are projected into the 3D coordinate system of the vehicle body.

[0081] See also Figure 3 , Figure 3 is a schematic diagram of a processed image shown in an exemplary embodiment of the present application.

[0082] An example inverse depth projection processing method is as follows:

[0083] The position of the 2D pixel point in the vehicle coordinate system is p v =[x v ,y v , z v ] T , the normalized coordinates of the point in the pixel coordinate system are p i =[u, v, 1] T .

[0084] First, each pixel can be identified as a vector in space as follows,

[0085] v:v=RK -1 p I Formula (1);

[0086] Get the normalized vector of v as Then the coordinates in the vehicle coordinate system are obtained as follows:

[0087]

[0088] This allows us to project the 2D pixel points of the image plane into the 3D space in the vehicle coordinate system.

[0089] If the environmental image information includes multiple image frames, the 3D points (three-dimensional point cloud) of each image frame can be obtained by the above method, and then a continuous point cloud can be obtained.

[0090] In an optional embodiment, inverse depth projection relies on the intrinsic and extrinsic parameter matrices of the camera (onboard image acquisition device). These parameters (intrinsic and extrinsic matrices) are obtained through field calibration and online correction, and can be directly obtained from the camera parameters.

[0091] In one embodiment, before segmenting the processed image to obtain a plurality of sub-images, the method further includes:

[0092] If the vehicle position information is obtained normally, the processed image is segmented according to a preset distance threshold, the processed image is segmented according to a preset horizontal rotation angle threshold, and the processed image is segmented according to a preset elevation angle threshold;

[0093] If an abnormality occurs in obtaining the vehicle position information, image segmentation will not be performed until the vehicle position information is obtained normally.

[0094] In other words, image segmentation is performed on the processed image only for the environmental image information having the corresponding vehicle position information, and otherwise, image segmentation is not performed.

[0095] For example, 1) when the GPS is normal, truncate it into a sub-graph every 80 meters; 2) when the horizontal rotation exceeds 85°, truncate it into a sub-graph; 3) when the pitch angle exceeds 5°, truncate it into a sub-graph; 4) if the GPS is blocked or interfered with (that is, an abnormality occurs in obtaining vehicle location information), ignore the above three rules until the GPS is normal.

[0096] Step S230 , obtaining trajectory data of the image acquisition device from the start time to the end time in a sub-image according to the position of the image acquisition device at different times in the sub-image.

[0097] In an optional embodiment, obtaining trajectory data of the image acquisition device from the start time to the end time in a sub-image according to the position of the image acquisition device at different times in the sub-image includes:

[0098] If the vehicle position information is obtained normally, the position of the image acquisition device at different times within the sub-image is obtained by constraining the vehicle position information;

[0099] If an abnormality occurs in obtaining the vehicle position information, the vehicle's posture difference from the start time to the end time when obtaining the environmental image information is calculated based on the position information and state parameters. Based on the posture difference, the position of the image acquisition device at different times in the sub-image is obtained. The state parameters include the vehicle's linear velocity, angular velocity and acceleration.

[0100] In an optional embodiment, obtaining the position of the image acquisition device at different moments in the sub-image based on the posture difference includes:

[0101] Record the vehicle's posture at the start time and the vehicle's posture at the end time to obtain the posture difference;

[0102] The state parameters between adjacent frame images in the sub-image are used as posture compensation to perform dead reckoning to obtain the dead reckoning result;

[0103] According to the pose difference and dead reckoning results, an optimization model is constructed to obtain the position of the image acquisition device at different times within the sub-image.

[0104] In one embodiment, constructing an optimization model based on the pose difference and dead reckoning results includes:

[0105] Determine the total error based on the attitude difference and dead reckoning results;

[0106] The optimization model is determined based on the goal that the total error gradually approaches zero.

[0107] For example, when the GPS is normal (vehicle position information is obtained normally), the GPS observation is used as the error equation:

[0108]

[0109] Among them, T wv For optimization variables, a 4x4 matrix; is the GPS position; Log(.) is the logarithmic mapping under the Lie group. When GPS is interfered with, this error equation does not exist.

[0110] Use the dead reckoning result between the two frames as the second set of error equations:

[0111]

[0112] in, is the optimization variable at two adjacent moments, It is dead reckoning calculated by linear velocity, angular velocity and acceleration.

[0113] here The calculation requires calculating the difference in the vehicle's posture from t1 to t2 when semantic data is obtained. In addition to recording the posture at t1 and t2, all linear and angular velocity information between these two times is also required. We use a point mass model and assume that the center of the vehicle is at the center of the rear axle. The resulting differential equation for dead reckoning is:

[0114]

[0115] Where p represents position, R represents attitude, and v represents velocity. Here, the vehicle linear velocity is taken as v x , then v=[v x , 0, 0] T w is the angular velocity, [.] × represents an antisymmetric matrix.

[0116] In summary, the two sets of error equations constitute the total error equation:

[0117] E=∑||e1||2+∑||e2||2 Formula (12);

[0118] According to formulas (9) to (11), the optimization equation can be constructed Solve T by letting the error E gradually approach 0 wv , to calculate the trajectory T of all cameras in the subgraph wv .

[0119] In an optional embodiment, since the sampling rhythm of GPS and the sampling rhythm of image are not consistent, the sampling time of GPS and the sampling time of image may not coincide, which requires the use of position interpolation to obtain the actual position of the image. At this time, according to the position of the image acquisition device at different times in a sub-image, the trajectory data of the image acquisition device in the sub-image from the start time to the end time is obtained, including

[0120] The vehicle position information is interpolated using the following formula:

[0121]

[0122] Among them, t c is the acquisition time of the environmental image information, t1 and t2 are the acquisition times of the adjacent position information, p1 and p2 are the acquired adjacent position information, and p c t c The location information collected at all times, and t1<t c <t2;

[0123] Based on the position information after position interpolation processing, the trajectory data of the image acquisition device in the sub-image from the starting time to the ending time is obtained.

[0124] Step S240 : determining a projection plane according to the pose difference between the first frame image and the last frame image in the trajectory data, and making the projection plane pass through the first frame image and the last frame image.

[0125] In one embodiment, determining a projection plane according to a pose difference between a first frame image and a last frame image in the trajectory data, and making the projection plane pass through the first frame image and the last frame image includes:

[0126] The position of the first frame image in the trajectory data is used as the initial reference position, and the posture difference between the first frame image and the last frame image is calculated;

[0127] Pitch angle compensation is performed according to the attitude difference to obtain a reference pose, and projection is performed based on the reference pose so that the projection plane passes through the first frame image and the last frame image.

[0128] In one embodiment, performing projection based on a reference pose includes:

[0129] Get the transformation relationship between the projection plane and the world coordinate system;

[0130] A three-dimensional point cloud in the vehicle coordinate system is generated based on the vehicle's environmental image information and vehicle position information, and the three-dimensional point cloud is projected to the world coordinate system. A secondary projection is performed based on the transformation relationship, and the three-dimensional point cloud projected to the world coordinate system is converted into a two-dimensional point cloud by compressing the occupied grid map.

[0131] For example, all 3D semantic point clouds in the vehicle coordinate system are projected into the world coordinate system and then projected into a 2D point cloud based on the projection plane. When projecting into a 2D point cloud, an occupancy grid is used, with each grid representing 0.1m x 0.1m. The 2D point cloud is also compressed by converting it into an occupancy grid. Finally, the connection between the previous sub-graph and the current sub-graph is recorded.

[0132] In one embodiment, performing pitch angle compensation according to the attitude difference to obtain a reference posture includes:

[0133] Decompose the attitude difference to obtain the pitch angle, yaw angle and roll angle;

[0134] Mapping to pitch angle, yaw angle and roll angle through the preset rotation matrix;

[0135] Set the yaw and roll angles to zero to determine the compensation matrix;

[0136] The attitude difference is compensated for the pitch angle according to the compensation matrix to obtain the reference attitude.

[0137] For example, assuming there are n trajectories, Use the position of the first frame as the reference position for initialization The projection plane needs to pass through the first frame and the last frame, so for T wm Compensate a pitch angle. First calculate the attitude difference:

[0138]

[0139] Then the pose difference between the first frame and the last frame is The pitch angle can be solved by decomposing the yaw-pitch-roll.

[0140]

[0141] Among them, g(.) expresses the mapping of the rotation matrix to yaw (yaw angle), pitch (pitch angle), and roll (roll angle). Then set yaw and roll to zero and calculate the compensated rotation matrix R pitch (Compensation matrix). At this time, the reference pose is:

[0142]

[0143] In one embodiment, all 3D point clouds p w Projected to the map coordinate system:

[0144]

[0145] p m Setting the z-axis coordinate to zero allows projection onto the 2D plane while ensuring that the projection plane passes through the first and last frames.

[0146] Step S250 : Using the last frame image as the first frame image of the next sub-image, so that all sub-images are connected through the projection plane to form a driving map.

[0147] For example, if the current subgraph contains n frames of data, then delete frames 1 to n-1 and keep the nth frame as the first frame of the next subgraph.

[0148] In one embodiment, the driving map may be a 2.5D semantic map. Figure 4 , Figure 4 This is a schematic diagram of a driving map and sub-map data structure shown in an exemplary embodiment of the present application. Figure 4 As shown, the data structure of the subgraph includes: 3D reference pose (which can be determined based on vehicle position information), 2D point cloud within the subgraph, reference path within the subgraph, and connection relationship with the next subgraph.

[0149] The autonomous driving map generation method provided by the above embodiment obtains the vehicle's environmental image information and vehicle position information, performs inverse depth projection processing on the environmental image information to obtain a processed image, performs image segmentation on the processed image to obtain a number of sub-images, and obtains the trajectory data of the image acquisition device in the sub-image from the start time to the end time based on the position of the image acquisition device at different times in a sub-image. The projection plane is determined based on the posture difference between the first frame image and the last frame image in the trajectory data, and the projection plane is made to pass through the first frame image and the last frame image. The last frame image is used as the first frame image of the next sub-image, so that all sub-images are connected through the projection plane to form a driving map. This provides a low-cost autonomous driving map generation method that does not require additional equipment, is short in time, and is conducive to assisting vehicles in performing high-precision positioning.

[0150] The first step in achieving autonomous driving is obtaining a map of the environment. With this map, positioning and path planning are performed, and then vehicle-side control is completed. Therefore, obtaining a map or high-precision map for autonomous driving is the first step in autonomous driving.

[0151] In this application, a map between 2D and 3D is used, which is called a 2.5D map. This application splits the map into multiple local sub-graphs, with the sub-graphs in 2D form, and the connections between the sub-graphs in 3D form. Doing so can, on the one hand, reduce the amount of data storage, and at the same time, the ultimate goal of autonomous driving is actually to control the vehicle's throttle, brakes and steering. This type of control algorithm is essentially completed in a planar 2D coordinate system, so providing a local 2D map is more beneficial for controlling the vehicle. Finally, a local 2D map can help with multi-spatiotemporal data fusion. That is, the local maps collected by different vehicles at different times will have certain errors in 3D positions due to the limited cost of sensors. This patent uses a 2D plane to fuse them, which has more mature operability than 3D.

[0152] The high-precision maps in related technologies are mainly collected through million-level fiber-optic inertial navigation and produced manually. The following is an example of the above-mentioned autonomous driving map generation method through a specific embodiment. This method collects data through mass-produced vehicle-mounted sensors to generate maps. The 3D spatial position and posture of the vehicle are mainly obtained through the combined inertial navigation on the vehicle, and then the semantic information is obtained through the on-board camera. The so-called semantic information is the type of element captured in the image, such as vehicles, zebra crossings, arrows or lane lines, etc. Through the three-dimensional posture and semantic segmentation results of the vehicle, and the inverse depth projection algorithm, a continuous point cloud in three-dimensional space can be obtained, and then a 2.5D semantic map can be obtained through plane compression. The implementation of the above method depends on the following parts:

[0153] 1. The intrinsic parameter matrix and extrinsic parameter matrix of the vehicle-mounted camera;

[0154] 2. Semantic segmentation results of the vehicle-mounted camera;

[0155] 3. Vehicle-mounted combined inertial navigation system.

[0156] Among them, the intrinsic parameter matrix and extrinsic parameter matrix of the vehicle-mounted camera are synchronized with the time of the vehicle-mounted combined inertial navigation system. The intrinsic parameters of the camera are mainly focal length (f x , f y ) and the center point (c x , c y ) is usually expressed as a 3x3 matrix K, as follows:

[0157]

[0158] The camera's extrinsic parameters consist of two parts: rotation (R) and translation (t). The extrinsic parameters are usually expressed as T, where Here T is a 4x4 matrix, R is a 3x3 matrix, t=[t x t y t z ] T is a 3x1 vector.

[0159] Semantic segmentation of the vehicle camera, projecting the required 2D pixels into the vehicle coordinate system through inverse depth projection Figure 3 Typically, during the inverse depth projection process, in this embodiment, the vehicle and the viewing range of the vehicle-mounted camera are considered to be in the same plane.

[0160] The vehicle-mounted combined inertial navigation system acquires GPS and Beidou signals via antennas, calculating the vehicle's latitude and longitude. RTK differential compensation improves accuracy from 10 meters to 20 centimeters. The inertial components in the system measure acceleration and angular velocity. The system also reads wheel speed pulses to estimate the vehicle's precise speed. This information, along with angular velocity and acceleration, can be used to compensate for the inter-frame pose of the image.

[0161] See also Figure 5 , Figure 5 This is a flowchart of a specific method for generating an autonomous driving map, as shown in an exemplary embodiment of the present application. Figure 5 As shown, the execution flow of this method is as follows.

[0162] First, acquire imagery and GPS data.

[0163] That is, environmental image information (images) may be collected by the vehicle-mounted image collection device, and the environmental image information may include multiple video frame images or multiple images in a time sequence.

[0164] It should be noted that image data (collected environmental image information) requires semantic segmentation. GPS (vehicle location information) requires RTK correction to improve positioning accuracy to 20cm. Image data and GPS data must be timed based on the same clock source to ensure time synchronization. Semantic segmentation can be implemented using methods known to those skilled in the art and is not limited here.

[0165] Second, the image data is inverse depth projected.

[0166] An example inverse depth projection processing method is as follows:

[0167] The position of the 2D pixel point in the vehicle coordinate system is p v =[x v ,y v , z v ] T , the normalized coordinates of the point in the pixel coordinate system are

[0168] p I =[u, v, 1] T .

[0169] First, each pixel can be identified as a vector in space as follows,

[0170] v:v=RK -1 p I Formula (1);

[0171] Get the normalized vector of v as Then the coordinates in the vehicle coordinate system are obtained as follows:

[0172]

[0173] This allows us to project the 2D pixel points of the image plane into the 3D space in the vehicle coordinate system.

[0174] If the environmental image information includes multiple image frames, the 3D points (three-dimensional point cloud) of each image frame can be obtained by the above method, and then a continuous point cloud can be obtained.

[0175] The inverse depth projection is performed by formula (1) and (2), and the effect is shown in Figure 3 This step requires the camera's intrinsic and extrinsic parameter matrices. These parameters are obtained through field calibration and online correction, and can be directly obtained from the camera parameters.

[0176] Third, GPS data is interpolated to obtain the position, preserving linear velocity, angular velocity, and acceleration information.

[0177] Since the sampling beat of GPS and the sampling beat of image are not consistent, the sampling time of GPS and the sampling time of image do not coincide, which requires the use of position interpolation to obtain the actual position of the image. c , when the nearest GPS time is t1 and t2, the positions are p1 and p2 respectively. Then, t c Position at time p c for:

[0178]

[0179] Assume t1 < t c <t2. After position interpolation, it is considered that t c The linear velocity, angular velocity, and acceleration at time t1 are the same as those at time t1.

[0180] Fourth, determine whether the subgraph needs to be sliced.

[0181] The subgraph segmentation conditions must meet the following rules: 1) When GPS is normal, truncate into a subgraph every 80 meters; 2) When the horizontal rotation exceeds 85°, truncate into a subgraph; 3) When the pitch angle exceeds 5°, truncate into a subgraph; 4) If the GPS is blocked or interfered with, ignore the above three rules until GPS is normal. The above 80 meters, 85°, and 5° are merely illustrative and do not limit the method used in this embodiment.

[0182] When splitting sub-images, ensure that the last frame of the previous sub-image and the first frame of the current sub-image are at the same position. That is, the position will be saved in both the previous sub-image and the current sub-image. Figure 6 , Figure 6 is a schematic diagram of the camera poses included in the two sub-graphs shown in an exemplary embodiment of the present application, Figure 6 It can be seen that two adjacent sub-images will share the same camera pose, that is, the last frame of the previous sub-image and the first frame of the current sub-image are at the same position.

[0183] Fifth, 3D pose graph optimization.

[0184] After cutting the subgraph, pose graph optimization is required, see Figure 7 When GPS is available, it is used as a position constraint. Linear velocity, angular velocity, and acceleration are used between frames to provide dead reckoning pose compensation. This forms a pose graph, and an optimization algorithm is used to obtain the optimal position and pose.

[0185] For example, the main optimization value of 3D pose graph optimization is the camera position at each moment in the subgraph, and two sets of error equations need to be introduced in sequence. When GPS is normal, GPS observations are used as error equations:

[0186]

[0187] Among them, T wv For optimization variables, a 4x4 matrix; is the GPS position; Log(.) is the logarithmic mapping under the Lie group. When GPS is interfered with, this error equation does not exist.

[0188] Use the dead reckoning result between the two frames as the second set of error equations:

[0189]

[0190] in, is the optimization variable at two adjacent moments, Dead reckoning is calculated using linear velocity, angular velocity, and acceleration.

[0191] here The calculation requires calculating the difference in the vehicle's posture from t1 to t2 when semantic data is obtained. In addition to recording the posture at t1 and t2, all linear and angular velocity information between these two times is also required. We use a point mass model and assume that the center of the vehicle is at the center of the rear axle. The resulting differential equation for dead reckoning is:

[0192]

[0193] Where p represents position, R represents attitude, and v represents velocity. Here, the vehicle linear velocity is taken as v x , then v=[v x , 0, 0] T w is the angular velocity, [.] × represents an antisymmetric matrix.

[0194] In summary, the two sets of error equations constitute the total error equation:

[0195] E=∑||e1||2+∑||e2||2 Formula (12);

[0196] According to formulas (9) to (11), the optimization equation can be constructed Solve T by letting the error E gradually approach 0 wv , to calculate the trajectory T of all cameras in the subgraph wv .

[0197] Sixth, projection determines the projection plane.

[0198] After obtaining the exact position within the sub-image, we need to determine the projection plane. The so-called projection of a 3D point cloud onto a 2D plane does not necessarily mean projection onto the plane at z = 0. The projection plane must pass through the positions of the first and last frames within the sub-image. In other words, the projection plane can be determined by the positions of the first and last frames. This allows us to determine the transformation relationship between this plane and the world coordinate system.

[0199] For example, assuming that there are n trajectories, Use the position of the first frame as the reference position for initialization The projection plane needs to pass through the first frame and the last frame, so for T wm Compensate a pitch angle. First calculate the attitude difference:

[0200]

[0201] Then the pose difference between the first frame and the last frame is The pitch angle can be solved by decomposing the yaw-pitch-roll.

[0202]

[0203] Among them, g(.) expresses the mapping of the rotation matrix to yaw (yaw angle), pitch (pitch angle), and roll (roll angle). Then set yaw and roll to zero and calculate the compensated rotation matrix R pitch (Compensation matrix). At this time, the reference pose is:

[0204]

[0205] In one embodiment, all 3D point clouds p w Projected to the map coordinate system:

[0206]

[0207] p m Setting the z-axis coordinate to zero allows projection onto the 2D plane while ensuring that the projection plane passes through the first and last frames.

[0208] Seventh, delete all image data and only keep the last frame.

[0209] All 3D semantic point clouds in the vehicle coordinate system are projected into the world coordinate system and then projected into a 2D point cloud based on the projection plane. When projecting into a 2D point cloud, an occupancy grid is used, with each grid representing 0.1m x 0.1m. Compression is also performed by converting the 2D point cloud into an occupancy grid.

[0210] Finally, the connection relationship between the previous subgraph and the current subgraph is recorded.

[0211] For example, if the current subgraph contains data of n frames, then delete frames 1 to n-1 and keep the nth frame as the first frame of the next subgraph.

[0212] A semantic 2.5D map (autonomous driving map) is constructed using the semantic segmentation results of the on-board camera (semantic segmentation results of environmental image information) and the on-board combined inertial navigation device (vehicle position information). This map will be used to assist the vehicle in high-precision positioning.

[0213] The following describes an embodiment of the apparatus of the present application, which can be used to execute the autonomous driving map generation method described in the above embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the embodiment of the image processing method described above in the present application.

[0214] Figure 8 This is a block diagram of an autonomous driving map generation device shown in an exemplary embodiment of the present application. The device can be applied to Figure 1 The implementation environment shown in FIG1 is specifically configured in the computer device 103. The apparatus may also be applicable to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the apparatus is applicable.

[0215] like Figure 8 As shown, the exemplary autonomous driving map generating apparatus 800 includes:

[0216] Image acquisition module 801, used to obtain environmental image information of the vehicle;

[0217] Position information module 802, used to obtain vehicle position information;

[0218] The image processing module 803 is used to perform inverse depth projection processing on the environmental image information to obtain a processed image, and perform image segmentation on the processed image to obtain a plurality of sub-images;

[0219] The trajectory module 804 is used to obtain trajectory data of the image acquisition device from the start time to the end time in a sub-image according to the position of the image acquisition device at different times in the sub-image;

[0220] The projection plane module 805 is used to determine the projection plane according to the posture difference between the first frame image and the last frame image in the trajectory data, and make the projection plane pass through the first frame image and the last frame image;

[0221] The output module 806 is configured to use the last frame image as the first frame image of the next sub-image, so that all sub-images are connected through the projection plane to form a driving map.

[0222] It should be noted that the apparatus provided in the above embodiments and the methods provided in the above embodiments are based on the same concept. The specific manner in which the various modules and units perform their operations has been described in detail in the method embodiments and will not be repeated here. In actual applications, the autonomous driving map generation device provided in the above embodiments can, as needed, allocate the above functions to different functional modules. That is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0223] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements the image processing method provided in the above-mentioned embodiments.

[0224] Figure 9 The following is a schematic diagram showing the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application. Figure 9 The computer system 900 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0225] like Figure 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 902 or the program loaded from the storage part 908 into the random access memory (RAM) 903, such as executing the method described in the above embodiment. Various programs and data required for system operation are also stored in the RAM 903. The CPU 901, ROM 902 and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0226] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, and the like; an output section 907 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 908 including a hard disk and the like; and a communication section 909 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. Removable media 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like, are installed in the drive 910 as needed, so that computer programs read therefrom can be installed into the storage section 908 as needed.

[0227] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from a removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the various functions defined in the system of the present application are executed.

[0228] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable computer program. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0229] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0230] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. In some cases, the names of these units do not constitute limitations on the units themselves.

[0231] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method. The computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist independently without being incorporated into the electronic device.

[0232] Another aspect of the present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the above embodiments.

[0233] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, any equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A method for generating an autonomous driving map, characterized in that: include: Acquire environmental image information and vehicle position information of the vehicle, wherein the environmental image information is acquired based on an onboard image acquisition device; Performing inverse depth projection processing on the environmental image information to obtain a processed image, and performing image segmentation on the processed image to obtain a plurality of sub-image images; Obtaining trajectory data of the image acquisition device from a start time to an end time in a sub-image according to the position of the image acquisition device at different times in the sub-image; Determine a projection plane based on a posture difference between a first frame image and a last frame image in the trajectory data, and make the projection plane pass through the first frame image and the last frame image; wherein, the position of the first frame image in the trajectory data is used as an initialization reference position, and the posture difference between the first frame image and the last frame image is calculated; perform pitch angle compensation based on the posture difference to obtain a reference posture, and perform projection based on the reference posture so that the projection plane passes through the first frame image and the last frame image; The last frame image is used as the first frame image of the next sub-image, so that all sub-images are connected through the projection plane to form a driving map.

2. The method for generating an autonomous driving map according to claim 1, wherein: Obtaining trajectory data of the image acquisition device from a start time to an end time in a sub-image according to positions of the image acquisition device at different times in the sub-image, including: If the vehicle position information is obtained normally, position constraints are applied to the vehicle position information to obtain the positions of the image acquisition device at different moments in the sub-image; If an abnormality occurs in obtaining the vehicle position information, the vehicle's posture difference when obtaining the environmental image information from the start time to the end time is calculated based on the position information and state parameters, and the position of the image acquisition device at different times in the sub-image is obtained based on the posture difference. The state parameters include the vehicle's linear velocity, angular velocity and acceleration.

3. The method for generating an autonomous driving map according to claim 2, wherein: Obtaining positions of image acquisition devices at different moments within the sub-image based on the posture differences includes: Recording the vehicle's posture at the start time and the vehicle's posture at the end time to obtain the posture difference; The state parameters between adjacent frame images in the sub-image are used as posture compensation to perform dead reckoning to obtain the dead reckoning result; An optimization model is constructed based on the pose difference and dead reckoning results to obtain the position of the image acquisition device at different moments in the sub-image.

4. The method for generating an autonomous driving map according to claim 3, wherein: According to the pose difference and dead reckoning results, constructing an optimization model includes: determining a total error based on the pose difference and the dead reckoning result; The optimization model is determined based on the goal that the total error gradually approaches zero.

5. The method for generating an autonomous driving map according to any one of claims 1 to 3, wherein: Before performing image segmentation on the processed image to obtain a plurality of sub-images, the method further includes: If the vehicle position information is normally acquired, the processed image is segmented according to a preset distance threshold, the processed image is segmented according to a preset horizontal rotation angle threshold, and the processed image is segmented according to a preset elevation angle threshold; If an abnormality occurs in obtaining the vehicle position information, image segmentation will not be performed until the vehicle position information is obtained normally.

6. The method for generating an autonomous driving map according to any one of claims 1 to 3, wherein: Obtaining trajectory data of the image acquisition device from a start time to an end time in a sub-image according to positions of the image acquisition device at different times in the sub-image, including: The vehicle position information is interpolated using the following formula: Among them, t c is the acquisition time of the environmental image information, t1 and t2 are the acquisition times of the adjacent position information, p1 and p2 are the acquired adjacent position information, and p c t c The location information collected at all times, and t1<t c <t2; Based on the position information after position interpolation processing, the trajectory data of the image acquisition device in the sub-image from the starting time to the ending time is obtained.

7. The method for generating an autonomous driving map according to claim 1, wherein: Projecting based on the reference pose includes: Obtaining a transformation relationship between the projection plane and the world coordinate system; A three-dimensional point cloud is generated in a vehicle coordinate system based on the environmental image information and vehicle position information of the vehicle, and the three-dimensional point cloud is projected to a world coordinate system. A secondary projection is performed based on the transformation relationship, and the three-dimensional point cloud projected to the world coordinate system is converted into a two-dimensional point cloud by compressing the occupied grid map.

8. The method for generating an autonomous driving map according to claim 1, wherein: Performing pitch angle compensation according to the attitude difference to obtain a reference attitude includes: Decomposing the attitude difference to obtain pitch angle, yaw angle and roll angle; Mapping the pitch angle, the yaw angle, and the roll angle to each other through a preset rotation matrix; Setting the yaw angle and the roll angle to zero to determine a compensation matrix; The attitude difference is compensated for a pitch angle according to the compensation matrix to obtain a reference attitude.

9. An autonomous driving map generation device, characterized in that: include: An image acquisition module is used to obtain environmental image information of the vehicle; Position information module, used to obtain vehicle position information; An image processing module is used to perform inverse depth projection processing on the environmental image information to obtain a processed image, and perform image segmentation on the processed image to obtain a plurality of sub-images; A trajectory module, configured to obtain trajectory data of the image acquisition device from a start time to an end time within a sub-image according to the position of the image acquisition device at different times within the sub-image; a projection plane module, configured to determine a projection plane based on a posture difference between a first frame image and a last frame image in the trajectory data, and to pass the projection plane through the first frame image and the last frame image; wherein the position of the first frame image in the trajectory data is used as an initialization reference position, and the posture difference between the first frame image and the last frame image is calculated; pitch angle compensation is performed based on the posture difference to obtain a reference posture, and projection is performed based on the reference posture so that the projection plane passes through the first frame image and the last frame image; The output module is configured to use the last frame image as the first frame image of the next sub-image, so that all sub-images are connected through the projection plane to form a driving map.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that Computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the method according to any one of claims 1 to 8.

Images