A method and apparatus for memorizing driving

Abstract

The application belongs to the field of memory driving, and discloses a memory driving method and device. The method comprises the following steps: obtaining a first vehicle position and a first road condition image during driving; performing semantic segmentation on the first road condition image to obtain a first road condition element contained in the first road condition image and a first relative position between the first road condition element and the autonomous vehicle; if a reference road condition element and a reference relative position corresponding to the first vehicle position in a first semantic map downloaded from a cloud server are the same as the first road condition element and the first relative position respectively, then controlling the autonomous vehicle to perform memory driving according to a first vehicle pose corresponding to the first vehicle position in the first semantic map. The application is used for memory driving in complex urban areas.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of memory driving, in particular to a memory driving method and device. BACKGROUND

[0002] The navigation assistance function of a vehicle is that after the vehicle enters a defined road section, the vehicle can follow the navigation to autonomously select a correct route, and in the middle of the way, complete lane keeping, adaptive cruise, avoidance, overtaking and lane changing and other actions according to factors such as road conditions and traffic environment.

[0003] However, the current navigation assistance function of the vehicle is mainly realized through real-time dynamic measurement technology and high-precision maps, and the existing navigation assistance function is set in sections such as highways, elevated roads and urban expressways covered by high-precision maps. When memory driving in a complex urban area, the coverage rate of the urban scene is low, and the freshness is not enough. SUMMARY

[0004] Therefore, it is necessary to provide a memory driving method and device in view of the above technical problems.

[0005] In a first aspect, a memory driving method is provided, which is applied to an autonomous vehicle, and the method comprises:

[0006] In the driving process, a first vehicle position and a first road condition image are acquired;

[0007] The first road condition image is subjected to semantic segmentation to obtain a first road condition element contained in the first road condition image and a first relative position between the first road condition element and the autonomous vehicle;

[0008] If a reference road condition element and a reference relative position corresponding to the first vehicle position in a first semantic map downloaded from a cloud server are the same as the first road condition element and the first relative position respectively, the autonomous vehicle is controlled to perform memory driving according to a first vehicle pose corresponding to the first vehicle position in the first semantic map.

[0009] As an optional implementation, the method further comprises:

[0010] In the memory driving process, a second vehicle pose of the autonomous vehicle at the first vehicle position is acquired;

[0011] If the second vehicle pose is different from the first vehicle pose corresponding to the first vehicle position in the first semantic map, the second vehicle pose is real-time corrected to the first vehicle pose.

[0012] As an optional implementation, the acquiring the second vehicle pose of the autonomous vehicle at the first vehicle position comprises:

[0013] acquiring a first initial attitude angle;

[0014] performing error processing on the first initial attitude angle by using vehicle dynamics fusion processing technology to obtain a first target attitude angle;

[0015] determining the second vehicle pose of the autonomous vehicle by performing feature matching between the first road elements and the first semantic map based on the first vehicle position and the first target attitude angle, and using an iterative closest point (ICP) algorithm.

[0016] As an optional implementation, the method further comprises:

[0017] acquiring a second vehicle position, a second road image, and a second initial attitude angle during driving;

[0018] performing error processing on the second initial attitude angle by using vehicle dynamics fusion processing technology to obtain a second target attitude angle;

[0019] determining a third vehicle pose of the autonomous vehicle according to the second vehicle position and the second target attitude angle;

[0020] performing semantic segmentation on the second road image to obtain second road elements contained in the second road image and a second relative position between the second road elements and the autonomous vehicle;

[0021] performing dynamic inverse projection on the second vehicle position, the second road elements, and the second relative position to obtain a second semantic map;

[0022] performing projection image adjustment on the second semantic map according to the third vehicle pose to obtain a third semantic map, and sending the third semantic map to a cloud server.

[0023] In a first aspect, another method for memorizing driving is provided, and the method is applied to a cloud server, and the method comprises:

[0024] receiving first carrier phase observation data of a monitoring satellite sent by a reference station, vehicle positioning data sent by an autonomous vehicle, second carrier phase observation data of the monitoring satellite, and a first semantic map;

[0025] correcting a vehicle position in the first semantic map based on the first carrier phase observation data, the vehicle positioning data, the second carrier phase observation data, and a dynamic post-processing (PPK) algorithm to obtain a corrected first semantic map.

[0026] perform data fusion processing on the cloud-stored first semantic map and the corrected first semantic map to obtain a fused first semantic map, and send the fused first semantic map to the autonomous vehicle.

[0027] In a second aspect, a device for memorizing driving is provided, which is applied to an autonomous vehicle, and includes:

[0028] a first obtaining module, configured to obtain a first vehicle position and a first road condition image during driving;

[0029] a first obtaining module, configured to obtain a first vehicle position and a first road condition image during driving;

[0030] a control module, configured to, if a reference road condition element and a reference relative position corresponding to the first vehicle position in a first semantic map downloaded from a cloud server are the same as the first road condition element and the first relative position respectively, control the autonomous vehicle to perform memorized driving according to a first vehicle pose corresponding to the first vehicle position in the first semantic map.

[0031] As an optional implementation, the device further includes:

[0032] a second obtaining module, configured to obtain a second vehicle pose of the autonomous vehicle at the first vehicle position during memorized driving;

[0033] a rectification module, configured to, if the second vehicle pose is different from a first vehicle pose corresponding to the first vehicle position in the first semantic map, rectify the second vehicle pose in real time to the first vehicle pose.

[0034] As an optional implementation, the second obtaining module is specifically configured to:

[0035] obtain a first initial attitude angle;

[0036] perform error processing on the first initial attitude angle by using vehicle dynamics fusion processing technology to obtain a first target attitude angle;

[0037] determine a second vehicle pose of the autonomous vehicle based on the first vehicle position and the first target attitude angle by performing feature matching on the first road condition element and the first semantic map through an iterative closest point (ICP) algorithm.

[0038] As an optional implementation, the device further includes:

[0039] a third acquisition module, configured to acquire a second vehicle position, a second road condition image and a second initial attitude angle during driving;

[0040] a fourth acquisition module, configured to perform error processing on the second initial attitude angle by using a vehicle dynamics fusion processing technology to obtain a second target attitude angle;

[0041] a determination module, configured to determine a third vehicle pose of the autonomous vehicle according to the second vehicle position and the second target attitude angle;

[0042] a second obtaining module, configured to perform semantic segmentation on the second road condition image to obtain a second road condition element contained in the second road condition image and a second relative position between the second road condition element and the autonomous vehicle;

[0043] a first obtaining module, configured to perform dynamic inverse projection on the second vehicle position, the second road condition element and the second relative position to obtain a second semantic map;

[0044] a second obtaining module, configured to perform projection image adjustment on the second semantic map according to the third vehicle pose to obtain a third semantic map, and send the third semantic map to a cloud server.

[0045] In a second aspect, another device for memorizing driving is provided, and the device is applied to a cloud server, and the device comprises:

[0046] a receiving module, configured to receive first carrier phase observation data of a monitoring satellite sent by a reference station, vehicle positioning data sent by an autonomous vehicle, second carrier phase observation data of the monitoring satellite and a first semantic map;

[0047] a correction module, configured to correct a vehicle position in the first semantic map based on the first carrier phase observation data, the vehicle positioning data, the second carrier phase observation data and a dynamic post-processing PPK algorithm to obtain a corrected first semantic map;

[0048] a fusion module, configured to perform data fusion processing on the first semantic map stored in the cloud and the corrected first semantic map to obtain a fused first semantic map, and send the fused first semantic map to the autonomous vehicle.

[0049] In a third aspect, a system for memorizing driving is provided, and the system for memorizing driving comprises the method for memorizing driving according to the first aspect and the device for memorizing driving according to the second aspect.

[0050] In a fourth aspect, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and the processor implements the method steps of the first aspect when executing the computer program.

[0051] In a fifth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program, when executed by a processor, implements the method steps of the first aspect.

[0052] The application provides a method and device for memorizing driving, and the embodiments of the application bring at least the following beneficial effects: in the driving process, a semantic map is constructed by acquiring a vehicle position, road condition elements contained in a road condition image, a relative position between the road condition elements and an autonomous driving vehicle, and a vehicle pose. In the process of memorizing driving, the vehicle is controlled to perform memorizing driving according to the semantic map. Through the above method, the semantic map can be constructed in a complex urban area, the vehicle is controlled to perform memorizing driving, the urban scene coverage is improved, and point-to-point memorized route driving is achieved.

[0053] The application can be used to perform memorized driving in a complex urban area, the urban scene coverage area is large, and point-to-point memorized route driving can be achieved.

[0054] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only show some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0056] Figure 1 A structural schematic diagram of a system for memorizing driving provided by an embodiment of the application;

[0057] Figure 2 A flowchart of a method for memorizing driving provided by an embodiment of the application;

[0058] Figure 3 A flowchart of another method for memorizing driving provided by an embodiment of the application;

[0059] Figure 4 A structural schematic diagram of a device for memorizing driving provided by an embodiment of the application;

[0060] Figure 5Another structural schematic diagram of a device for memorizing driving provided by an embodiment of the present application is provided.

[0061] Figure 6 A structural schematic diagram of a computer device provided by an embodiment of the present application is provided. DETAILED DESCRIPTION

[0062] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0063] The method for memorizing driving provided by an embodiment of the present application can be applied to a system for memorizing driving. As shown in the figure, Figure 1 The system for memorizing driving includes an autonomous vehicle 101 and a cloud server 102.

[0064] The autonomous vehicle 101 is configured to acquire a first vehicle position and a first road condition image during driving. The autonomous vehicle 101 performs semantic segmentation on the first road condition image to obtain a first road condition element contained in the first road condition image and a first relative position between the first road condition element and the autonomous vehicle 101. If a reference road condition element corresponding to the first vehicle position and a reference relative position in a first semantic map downloaded from the cloud server are the same as the first road condition element and the first relative position respectively, the autonomous vehicle 101 controls the autonomous vehicle 101 to perform memorizing driving according to a first vehicle pose corresponding to the first vehicle position in the first semantic map.

[0065] The cloud server 102 receives first carrier phase observation data of a monitoring satellite sent by a reference station, vehicle positioning data sent by the autonomous vehicle 101, second carrier phase observation data of the monitoring satellite and a first semantic map. Based on the first carrier phase observation data, the vehicle positioning data, the second carrier phase observation data and a dynamic post-processing PPK (Post-Processing Kinematic, GPS dynamic post-processing) algorithm, the cloud server 102 corrects the vehicle position in the first semantic map to obtain a corrected first semantic map. The cloud server 102 performs data fusion processing on the first semantic map stored in the cloud and the corrected first semantic map to obtain a fused first semantic map, and sends the fused first semantic map to the autonomous vehicle 101.

[0066] Next, a method for memorizing driving provided by an embodiment of the present application will be described in detail, Figure 2 A flowchart of a method for memorizing driving provided by an embodiment of the present application is shown in the figure, Figure 2 The specific steps are as follows:

[0067] Step 201, acquiring the first vehicle position and the first road condition image during driving.

[0068] In implementation, when the autonomous vehicle is driving, the current vehicle position needs to be determined, and the current vehicle position is determined in the position of the first semantic map, and then the memory driving is completed according to the first semantic map. The autonomous vehicle can acquire the first vehicle position through GNSS (Global Navigation Satellite System). Further, the first road condition image during driving can also be collected through the information collection device in the autonomous vehicle. The information collection device can be a camera, a camera, and a laser radar, etc. The first vehicle position of the autonomous vehicle is determined through the first road condition image. Therefore, the autonomous vehicle acquires the first vehicle position and the first road condition image during driving.

[0069] Step 202, performing semantic segmentation on the first road condition image to obtain the first road condition element contained in the first road condition image and the first relative position between the first road condition element and the autonomous vehicle.

[0070] In implementation, during driving of the autonomous vehicle, the first road condition image collected is subjected to semantic segmentation to obtain the first road condition element contained in the first road condition image and the first relative position between the first road condition element and the autonomous vehicle. The semantic segmentation can be performed through a constructed neural network model. The construction of the neural network model is a mature prior art, which is not described here. Through the first road condition element and the first relative position between the first road condition element and the autonomous vehicle, the first vehicle position of the autonomous vehicle is determined in the subsequent step.

[0071] Step 203, if the reference road condition element and the reference relative position corresponding to the first vehicle position in the first semantic map downloaded from the cloud server are the same as the first road condition element and the first relative position respectively, the autonomous vehicle is controlled to perform memory driving according to the first vehicle pose corresponding to the first vehicle position in the first semantic map.

[0072] In implementation, the first semantic map downloaded by the autonomous vehicle from the cloud server, and the reference road element and the reference relative position corresponding to the first vehicle position in the first semantic map are obtained. The autonomous vehicle compares the first road element contained in the first road image obtained in the above steps and the first relative position between the first road element and the autonomous vehicle with the reference road element and the reference relative position corresponding to the first vehicle position in the first semantic map. If the reference road element and the reference relative position corresponding to the first vehicle position in the first semantic map are the same as the first road element and the first relative position respectively, it indicates that the current autonomous vehicle reaches the first vehicle position of the first semantic map. Wherein, the autonomous vehicle determines the position of the autonomous vehicle in the first semantic map by the positioning method of the image. At this time, the autonomous vehicle can be controlled to perform memory driving according to the first vehicle pose corresponding to the first vehicle position in the first semantic map.

[0073] Further, in the memory driving process, in order to avoid the autonomous vehicle deviating from the route of the first semantic map, it is also necessary to detect the pose of the autonomous vehicle in real time. The specific operation is as follows.

[0074] Step one, in the memory driving process, the second vehicle pose of the autonomous vehicle at the first vehicle position is obtained.

[0075] In implementation, in the memory driving process, in order to avoid the autonomous vehicle deviating from the route of the first semantic map, it is also necessary to detect the pose of the autonomous vehicle in real time, so as to ensure that the pose of the autonomous vehicle obtained is according to the vehicle pose of the vehicle position in the first semantic map. Therefore, in the memory driving process, the second vehicle pose of the autonomous vehicle at the first vehicle position is needed.

[0076] Specifically, the specific steps of executing the second vehicle pose of the autonomous vehicle at the first vehicle position in the memory driving process are as follows.

[0077] Step A, obtaining a first initial attitude angle.

[0078] In implementation, in the memory driving process, in order to avoid the autonomous vehicle deviating from the route of the semantic map, it is also necessary to detect the pose of the autonomous vehicle in real time. At the same time, the autonomous vehicle can determine the vehicle pose according to the vehicle position and the attitude angle. Wherein, the autonomous vehicle obtains the first initial attitude angle through the multi-axis IMU (Inertial Measurement Unit). Therefore, in the memory driving process, the autonomous vehicle obtains the first initial attitude angle, wherein the first initial attitude angle can be obtained by integrating the acceleration and angular velocity output by the multi-axis IMU.

[0079] Step B, using vehicle dynamics fusion processing technology to process the error of the first initial attitude angle, obtaining the first target attitude angle.

[0080] In implementation, during the process of memory driving, the attitude angle estimated based on multi-axis IMU will have some cumulative errors as the vehicle continues to drive. In order to reduce the error, vehicle dynamics fusion processing technology can be used to estimate the vehicle attitude angle according to the steering wheel angle and wheel speed information in the vehicle chassis signal combined with the vehicle dynamics model. Therefore, the autonomous vehicle can use vehicle dynamics fusion processing technology to process the error of the first initial attitude angle, obtaining the first target attitude angle.

[0081] Step C, using the Iterative Closest Point (ICP) algorithm, by matching the first road element with the first semantic map, and based on the first vehicle position and the first target attitude angle, determining the second vehicle pose of the autonomous vehicle.

[0082] In implementation, the autonomous vehicle obtains the first vehicle position according to the GNSS technology. The autonomous vehicle determines the second vehicle pose of the autonomous vehicle according to the first target attitude angle, and the ICP algorithm continuously updates the most accurate pose by matching the feature points of the first road element with the feature points of the first semantic map to minimize the pose estimation error.

[0083] Step two, if the second vehicle pose is different from the first vehicle pose corresponding to the first vehicle position in the first semantic map, the second vehicle pose is real-time corrected to the first vehicle pose.

[0084] In implementation, if you want to control the autonomous vehicle to drive according to the first semantic map, you need to control the autonomous vehicle to drive according to the first vehicle pose corresponding to the first vehicle position in the first semantic map. Therefore, the second vehicle pose is compared with the first vehicle pose corresponding to the first vehicle position in the first semantic map. If the second vehicle pose is different from the first vehicle pose corresponding to the first vehicle position in the first semantic map, the autonomous vehicle can send the drive-by-wire instruction through the CAN (Controller Area Network) bus to control the power module, brake module and steering module to real-time correct the second vehicle pose to the first vehicle pose. Thus, it is ensured that the autonomous vehicle drives along the route memorized according to the first semantic map. Further, visual feature matching can also be combined to confirm the second vehicle pose of the autonomous vehicle.

[0085] Further, before memory driving, a semantic map needs to be constructed, and the steps of constructing the semantic map are as follows.

[0086] Step one, during driving, obtaining the second vehicle position, the second road image and the second initial attitude angle.

[0087] In the implementation, the existing automatic driving vehicle is usually driven on the highway, elevated road and urban expressway covered by high-precision map in the prior art, but the coverage rate of the urban scene is low when the vehicle is driven in the complex urban area. Meanwhile, the high-precision map mainly depends on the expensive map collection vehicle of the map vendor, and the production cycle of the map is long. Therefore, the application proposes a method for constructing a semantic map in the complex urban area, and the production of the semantic map is short and efficient. When constructing the semantic map, the road position and road conditions and other elements need to be obtained. Therefore, the second vehicle position is obtained by the GNSS during the driving process of the automatic driving vehicle. The second road condition image is collected by the information collection device of the automatic driving vehicle during the driving process. The information collection device can be a camera, a camera and a laser radar, etc. Generally, the elements in the map correspond to the pose of the vehicle when the automatic driving vehicle drives according to the map. Therefore, the vehicle pose needs to be obtained when constructing the map. Since the vehicle pose needs to be determined, the attitude angle of the vehicle is needed. Therefore, the second initial attitude angle is obtained by the multi-axis IMU during the driving process.

[0088] Step two, the second initial attitude angle is error processed by the vehicle dynamics fusion processing technology to obtain the second target attitude angle.

[0089] In the implementation, the attitude angle estimated based on the multi-axis IMU during the driving process will have some cumulative errors with the continuous driving of the vehicle. In order to avoid errors and make the constructed semantic map more accurate, the errors need to be eliminated. Therefore, the second initial attitude angle can be error processed by the vehicle dynamics fusion processing technology to obtain the second target attitude angle.

[0090] Step three, the third vehicle pose of the automatic driving vehicle is determined according to the second vehicle position and the second target attitude angle.

[0091] In the implementation, the pose of the vehicle is the position and attitude of the vehicle, that is, the three-dimensional coordinates and the direction of the vehicle during the driving process, wherein the three-dimensional coordinates are the latitude, longitude and altitude, and the direction is the angle of the vehicle body relative to the driving route. Since the second road condition image used by the constructed semantic map is collected by the automatic driving vehicle, it is necessary to ensure that the second road condition image used by the constructed semantic map is shot in the forward direction of the vehicle body of the automatic driving vehicle. Therefore, the vehicle pose of the automatic driving vehicle needs to be obtained in real time during the driving process. Therefore, the third vehicle pose of the automatic driving vehicle can be determined according to the second vehicle position and the second target attitude angle obtained in the above steps.

[0092] Step four, performing semantic segmentation on the second road condition image to obtain second road condition elements contained in the second road condition image and a second relative position between the second road condition elements and the autonomous vehicle.

[0093] In implementation, the most basic condition of the constructed semantic map requires obtaining road condition elements from the collected road condition image, and obtaining the relative distance and relative direction between the road condition elements and the autonomous vehicle. Therefore, when constructing the semantic map, the autonomous vehicle needs to obtain the second road condition elements and the second relative position between the second road condition elements and the autonomous vehicle from the second road condition image. The second relative position includes the relative distance and the relative direction. The second road condition elements include lane lines, zebra crossings, stop lines, ground markings, road signs, and power poles. The autonomous vehicle can use semantic segmentation to perform image segmentation on the second road condition image to obtain the second road condition elements contained in the second road condition image and the second relative position between the second road condition elements and the autonomous vehicle. The semantic segmentation can be performed by constructing a neural network model. In actual application, other methods can also be used according to actual application conditions, which are not limited herein.

[0094] Step five, performing dynamic inverse projection on the second vehicle position, the second road condition elements, and the second relative position to obtain a second semantic map.

[0095] In implementation, the second vehicle position, the second road condition elements, and the second relative position obtained in the previous step are subjected to dynamic inverse projection. That is, the second road condition elements in the second road condition image are subjected to coordinate conversion according to the second vehicle position of the autonomous vehicle and the relative position between the second road condition elements and the second vehicle position. In other words, based on the second vehicle position and the second relative position, the second road condition elements in the multiple frames of second road condition images collected by the autonomous vehicle are fused and restored to the 3D space. In this way, the second semantic map can be obtained according to the second vehicle position, the second road condition elements, and the second relative position.

[0096] Step six, performing projection image adjustment on the second semantic map according to the third vehicle pose to obtain a third semantic map, and sending the third semantic map to the cloud server.

[0097] In the implementation, since the second road condition image used by the established semantic map is taken by the automatic driving vehicle in the forward direction, the pose of the automatic driving vehicle needs to be acquired in real time during driving. If the vehicle body deviates from the relative driving route when the second road condition image is collected, the second road condition image collected when deviating needs to be adjusted. The projection image adjustment is to adjust the coordinates of the second road condition elements in the second semantic map to the corresponding coordinates when not deviating. Since the intrinsic and extrinsic parameters of the vehicle-mounted camera for collecting the second road condition image are known parameters, the pixel positions of the feature points in the image of the second road condition elements in the collected second road condition image are converted to the vehicle coordinate system, and then converted from the vehicle coordinate system to the latitude, longitude and altitude in the world coordinate system, so that the position of the second road condition elements in the world coordinate system can be known, thereby completing the projection image adjustment of the second semantic map and obtaining a third semantic map. The automatic driving vehicle sends the third semantic map to the cloud server through the shadow mode data link.

[0098] Figure 3 Another flowchart of the method for memorizing driving provided by the embodiment of the application is shown in FIG. 6, and the specific steps are as follows: Figure 3

[0099] In step 301, the first carrier phase observation data of the monitoring satellite sent by the reference station, the vehicle positioning data sent by the automatic driving vehicle, the second carrier phase observation data of the monitoring satellite and the first semantic map are received.

[0100] In the implementation, after the automatic driving vehicle constructs the first semantic map and sends the first semantic map to the cloud server, the cloud server corrects the vehicle position of the automatic driving vehicle in the first semantic map. Since the reference station continuously and continuously monitors the first carrier phase observation of the monitoring satellite at the same time point during the driving of the automatic driving vehicle, the automatic driving vehicle also monitors the second carrier phase observation of the monitoring satellite and the real-time vehicle positioning data of the vehicle itself during movement. The cloud server receives the first carrier phase observation data of the monitoring satellite sent by the reference station, the vehicle positioning data sent by the automatic driving vehicle, the second carrier phase observation data of the monitoring satellite and the first semantic map. The cloud server can correct the vehicle position of the automatic driving vehicle in the first semantic map by using the first carrier phase observation data, the second carrier phase observation data and the vehicle positioning data. Therefore, the cloud server receives the first carrier phase observation data of the monitoring satellite sent by the reference station, the vehicle positioning data sent by the automatic driving vehicle, the second carrier phase observation data of the monitoring satellite and the first semantic map.

[0101] ​At step 302, the vehicle position in the first semantic map is corrected based on the first carrier phase observation data, the vehicle positioning data, the second carrier phase observation data and the dynamic post-processing PPK algorithm, to obtain a corrected first semantic map.

[0102] In implementation, after the first semantic map is constructed, the automatic driving vehicle sends the first semantic map to the cloud server, and the cloud server corrects the vehicle position of the automatic driving vehicle in the first semantic map. However, in the prior art, the correction of the vehicle position in the map is usually real-time processing. However, real-time positioning correction is affected by the communication distance of the reference station, and the transmission of intermediate signals is relatively cumbersome. In order to ensure the accuracy of the map, the cost is relatively high. In order to ensure the accuracy and lightness of the first semantic map while further reducing the cost, the PPK algorithm is adopted in the present application, the first carrier phase observation data of the monitoring satellite sent by the reference station is received, the second carrier phase observation of the satellite is monitored in real time during the movement of the automatic driving vehicle, and linear combination is performed to form a virtual third carrier phase observation. The cloud server can determine the linear relationship of the relative position between the reference station and the automatic driving vehicle according to the virtual third carrier phase observation. Since the coordinates of the latitude, longitude and altitude of the reference station are known and unchanged, the predicted positioning data of the automatic driving vehicle can be determined in real time. The cloud server corrects the positioning data of the automatic driving vehicle through the predicted positioning data and the vehicle positioning data of the automatic driving vehicle, thereby correcting the vehicle position in the first semantic map, obtaining a corrected first semantic map, and achieving a high-precision effect of the first semantic map.

[0103] At step 303, the first semantic map stored in the cloud and the corrected first semantic map are subjected to data fusion processing to obtain a fused first semantic map, and the fused first semantic map is sent to the automatic driving vehicle.

[0104] In implementation, after the cloud server corrects the first semantic map, the cloud server performs data fusion processing on the first semantic map sent by the same automatic driving vehicle received and saved in the cloud before, updates the first semantic map, and sends the fused first semantic map to the automatic driving vehicle. Further, the cloud server also performs data fusion processing on the first semantic map sent by each automatic driving vehicle. For example, the first automatic driving vehicle sends a route semantic map of position A-position B to the cloud server, and the second automatic driving vehicle sends a route semantic map of position B-position C to the cloud server. The cloud server can perform data fusion processing on the route semantic map of position A-position B and the route semantic map of position B-position C to obtain a route semantic map of position A-position C, so that the city area covered by the first semantic map gradually expands and the freshness is higher.

[0105] The embodiment of the present application provides a method for memorizing driving, in the driving process, a semantic map is constructed by acquiring a vehicle position, a road condition element contained in a road condition image, a relative position between the road condition element and an autonomous vehicle and a vehicle pose. In the process of memorizing driving, the vehicle is controlled to perform memorizing driving according to the semantic map. Through the above method, the semantic map can be constructed in a complex urban area, the vehicle is controlled to perform memorizing driving, the scene coverage rate in the city is improved, and point-to-point memorizing driving is realized.

[0106] It should be understood that although Figures 2 to 3 The steps in the flowchart of the method are displayed in sequence according to the arrows, but these steps are not necessarily executed in sequence according to the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, Figures 2 to 3 At least part of the steps in the method can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0107] It can be understood that the same / similar parts of each embodiment of the above method in the specification can be mutually referred to, and each embodiment focuses on the differences from other embodiments, and the related parts can be referred to the description of other method embodiments.

[0108] The embodiment of the present application also provides a device for memorizing driving, as shown in the device includes: Figure 4

[0109] The first acquisition module 401 is configured to acquire a first vehicle position and a first road condition image in the driving process.

[0110] The first obtaining module 402 is configured to perform semantic segmentation on the first road condition image to obtain a first road condition element contained in the first road condition image and a first relative position between the first road condition element and the autonomous vehicle.

[0111] The control module 403 is configured to control the autonomous vehicle to perform memorizing driving according to a first vehicle pose corresponding to the first vehicle position in the first semantic map if a reference road condition element and a reference relative position corresponding to the first vehicle position in the first semantic map downloaded from the cloud server are the same as the first road condition element and the first relative position respectively.

[0112] As an optional implementation, the device further includes: ​

[0113] a second obtaining module, configured to obtain a second vehicle pose of the autonomous vehicle at the first vehicle position during the memory driving;

[0114] a rectification module, configured to rectify the second vehicle pose to a first vehicle pose in real time if the second vehicle pose is different from the first vehicle pose corresponding to the first vehicle position in the first semantic map.

[0115] As an optional implementation, the second obtaining module is specifically configured to:

[0116] obtain a first initial attitude angle;

[0117] perform error processing on the first initial attitude angle by using a vehicle dynamics fusion processing technology to obtain a first target attitude angle;

[0118] determine a second vehicle pose of the autonomous vehicle by matching the first road element with the first semantic map and based on the first vehicle position and the first target attitude angle by using an iterative closest point (ICP) algorithm.

[0119] As an optional implementation, the device further includes:

[0120] a third obtaining module, configured to obtain a second vehicle position, a second road image, and a second initial attitude angle during the driving;

[0121] a fourth obtaining module, configured to perform error processing on the second initial attitude angle by using a vehicle dynamics fusion processing technology to obtain a second target attitude angle;

[0122] a determining module, configured to determine a third vehicle pose of the autonomous vehicle based on the second vehicle position and the second target attitude angle;

[0123] a second obtaining module, configured to perform semantic segmentation on the second road image to obtain a second road element included in the second road image and a second relative position between the second road element and the autonomous vehicle;

[0124] a first obtaining module, configured to perform dynamic inverse projection on the second vehicle position, the second road element, and the second relative position to obtain a second semantic map;

[0125] a second obtaining module, configured to perform projection image adjustment on the second semantic map based on the third vehicle pose to obtain a third semantic map, and send the third semantic map to a cloud server.

[0126] The application also provides another device for memory driving, which includes:Figure 5 As shown in the figure, the device comprises:

[0127] The receiving module 501 is configured to receive first carrier phase observation data of a monitoring satellite sent by a reference station, vehicle positioning data sent by an autonomous vehicle, second carrier phase observation data of the monitoring satellite, and a first semantic map.

[0128] The correction module 502 is configured to correct a vehicle position in the first semantic map based on the first carrier phase observation data, the vehicle positioning data, the second carrier phase observation data, and a dynamic post-processing PPK algorithm, to obtain a corrected first semantic map.

[0129] The fusion module 503 is configured to perform data fusion processing on the first semantic map stored in the cloud and the corrected first semantic map, to obtain a fused first semantic map, and send the fused first semantic map to the autonomous vehicle.

[0130] Embodiments of the present application provide a device for memorizing driving, which constructs a semantic map by acquiring a vehicle position, a road condition element contained in a road condition image, a relative position between the road condition element and the autonomous vehicle, and a vehicle pose during driving. During the process of memorizing driving, the vehicle is controlled to perform memorizing driving according to the semantic map. Through the above method, a semantic map can be constructed in a complex urban area, the vehicle is controlled to perform memorizing driving, the scene coverage rate in the city is improved, and point-to-point memorized route driving is realized.

[0131] The specific limitations of the device for memorizing driving can be referred to the limitations of the method for memorizing driving in the above, which will not be described here. Each module in the device for memorizing driving can be realized by software, hardware, and a combination thereof in whole or in part. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform the operations corresponding to each module.

[0132] In one embodiment, a computer device is provided, as shown in the figure, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the steps of the method for memorizing driving when executing the computer program. Figure 6

[0133] In one embodiment, a computer readable storage medium stores a computer program, which is executed by a processor to implement the steps of the method for memorizing driving.

[0134] ​Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, databases, or other media in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0135] It should be noted that, in this paper, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or equipment including the element.

[0136] It should also be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for display, analyzed data, etc.) involved in this application are information and data authorized by the user or authorized by all parties.

[0137] The various embodiments in the specification are described in a related manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, the system embodiments are described simply because they are basically similar to the method embodiments, and the related parts can be referred to the part of the method embodiments.

[0138] The technical features of the above embodiments can be combined in any manner. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the specification.

[0139] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for memorizing driving routes, characterized in that, The method is applied to autonomous vehicles, and the method includes: During the journey, the first vehicle position and the first road condition image are acquired; The first road condition image is semantically segmented to obtain a first road condition element contained in the first road condition image and a first relative position between the first road condition element and the autonomous vehicle. If the reference road condition element and reference relative position corresponding to the first vehicle position in the first semantic map downloaded from the cloud server are the same as the first road condition element and the first relative position, then the autonomous vehicle is controlled to drive according to the first vehicle pose corresponding to the first vehicle position in the first semantic map. During the driving process, the second vehicle pose of the autonomous vehicle at the first vehicle position is obtained. The process of performing this step is as follows: obtaining a first initial attitude angle; using vehicle dynamics fusion processing technology to process the error of the first initial attitude angle to obtain a first target attitude angle; using the iterative nearest point ICP algorithm, by matching the first road condition element with the first semantic map, and based on the first vehicle position and the first target attitude angle, the second vehicle pose of the autonomous vehicle is determined. If the second vehicle pose is different from the first vehicle pose corresponding to the first vehicle position in the first semantic map, the second vehicle pose will be corrected to the first vehicle pose in real time.

2. The method according to claim 1, characterized in that, The method further includes: During the driving process, the second vehicle position, the second road condition image, and the second initial attitude angle are acquired; The second initial attitude angle is processed using vehicle dynamics fusion processing technology to obtain the second target attitude angle; The third vehicle pose of the autonomous vehicle is determined based on the second vehicle position and the second target attitude angle. The second road condition image is semantically segmented to obtain the second road condition element contained in the second road condition image and the second relative position between the second road condition element and the autonomous vehicle. A second semantic map is obtained by dynamically inversely projecting the second vehicle position, the second road condition element, and the second relative position. Based on the third vehicle pose, the second semantic map is projected and adjusted to obtain a third semantic map, which is then sent to the cloud server.

3. A method for memorizing driving records, characterized in that, The method is applied to a cloud server, which is used to implement the method as described in any one of claims 1-2, including: The system receives the first carrier phase observation data from the monitoring satellite transmitted by the base station, the vehicle positioning data transmitted by the autonomous vehicle, the second carrier phase observation data from the monitoring satellite, and the first semantic map. Based on the first carrier phase observation data, the vehicle positioning data, the second carrier phase observation data, and the dynamic post-processing PPK algorithm, the vehicle position in the first semantic map is corrected to obtain the corrected first semantic map. The first semantic map stored in the cloud and the corrected first semantic map are fused together to obtain a fused first semantic map, which is then sent to the autonomous vehicle.

4. A device for remembering driving records, characterized in that, The device is used in autonomous vehicles, and the device includes: The first acquisition module is used to acquire the position of the first vehicle and the first road condition image during the driving process; The first obtaining module is used to perform semantic segmentation on the first road condition image to obtain a first road condition element contained in the first road condition image and a first relative position between the first road condition element and the autonomous vehicle. The control module is configured to control the autonomous vehicle to perform memory driving according to the first vehicle pose corresponding to the first vehicle position in the first semantic map if the reference road condition element and reference relative position corresponding to the first vehicle position in the first semantic map downloaded from the cloud server are the same as the first road condition element and the first relative position, respectively. The second acquisition module is used to acquire the second vehicle pose of the autonomous vehicle at the first vehicle position during the driving process. The process of performing this step is as follows: acquiring a first initial attitude angle; using vehicle dynamics fusion processing technology to process the error of the first initial attitude angle to obtain a first target attitude angle; using the iterative nearest point ICP algorithm to match the first road condition element with the first semantic map, and based on the first vehicle position and the first target attitude angle, determining the second vehicle pose of the autonomous vehicle. The correction module is used to correct the second vehicle pose to the first vehicle pose in real time if the second vehicle pose is different from the first vehicle pose corresponding to the first vehicle position in the first semantic map.

5. The apparatus according to claim 4, characterized in that, The device further includes: The third acquisition module is used to acquire the second vehicle position, the second road condition image, and the second initial attitude angle during driving. The fourth acquisition module is used to perform error processing on the second initial attitude angle using vehicle dynamics fusion processing technology to obtain the second target attitude angle; The determination module is used to determine the third vehicle pose of the autonomous vehicle based on the second vehicle position and the second target attitude angle; The second obtaining module is used to perform semantic segmentation on the second road condition image to obtain the second road condition element contained in the second road condition image and the second relative position between the second road condition element and the autonomous vehicle. The first acquisition module is used to perform dynamic inverse projection on the second vehicle position, the second road condition element and the second relative position to obtain a second semantic map; The second acquisition module is used to adjust the projection image of the second semantic map according to the third vehicle pose to obtain a third semantic map, and send the third semantic map to the cloud server.

6. A device for remembering driving records, characterized in that, The apparatus is applied to a cloud server, the cloud server being used to implement the apparatus as described in any one of claims 4-5, comprising: The receiving module is used to receive the first carrier phase observation data of the monitoring satellite sent by the base station, the vehicle positioning data sent by the autonomous vehicle, the second carrier phase observation data of the monitoring satellite, and the first semantic map. The correction module is used to correct the vehicle position in the first semantic map based on the first carrier phase observation data, the vehicle positioning data, the second carrier phase observation data, and the dynamic post-processing PPK algorithm, so as to obtain the corrected first semantic map. The fusion module is used to perform data fusion processing on the first semantic map stored in the cloud and the corrected first semantic map to obtain the fused first semantic map, and send the fused first semantic map to the autonomous vehicle.

Images