Coloring method and device of three-dimensional road surface, storage medium, electronic equipment and vehicle
By projecting 3D road surface point clouds onto road images and directly coloring them, the problem of time-consuming and inaccurate generation of 3D road surface edge lines is solved, achieving efficient and accurate road edge line generation, which is suitable for 3D road surface edge line generation in 3D electronic maps.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- MOMENTA (SUZHOU) TECHNOLOGY CO LTD
- Filing Date
- 2022-08-23
- Publication Date
- 2026-06-09
Smart Images
Figure CN117689784B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of automotive technology, and more specifically, to a method, apparatus, storage medium, electronic device, and vehicle for coloring a three-dimensional road surface. Background Technology
[0002] Electronic maps are maps stored and viewed digitally using computer technology. People can consult electronic maps to find routes to their destinations while walking, cycling, or driving, and can also use intelligent navigation functions for real-time navigation. To improve user experience and the accuracy of electronic map queries, two-dimensional electronic maps have gradually evolved into three-dimensional electronic maps. One of the most crucial steps in generating a three-dimensional electronic map is generating the three-dimensional road surface. Related technologies primarily rely on LiDAR to collect radar point clouds to generate the three-dimensional road surface. Furthermore, when performing quality checks and corrections on road surface elements such as stop lines, sidewalks, and median strips based on this three-dimensional road surface, road edge lines can be generated as reference lines within the three-dimensional road surface to improve accuracy and efficiency.
[0003] Methods for generating road edge lines in related technologies mainly include: acquiring each frame of road image captured by the camera of the acquisition vehicle when acquiring radar point clouds using a LiDAR on the acquisition vehicle; perceiving the road edge lines in each frame of road image; projecting the ground parameters of a pixel on the road edge line in each frame of road image onto the 3D road surface to obtain a projection point, and fitting a plane containing the projection point with the projection point as the center and the ground parameters of the road image as the normal vector; projecting the pixel onto the corresponding plane, and the intersection with the plane is the point of the road edge line in the 3D road surface. However, when part of the surface of the 3D road surface has a slope, but the fitted plane does not have a slope, directly using the intersection point as the point of the road edge line will lead to inaccuracy. To improve accuracy, one can search for nearby surfaces based on the intersection point and iterate gradually to the correct position, but this blind iteration method is not only time-consuming, but may also fail to find the correct surface. Summary of the Invention
[0004] This application provides a method, apparatus, storage medium, electronic device, and vehicle for coloring three-dimensional road surfaces, which can solve the problems of time-consuming and inaccurate generation of road edge lines.
[0005] The specific technical solution is as follows:
[0006] In a first aspect, embodiments of this application provide a method for coloring a three-dimensional road surface, the method comprising:
[0007] Based on the preset spacing and surface parameters of the three-dimensional road surface, the three-dimensional road surface is restored to generate a first three-dimensional road surface point cloud. The three-dimensional road surface is an uncolored three-dimensional road surface fitted based on the original radar point cloud.
[0008] Based on the pose of each road image in the first three-dimensional road surface point cloud, a second three-dimensional road surface point cloud corresponding to the road image is determined. The road image is the road image captured by the camera of the acquisition vehicle when the LiDAR of the acquisition vehicle acquires the original radar point cloud. The second three-dimensional road surface point cloud includes three-dimensional road surface points in the first three-dimensional road surface point cloud that are less than or equal to the pose of the road image.
[0009] The second three-dimensional road surface point cloud is projected onto the corresponding road image to obtain the projected point cloud in the road image;
[0010] The target 3D road surface points in the first 3D road surface point cloud corresponding to the target projection point are colored with the target color to highlight the road edge of the 3D road surface. The target projection point includes the projection point located in the target area in the projection point cloud of each frame of the road image. The target area includes the area composed of pixels with the same horizontal coordinate as the perceived static road edge line in the road image and whose vertical coordinates differ within a preset difference range. The target 3D road surface points include 3D road surface points that have a direct mapping relationship with the target projection point, or 3D road surface points obtained by raising the elevation of 3D road surface points that have a direct mapping relationship with the target projection point by a preset height.
[0011] As can be seen from the above scheme, after restoring the uncolored 3D road surface into a first 3D road surface point cloud, the second 3D road surface point cloud near the road image can be projected onto the road image to obtain a projected point cloud. The target 3D road surface points in the first 3D road surface point cloud corresponding to the target projection points located within the target area in the projected point cloud are directly colored with the target color. The target area includes a region composed of pixels whose horizontal coordinates are the same as the perceived static road edge line in the road image and whose vertical coordinates differ within a preset range. The target 3D road surface points include 3D road surface points with a direct mapping relationship to the target projection points, or 3D road points obtained by adjusting the elevation of these 3D road surface points. Therefore, since the embodiment of this application projects the second 3D road surface point cloud from the 3D road surface directly onto the 2D road image, and directly determines and colors the 3D road surface points of the road edge line using the mapping relationship between the perceived road edge line and its nearby projection points in the road image, this embodiment does not require the construction of a plane and does not involve the problem of inconsistent slopes between the plane and the 3D road surface. Therefore, it does not require iteratively searching for nearby curved surfaces, thus improving both the generation efficiency and accuracy of the road edge line.
[0012] In a first possible implementation of the first aspect, before projecting the second three-dimensional road surface point cloud onto the corresponding road image to obtain the projected point cloud in the road image, the method further includes:
[0013] Transform the second three-dimensional road surface point cloud from the station center coordinate system to the camera coordinate system;
[0014] Filter out invalid 3D road points in the second 3D road point cloud in the camera coordinate system, wherein the invalid 3D road points include 3D road points whose elevation is outside the target elevation range and 3D road points whose forward coordinate is less than or equal to 0;
[0015] The step of projecting the second three-dimensional road surface point cloud onto the corresponding road image to obtain the projected point cloud in the road image includes:
[0016] The filtered second three-dimensional road surface point cloud in the camera coordinate system is projected onto the corresponding road image to obtain the projected point cloud in the road image.
[0017] As can be seen from the above scheme, the embodiments of this application can remove road sections on or under bridges or in upper and lower layers by filtering out three-dimensional road surface points whose elevation is outside the target elevation range, thereby making the obtained projection point cloud more accurate. By filtering out three-dimensional road surface points whose forward coordinates are less than or equal to 0, three-dimensional road surface points located outside the camera's field of view can be filtered out, thereby reducing the amount of data converted from the camera coordinate system to the image coordinate system and improving the generation efficiency of road edges.
[0018] In a second possible implementation of the first aspect, before coloring the target three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the target projection point with the target color to highlight the road edge of the three-dimensional road surface, the method further includes:
[0019] Filter out invalid projection points in the projection point cloud, wherein the invalid projection points include projection points located outside the image coordinate system, projection points located below the hood line of the acquisition vehicle, projection points located above the static road edge line, and projection points located above the dynamic road edge line.
[0020] As can be seen from the above scheme, the embodiments of this application can improve the accuracy of the projection point cloud by filtering out projection points outside the drivable area (i.e. invalid projection points), thereby not only improving the accuracy of generating road edges, but also improving the accuracy of subsequent generation of colored pavement or semantically segmented pavement.
[0021] In a third possible implementation of the first aspect, coloring the target three-dimensional road surface point in the first three-dimensional road surface point cloud corresponding to the target projection point with the target color includes:
[0022] The three-dimensional road points in the first three-dimensional road point cloud that meet the preset conditions are determined as the target three-dimensional road points, and the target three-dimensional road points are colored with the target color.
[0023] The preset conditions include any one of the following:
[0024] The three-dimensional road surface points in the first three-dimensional road surface point cloud that have a direct mapping relationship with the target projection point;
[0025] The three-dimensional road surface points obtained after raising the elevation of the three-dimensional road surface points with direct mapping relationship by the preset height;
[0026] The three-dimensional road surface points with direct mapping relationship are filtered based on the drivable area point cloud. The drivable area point cloud includes the three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the projection points in the area enclosed by the hood line, the static road edge line and the dynamic road edge line in the filtered projection point cloud.
[0027] After filtering the three-dimensional road surface points with direct mapping relationships based on the drivable area point cloud, the elevation of the remaining three-dimensional road surface points is increased by the preset height to obtain the three-dimensional road surface points.
[0028] As can be seen from the above solution, the elevation adjustment in this application embodiment can avoid the drawn road edges from obscuring the original road surface landmarks, and the point cloud of the drivable area can filter out the three-dimensional road surface points in the road, thereby improving the accuracy and display effect of the road edges.
[0029] In a fourth possible implementation of the first aspect, after filtering invalid 3D road points in the second 3D road point cloud in the camera coordinate system, the method further includes:
[0030] Based on the target distance of each unprocessed 3D road surface point in the filtered second 3D road surface point cloud under the camera coordinate system, the weight value corresponding to the unprocessed 3D road surface point is calculated, wherein the target distance is the target distance between the lateral coordinate and the forward coordinate of the unprocessed 3D road surface point, and the target distance is negatively correlated with the weight value;
[0031] After filtering invalid projection points in the projection point cloud, the method further includes:
[0032] In the case where there is only one valid projection point corresponding to a three-dimensional road surface point in the first three-dimensional road surface point cloud, the three-dimensional road surface point in the first three-dimensional road surface point cloud corresponding to the valid projection point is colored with the color value of the valid projection point, wherein the valid projection point is the projection point contained in the filtered projection point cloud.
[0033] In the case where multiple valid projection points correspond to the same three-dimensional road surface point in the first three-dimensional road surface point cloud, the color value and the weight value of the multiple valid projection points are weighted and calculated, and the three-dimensional road surface point corresponding to the multiple valid projection points is colored with the weighted color value, wherein each of the multiple valid projection points is located in a different road image.
[0034] As can be seen from the above scheme, the embodiments of this application can achieve the coloring of three-dimensional road surfaces by coloring the three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the effective projection points. For example, when the road image is a color image, a color road surface can be obtained; when the road image is a semantically segmented image, a semantically segmented road surface can be obtained. Furthermore, when coloring the three-dimensional road surface, weighted calculations can be performed based on weight values determined by distance information, thereby improving the accuracy of three-dimensional road surface coloring.
[0035] Secondly, embodiments of this application provide a coloring device for a three-dimensional road surface, the device comprising:
[0036] The restoration unit is used to restore the three-dimensional road surface according to the preset spacing and the surface parameters of the three-dimensional road surface to generate a first three-dimensional road surface point cloud, wherein the three-dimensional road surface is an uncolored three-dimensional road surface fitted based on the original radar point cloud.
[0037] The determining unit is used to determine the second three-dimensional road surface point cloud corresponding to the road image based on the pose of each frame of road image in the first three-dimensional road surface point cloud. The road image is the road image captured by the camera of the acquisition vehicle when the lidar of the acquisition vehicle acquires the original radar point cloud. The second three-dimensional road surface point cloud includes three-dimensional road surface points in the first three-dimensional road surface point cloud that are less than or equal to the pose of the road image.
[0038] A projection unit is used to project the second three-dimensional road surface point cloud onto the corresponding road image to obtain the projected point cloud in the road image.
[0039] A coloring unit is used to color the target three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the target projection point with a target color to highlight the road edge of the three-dimensional road surface. The target projection point includes projection points located within the target area in the projection point cloud of each frame of the road image. The target area includes a region composed of pixels whose horizontal coordinates are the same as those of the perceived static road edge line in the road image and whose vertical coordinates differ within a preset range. The target three-dimensional road surface points include three-dimensional road surface points that have a direct mapping relationship with the target projection point, or three-dimensional road surface points obtained by raising the elevation of three-dimensional road surface points that have a direct mapping relationship with the target projection point by a preset height.
[0040] In a first possible implementation of the second aspect, the device further includes:
[0041] The coordinate system transformation unit is used to transform the second three-dimensional road surface point cloud from the station center coordinate system to the camera coordinate system before projecting the second three-dimensional road surface point cloud onto the corresponding road image to obtain the projected point cloud in the road image;
[0042] The first filtering unit is used to filter invalid three-dimensional road points in the second three-dimensional road point cloud under the camera coordinate system, wherein the invalid three-dimensional road points include three-dimensional road points whose elevation is outside the target elevation range and three-dimensional road points whose forward coordinate is less than or equal to 0.
[0043] The projection unit is used to project the filtered second three-dimensional road surface point cloud in the camera coordinate system onto the corresponding road image to obtain the projected point cloud in the road image.
[0044] In a second possible implementation of the second aspect, the device further includes:
[0045] The second filtering unit is used to filter invalid projection points in the projection point cloud before coloring the target three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the target projection point with the target color to highlight the road edge of the three-dimensional road surface. The invalid projection points include projection points located outside the image coordinate system, projection points located below the hood line of the acquisition vehicle, projection points located above the static road edge line, and projection points located above the dynamic road edge line.
[0046] In a third possible implementation of the second aspect, the coloring unit is used to determine the three-dimensional road surface points in the first three-dimensional road surface point cloud that meet the preset conditions as the target three-dimensional road surface points, and to color the target three-dimensional road surface points with the target color.
[0047] The preset conditions include any one of the following:
[0048] The three-dimensional road surface points in the first three-dimensional road surface point cloud that have a direct mapping relationship with the target projection point;
[0049] The three-dimensional road surface points obtained after raising the elevation of the three-dimensional road surface points with direct mapping relationship by the preset height;
[0050] The three-dimensional road surface points with direct mapping relationship are filtered based on the drivable area point cloud. The drivable area point cloud includes the three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the projection points in the area enclosed by the hood line, the static road edge line and the dynamic road edge line in the filtered projection point cloud.
[0051] After filtering the three-dimensional road surface points with direct mapping relationships based on the drivable area point cloud, the elevation of the remaining three-dimensional road surface points is increased by the preset height to obtain the three-dimensional road surface points.
[0052] In a fourth possible implementation of the second aspect, the device further includes:
[0053] The calculation unit is used to calculate the weight value corresponding to each three-dimensional road surface point to be processed in the second three-dimensional road surface point cloud under the camera coordinate system after filtering out invalid three-dimensional road surface points in the second three-dimensional road surface point cloud under the camera coordinate system, based on the target distance of each three-dimensional road surface point to be processed in the filtered second three-dimensional road surface point cloud under the camera coordinate system, wherein the target distance is the target distance between the lateral coordinate and the forward coordinate of the three-dimensional road surface point to be processed, and the target distance is negatively correlated with the weight value;
[0054] The coloring unit is further configured to, after filtering invalid projection points in the projection point cloud, color the three-dimensional road surface point in the first three-dimensional road surface point cloud corresponding to the valid projection point as the color value of the valid projection point when there is only one valid projection point corresponding to a three-dimensional road surface point in the first three-dimensional road surface point cloud, wherein the valid projection point is a projection point included in the filtered projection point cloud; and when there are multiple valid projection points corresponding to the same three-dimensional road surface point in the first three-dimensional road surface point cloud, perform a weighted calculation on the color value and the weight value of the multiple valid projection points, and color the three-dimensional road surface point corresponding to the multiple valid projection points as the weighted color value, wherein each of the multiple valid projection points is located in a different road image.
[0055] The three-dimensional road surface coloring device provided in this application embodiment, after restoring the uncolored three-dimensional road surface to a first three-dimensional road surface point cloud, can project a second three-dimensional road surface point cloud near the road image onto the road image to obtain a projected point cloud. It then directly colors the target three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the target projection points located within the target area in the projected point cloud to the target color. The target area includes a region composed of pixels whose horizontal coordinates are the same as the perceived static road edge line in the road image and whose vertical coordinates differ within a preset range. The target three-dimensional road surface points include three-dimensional road surface points with a direct mapping relationship to the target projection points, or three-dimensional road points obtained by adjusting the elevation of these three-dimensional road surface points. Therefore, since this application embodiment projects the second three-dimensional road surface point cloud from the three-dimensional road surface directly onto a two-dimensional road image, and directly determines and colors the three-dimensional road surface points of the road edge line using the mapping relationship between the perceived road edge line and its nearby projection points in the road image, this application embodiment does not require the construction of a plane and does not involve the problem of inconsistent slopes between the plane and the three-dimensional road surface. Thus, it eliminates the need for iteratively searching for nearby curved surfaces, thereby improving both the generation efficiency and accuracy of the road edge line.
[0056] Thirdly, embodiments of this application provide a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any possible implementation of the first aspect.
[0057] Fourthly, embodiments of this application provide an electronic device, which includes:
[0058] One or more processors;
[0059] Storage device for storing one or more programs.
[0060] When one or more programs are executed by one or more processors, the electronic device performs the method as described in any possible implementation of the first aspect.
[0061] Fifthly, embodiments of this application provide a vehicle that includes the means as described in any possible implementation of the second aspect, or includes electronic equipment as described in the fourth aspect. Attached Figure Description
[0062] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.
[0063] Figure 1 A schematic flowchart illustrating a three-dimensional road surface coloring method provided in an embodiment of this application;
[0064] Figure 2 A schematic diagram of the coordinate system provided in the embodiments of this application;
[0065] Figure 3 An example diagram of an invalid projection point and a valid projection point provided in an embodiment of this application;
[0066] Figure 4 A block diagram illustrating the composition of a three-dimensional road surface coloring device provided in this application embodiment;
[0067] Figure 5 This is a structural schematic diagram of a vehicle provided in an embodiment of this application. Detailed Implementation
[0068] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.
[0069] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.
[0070] Figure 1 This is a flowchart illustrating a method for coloring a three-dimensional road surface. This method can be applied to electronic or computer equipment, specifically to vehicles or servers, and may include the following steps:
[0071] S110: Based on the preset spacing and surface parameters of the three-dimensional road surface, perform a restoration operation on the three-dimensional road surface to generate the first three-dimensional road surface point cloud.
[0072] The three-dimensional road surface is an uncolored three-dimensional road surface fitted based on the original radar point cloud. This application does not limit the method for generating a three-dimensional road surface based on the original radar point cloud. For example, radar point clouds related to the road surface can be selected from the original radar point cloud first, then the selected radar point clouds can be fitted in the horizontal direction, and finally fitted in the vertical direction to generate a three-dimensional road surface as the three-dimensional road surface.
[0073] The spacing between adjacent 3D road surface points in the first 3D road surface point cloud is a preset spacing. The preset spacing can be the same as or different from the spacing in the original radar point cloud, for example, it can be 0.5m. Both the 3D road surface and the first 3D road surface point cloud can be in the station-centered coordinate system (East-North-Sky coordinate system ENU), or can be transformed from the world coordinate system to the station-centered coordinate system.
[0074] S120: Determine the second three-dimensional road surface point cloud corresponding to the road image based on the pose of each road image in the first three-dimensional road surface point cloud.
[0075] The road image refers to the road image captured by the camera of the acquisition vehicle during the acquisition of the original radar point cloud by the LiDAR of the acquisition vehicle. The road image may have been acquired by the acquisition vehicle multiple times and / or thinned, retaining a reasonable amount of data. This application embodiment does not limit the installation angle and position of the camera on the acquisition vehicle. The second three-dimensional road surface point cloud includes three-dimensional road surface points in the first three-dimensional road surface point cloud that are less than or equal to the pose of the road image. The pose includes position and orientation. The preset distance can be determined based on the final generated road edge accuracy. For example, when the preset distance is 25m, the second three-dimensional road surface point cloud corresponding to the road image can be the second three-dimensional road surface point cloud within a circular area defined by the center point of the road image and a radius of 25m.
[0076] It should be added that, in order to improve the efficiency of road edge generation, the embodiments of this application can first divide the first three-dimensional road surface point cloud into multiple region blocks, and then perform the step "determine the second three-dimensional road surface point cloud corresponding to the road image according to the pose of each frame of road image in the region block" and subsequent steps in parallel on multiple region blocks. The block division principle can be: traverse the first three-dimensional road surface point cloud, stop traversing when the number of traversed point clouds reaches the upper limit, and determine the traversed point clouds as the same region block, and then continue to traverse the untraversed first three-dimensional road surface point clouds using this method until all three-dimensional road surface points in the first three-dimensional road surface point cloud have been traversed.
[0077] S130: Project the second three-dimensional road surface point cloud onto the corresponding road image to obtain the projected point cloud in the road image.
[0078] This step involves first transforming the second 3D road surface point cloud from the station coordinate system to the camera coordinate system, and then from the camera coordinate system to the image coordinate system, thereby realizing the projection of the second 3D road surface point cloud onto the road image and obtaining the projected point cloud in the road image.
[0079] S140: Color the target 3D road surface point in the first 3D road surface point cloud corresponding to the target projection point with the target color to highlight the road edge of the 3D road surface.
[0080] The target projection points include projection points located within the target area in the projection point cloud of each frame of the road image. The target area includes the region composed of pixels whose horizontal coordinates are the same as those of the perceived static road edge lines in the road image and whose vertical coordinates differ from those within a preset difference range. Static road edge lines include fixed boundary lines such as road fences used to distinguish between roads and non-roads, while dynamic road edge lines are the rear edge of vehicles in front of the acquisition vehicle. Therefore, dynamic road edge lines may or may not exist. The preset difference range can be determined based on the accuracy of the final generated road edge, for example, it can be 5 pixels. Target colors include colors other than road surface colors, especially colors that differ significantly from the road surface color, such as red, green, etc. The method for perceiving static and dynamic road edge lines, as mentioned in the following embodiments, is as follows: at least one of these three types of lines is pre-annotated on a large number of historical road images, and then a model is trained using historical road images containing annotation information to obtain a target detection model. Finally, the target detection model is used to detect lines in the road image to be perceived. The target detection model can be a CNN (Convolutional Neural Network) model.
[0081] The target 3D road surface points include 3D road surface points that have a direct mapping relationship with the target projection point, or 3D road surface points obtained by raising the elevation of 3D road surface points that have a direct mapping relationship with the target projection point by a preset height. A 3D road surface point that has a direct mapping relationship with the target projection point means that the target projection point and the 3D road surface point are the same point in different coordinate systems. The preset height can be determined based on the final generated road edge accuracy.
[0082] Specifically, three-dimensional road points in the first three-dimensional road point cloud that meet the preset conditions can be identified as target three-dimensional road points, and the target three-dimensional road points can be colored with the target color.
[0083] The preset conditions include any one of the following:
[0084] Three-dimensional road surface points in the first three-dimensional road surface point cloud that have a direct mapping relationship with the target projection point;
[0085] The three-dimensional road surface points are obtained by raising the elevation of the three-dimensional road surface points with direct mapping relationships by a preset height;
[0086] The three-dimensional road surface points with direct mapping relationship are filtered based on the drivable area point cloud. The drivable area point cloud includes the three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the projection points in the area enclosed by the vehicle hood line, static road edge line and dynamic road edge line of the filtered projection point cloud.
[0087] The three-dimensional road surface points are obtained by filtering out the three-dimensional road surface points with direct mapping relationships based on the point cloud of the drivable area, and then raising the elevation of the remaining three-dimensional road surface points by a preset height.
[0088] In this embodiment, adjusting the elevation can prevent the drawn road edges from obscuring the original road surface landmarks. By using the point cloud of the drivable area, the three-dimensional road surface points within the road can be filtered out, thereby improving the accuracy and display effect of the road edges.
[0089] Furthermore, embodiments of this application can also uniformly color the three-dimensional road surface points corresponding to the drivable area point cloud to a certain color (different from the target color) to generate a drivable area road surface in the three-dimensional road surface, thereby enabling functions such as correcting three-dimensional maps generated by other means based on the drivable area road surface.
[0090] The three-dimensional road surface coloring method provided in this application embodiment, after restoring the uncolored three-dimensional road surface to a first three-dimensional road surface point cloud, can project a second three-dimensional road surface point cloud near the road image onto the road image to obtain a projected point cloud. Then, the target three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the target projection points located within the target area in the projected point cloud are directly colored with the target color. The target area includes a region composed of pixels whose horizontal coordinates are the same as the perceived static road edge line in the road image and whose vertical coordinates differ within a preset range. The target three-dimensional road surface points include three-dimensional road surface points with a direct mapping relationship to the target projection points, or three-dimensional road points obtained by adjusting the elevation of these three-dimensional road surface points. Therefore, since this application embodiment projects the second three-dimensional road surface point cloud from the three-dimensional road surface directly onto the two-dimensional road image, and directly determines and colors the three-dimensional road surface points of the road edge line using the mapping relationship between the perceived road edge line and its nearby projection points in the road image, this application embodiment does not require the construction of a plane and does not involve the problem of inconsistent slopes between the plane and the three-dimensional road surface. Therefore, it does not require iteratively searching for nearby curved surfaces, thus improving both the generation efficiency and accuracy of the road edge line.
[0091] In one embodiment, in order to improve the accuracy of the projected point cloud and the efficiency of road edge generation, before projecting the second three-dimensional road surface point cloud onto the corresponding road image to obtain the projected point cloud in the road image, the second three-dimensional road surface point cloud can be transformed from the station coordinate system to the camera coordinate system, then invalid three-dimensional road surface points in the second three-dimensional road surface point cloud in the camera coordinate system are filtered out, and finally the filtered second three-dimensional road surface point cloud in the camera coordinate system is projected onto the corresponding road image to obtain the projected point cloud in the road image.
[0092] Invalid 3D road surface points include those whose elevation is outside the target elevation range and those whose forward coordinate is less than or equal to 0. The target elevation range can be [0, 3m].
[0093] like Figure 2 As shown, the x-axis, y-axis, and z-axis constitute the camera coordinate system with the camera's optical center as the origin. The z-axis is the forward coordinate axis, the y-axis is the coordinate axis used to represent elevation, and the x-axis is the horizontal coordinate axis. The u-axis (horizontal axis) and v-axis (vertical axis) constitute the image coordinate system, and the z-axis passes through the center of the image coordinate system.
[0094] The three-dimensional road surface coloring method provided in this application can remove road sections on or under bridges or in upper and lower layers by filtering out three-dimensional road surface points whose elevation is outside the target elevation range, thereby making the obtained projection point cloud more accurate. By filtering out three-dimensional road surface points whose forward coordinates are less than or equal to 0, three-dimensional road surface points located outside the camera's field of view can be filtered out, thereby reducing the amount of data converted from the camera coordinate system to the image coordinate system and improving the efficiency of road edge generation.
[0095] In one implementation, to improve the accuracy of generating road edges, embodiments of this application can filter invalid projection points in the projection point cloud before coloring the target 3D road surface points in the first 3D road surface point cloud corresponding to the target projection point with the target color to highlight the road edge of the 3D road surface. Invalid projection points include projection points located outside the image coordinate system, projection points located below the hood line of the acquisition vehicle, projection points located above the static road edge line, and projection points located above the dynamic road edge line. Figure 3 As shown, the shaded area represents the effective projection point region. The area formed by the effective projection points within this region can be called the drivable region. The unshaded area represents the invalid projection point region. Effective projection points are all projection points in the projection point cloud except for the invalid projection points.
[0096] It should be added that filtering invalid projection points in the projection point cloud can not only improve the accuracy of generating road edges, but also improve the accuracy of subsequent generation of colored pavement or semantically segmented pavement.
[0097] In one embodiment, after filtering invalid 3D road points in the second 3D road point cloud in the camera coordinate system, this application embodiment can further calculate the weight value corresponding to the 3D road point to be processed based on the target distance of each 3D road point to be processed in the filtered second 3D road point cloud in the camera coordinate system. The target distance is the target distance between the lateral coordinate and the forward coordinate of the 3D road point to be processed, and the target distance is negatively correlated with the weight value. After filtering invalid projection points in the projection point cloud, if only one valid projection point corresponds to a 3D road point in the first 3D road point cloud, the 3D road point in the first 3D road point cloud corresponding to the valid projection point is colored with the color value of the valid projection point. The valid projection point is the projection point contained in the filtered projection point cloud. If multiple valid projection points correspond to the same 3D road point in the first 3D road point cloud, the color values and weight values of the multiple valid projection points are weighted and calculated, and the 3D road points corresponding to the multiple valid projection points are colored with the weighted color value. Each of the multiple valid projection points is located in a different road image.
[0098] When the road image used in this application embodiment is an original color road image captured by a camera, the three-dimensional road surface generated after coloring in this application embodiment is a colored road surface; when the road image used in this application embodiment is an image semantically segmented from an original color road image, the three-dimensional road surface generated after coloring in this application embodiment is a semantically segmented road surface. A colored road surface is a road surface with the same color as a real-world road surface; a semantically segmented road surface is a three-dimensional road surface that semantically segments out landmark and non-landmark portions. Road landmarks refer to markings on the road surface that use lines, arrows, text, etc., to convey traffic information such as guidance, restrictions, and warnings to road users.
[0099] In addition, embodiments of this application can also uniformly color uncolored 3D road surface points in the 3D road surface to a certain color (such as gray) to highlight the road surface color.
[0100] The three-dimensional road surface coloring method provided in this application embodiment can achieve three-dimensional road surface coloring by coloring the three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the effective projection points. For example, when the road image is a color image, a color road surface can be obtained; when the road image is a semantically segmented image, a semantically segmented road surface can be obtained. Furthermore, when coloring the three-dimensional road surface, weighted calculations can be performed based on weight values determined by distance information, thereby improving the accuracy of three-dimensional road surface coloring.
[0101] Corresponding to the above method embodiments, another embodiment of this application provides a coloring device for a three-dimensional road surface, such as... Figure 4 As shown, the device includes:
[0102] The restoration unit 21 is used to perform a restoration operation on the three-dimensional road surface according to the preset spacing and the surface parameters of the three-dimensional road surface to generate a first three-dimensional road surface point cloud, wherein the three-dimensional road surface is an uncolored three-dimensional road surface fitted based on the original radar point cloud.
[0103] The determining unit 22 is used to determine the second three-dimensional road surface point cloud corresponding to the road image based on the pose of each frame of road image in the first three-dimensional road surface point cloud. The road image is the road image captured by the camera of the acquisition vehicle when the lidar of the acquisition vehicle acquires the original radar point cloud. The second three-dimensional road surface point cloud includes three-dimensional road surface points in the first three-dimensional road surface point cloud that are less than or equal to the pose of the road image.
[0104] Projection unit 23 is used to project the second three-dimensional road surface point cloud onto the corresponding road image to obtain the projected point cloud in the road image;
[0105] The coloring unit 24 is used to color the target three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the target projection point into the target color to highlight the road edge of the three-dimensional road surface. The target projection point includes the projection point located in the target area in the projection point cloud of each frame of the road image. The target area includes the area composed of pixels with the same horizontal coordinate as the perceived static road edge line in the road image and whose vertical coordinates differ within a preset difference range. The target three-dimensional road surface points include three-dimensional road surface points that have a direct mapping relationship with the target projection point, or three-dimensional road surface points obtained after raising the elevation of the three-dimensional road surface points that have a direct mapping relationship with the target projection point by a preset height.
[0106] In one possible implementation, the device further includes:
[0107] The coordinate system transformation unit is used to transform the second three-dimensional road surface point cloud from the station center coordinate system to the camera coordinate system before projecting the second three-dimensional road surface point cloud onto the corresponding road image to obtain the projected point cloud in the road image.
[0108] The first filtering unit is used to filter invalid three-dimensional road points in the second three-dimensional road point cloud in the camera coordinate system. Invalid three-dimensional road points include three-dimensional road points whose elevation is outside the target elevation range and three-dimensional road points whose forward coordinate is less than or equal to 0.
[0109] Projection unit 23 is used to project the filtered second three-dimensional road surface point cloud in the camera coordinate system onto the corresponding road image to obtain the projected point cloud in the road image.
[0110] In one possible implementation, the device further includes:
[0111] The second filtering unit is used to filter invalid projection points in the projection point cloud before coloring the target three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the target projection point with the target color to highlight the road edge of the three-dimensional road surface. The invalid projection points include projection points located outside the image coordinate system, projection points located below the hood line of the acquisition vehicle, projection points located above the static road edge line, and projection points located above the dynamic road edge line.
[0112] In one possible implementation, the coloring unit 24 is used to determine the three-dimensional road points in the first three-dimensional road point cloud that meet the preset conditions as target three-dimensional road points, and color the target three-dimensional road points with the target color.
[0113] The preset conditions include any one of the following:
[0114] Three-dimensional road surface points in the first three-dimensional road surface point cloud that have a direct mapping relationship with the target projection point;
[0115] The three-dimensional road surface points are obtained by raising the elevation of the three-dimensional road surface points with direct mapping relationships by a preset height;
[0116] The three-dimensional road surface points with direct mapping relationship are filtered based on the drivable area point cloud. The drivable area point cloud includes the three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the projection points in the area enclosed by the vehicle hood line, static road edge line and dynamic road edge line of the filtered projection point cloud.
[0117] The three-dimensional road surface points are obtained by filtering out the three-dimensional road surface points with direct mapping relationships based on the point cloud of the drivable area, and then raising the elevation of the remaining three-dimensional road surface points by a preset height.
[0118] In one possible implementation, the device further includes:
[0119] The calculation unit is used to calculate the weight value of each unprocessed 3D road point in the second 3D road point cloud under the camera coordinate system after filtering out invalid 3D road points in the second 3D road point cloud under the camera coordinate system, based on the target distance of each unprocessed 3D road point in the filtered second 3D road point cloud under the camera coordinate system. The target distance is the target distance between the horizontal coordinate and the forward coordinate of the unprocessed 3D road point, and the target distance is negatively correlated with the weight value.
[0120] The coloring unit 24 is further configured to, after filtering invalid projection points in the projection point cloud, color the three-dimensional road surface point in the first three-dimensional road surface point cloud corresponding to the valid projection point as the color value of the valid projection point when there is only one valid projection point corresponding to a three-dimensional road surface point in the first three-dimensional road surface point cloud, wherein the valid projection point is the projection point contained in the filtered projection point cloud; and when there are multiple valid projection points corresponding to the same three-dimensional road surface point in the first three-dimensional road surface point cloud, perform a weighted calculation on the color values and weight values of the multiple valid projection points, and color the three-dimensional road surface point corresponding to the multiple valid projection points as the weighted color value, wherein each of the multiple valid projection points is located in a different road image.
[0121] The three-dimensional road surface coloring device provided in this application embodiment, after restoring the uncolored three-dimensional road surface to a first three-dimensional road surface point cloud, can project a second three-dimensional road surface point cloud near the road image onto the road image to obtain a projected point cloud. It then directly colors the target three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the target projection points located within the target area in the projected point cloud to the target color. The target area includes a region composed of pixels whose horizontal coordinates are the same as the perceived static road edge line in the road image and whose vertical coordinates differ within a preset range. The target three-dimensional road surface points include three-dimensional road surface points with a direct mapping relationship to the target projection points, or three-dimensional road points obtained by adjusting the elevation of these three-dimensional road surface points. Therefore, since this application embodiment projects the second three-dimensional road surface point cloud from the three-dimensional road surface directly onto a two-dimensional road image, and directly determines and colors the three-dimensional road surface points of the road edge line using the mapping relationship between the perceived road edge line and its nearby projection points in the road image, this application embodiment does not require the construction of a plane and does not involve the problem of inconsistent slopes between the plane and the three-dimensional road surface. Thus, it eliminates the need for iteratively searching for nearby curved surfaces, thereby improving both the generation efficiency and accuracy of the road edge line.
[0122] Based on the above method embodiments, another embodiment of this application provides a storage medium storing executable instructions thereon, which, when executed by a processor, cause the processor to implement the method described in any of the above embodiments.
[0123] Based on the above method embodiments, another embodiment of this application provides an electronic device or computer device, including:
[0124] One or more processors;
[0125] Storage device for storing one or more programs.
[0126] When the one or more programs are executed by the one or more processors, the electronic device or computer device performs the method as described in any of the above embodiments.
[0127] Based on the above method embodiments, another embodiment of this application provides a vehicle that includes the apparatus as described in any of the above embodiments, or includes electronic devices as described above.
[0128] like Figure 5 As shown, the vehicle includes a CPU (Central Processing Unit) 31, a GPS (Global Positioning System) positioning device 32, a T-Box (Telematics Box) 33, a V2X (Vehicle-to-Everything) module 34, a LiDAR 35, and a camera 36. The GPS positioning device 32 is used to obtain the vehicle's current geographical location; the T-Box 33 can act as a gateway to communicate with the server; the CPU 31 can execute the 3D road surface coloring method mentioned in the above embodiments; the LiDAR 35 is used to acquire radar point clouds in front of the vehicle; and the camera 36 is used to acquire road images in front of the vehicle.
[0129] The above-described apparatus embodiments correspond to the method embodiments and have the same technical effects. For detailed descriptions, please refer to the method embodiments. The apparatus embodiments are derived from the method embodiments; detailed descriptions can be found in the method embodiments section, and will not be repeated here. Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application.
[0130] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.
[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A method for coloring a three-dimensional road surface, characterized in that, The method includes: Based on the preset spacing and surface parameters of the three-dimensional road surface, the three-dimensional road surface is restored to generate a first three-dimensional road surface point cloud. The three-dimensional road surface is an uncolored three-dimensional road surface fitted based on the original radar point cloud. Based on the pose of each road image in the first three-dimensional road surface point cloud, a second three-dimensional road surface point cloud corresponding to the road image is determined. The road image is the road image captured by the camera of the acquisition vehicle when the LiDAR of the acquisition vehicle acquires the original radar point cloud. The second three-dimensional road surface point cloud includes three-dimensional road surface points in the first three-dimensional road surface point cloud that are less than or equal to the pose of the road image. The second three-dimensional road surface point cloud is projected onto the corresponding road image to obtain the projected point cloud in the road image; The target 3D road surface points in the first 3D road surface point cloud corresponding to the target projection point are colored with the target color to highlight the road edge of the 3D road surface. The target projection point includes the projection point located in the target area in the projection point cloud of each frame of the road image. The target area includes the area composed of pixels with the same horizontal coordinate as the perceived static road edge line in the road image and whose vertical coordinates differ within a preset difference range. The target 3D road surface points include 3D road surface points that have a direct mapping relationship with the target projection point, or 3D road surface points obtained by raising the elevation of 3D road surface points that have a direct mapping relationship with the target projection point by a preset height.
2. The method according to claim 1, characterized in that, Before projecting the second three-dimensional road surface point cloud onto the corresponding road image to obtain the projected point cloud in the road image, the method further includes: Transform the second three-dimensional road surface point cloud from the station center coordinate system to the camera coordinate system; Filter out invalid 3D road points in the second 3D road point cloud in the camera coordinate system, wherein the invalid 3D road points include 3D road points whose elevation is outside the target elevation range and 3D road points whose forward coordinate is less than or equal to 0; The step of projecting the second three-dimensional road surface point cloud onto the corresponding road image to obtain the projected point cloud in the road image includes: The filtered second three-dimensional road surface point cloud in the camera coordinate system is projected onto the corresponding road image to obtain the projected point cloud in the road image.
3. The method according to claim 2, characterized in that, Before coloring the target 3D road surface points in the first 3D road surface point cloud corresponding to the target projection point with the target color to highlight the road edge of the 3D road surface, the method further includes: Filter out invalid projection points in the projection point cloud, wherein the invalid projection points include projection points located outside the image coordinate system, projection points located below the hood line of the acquisition vehicle, projection points located above the static road edge line, and projection points located above the dynamic road edge line.
4. The method according to claim 3, characterized in that, The step of coloring the target 3D road surface point in the first 3D road surface point cloud corresponding to the target projection point with the target color includes: The three-dimensional road points in the first three-dimensional road point cloud that meet the preset conditions are determined as the target three-dimensional road points, and the target three-dimensional road points are colored with the target color. The preset conditions include any one of the following: The three-dimensional road surface points in the first three-dimensional road surface point cloud that have a direct mapping relationship with the target projection point; The three-dimensional road surface points obtained after raising the elevation of the three-dimensional road surface points with direct mapping relationship by the preset height; The three-dimensional road surface points with direct mapping relationship are filtered based on the drivable area point cloud. The drivable area point cloud includes the three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the projection points in the area enclosed by the hood line, the static road edge line and the dynamic road edge line in the filtered projection point cloud. After filtering the three-dimensional road surface points with direct mapping relationships based on the drivable area point cloud, the elevation of the remaining three-dimensional road surface points is increased by the preset height to obtain the three-dimensional road surface points.
5. The method according to claim 3 or 4, characterized in that, After filtering out invalid 3D road points in the second 3D road point cloud in the camera coordinate system, the method further includes: Based on the target distance of each unprocessed 3D road surface point in the filtered second 3D road surface point cloud under the camera coordinate system, the weight value corresponding to the unprocessed 3D road surface point is calculated, wherein the target distance is the target distance between the lateral coordinate and the forward coordinate of the unprocessed 3D road surface point, and the target distance is negatively correlated with the weight value; After filtering invalid projection points in the projection point cloud, the method further includes: In the case where there is only one valid projection point corresponding to a three-dimensional road surface point in the first three-dimensional road surface point cloud, the three-dimensional road surface point in the first three-dimensional road surface point cloud corresponding to the valid projection point is colored with the color value of the valid projection point, wherein the valid projection point is the projection point contained in the filtered projection point cloud. In the case where multiple valid projection points correspond to the same three-dimensional road surface point in the first three-dimensional road surface point cloud, the color value and the weight value of the multiple valid projection points are weighted and calculated, and the three-dimensional road surface point corresponding to the multiple valid projection points is colored with the weighted color value, wherein each of the multiple valid projection points is located in a different road image.
6. A coloring device for a three-dimensional road surface, characterized in that, The device includes: The restoration unit is used to restore the three-dimensional road surface according to the preset spacing and the surface parameters of the three-dimensional road surface to generate a first three-dimensional road surface point cloud, wherein the three-dimensional road surface is an uncolored three-dimensional road surface fitted based on the original radar point cloud. The determining unit is used to determine the second three-dimensional road surface point cloud corresponding to the road image based on the pose of each frame of road image in the first three-dimensional road surface point cloud. The road image is the road image captured by the camera of the acquisition vehicle when the lidar of the acquisition vehicle acquires the original radar point cloud. The second three-dimensional road surface point cloud includes three-dimensional road surface points in the first three-dimensional road surface point cloud that are less than or equal to the pose of the road image. A projection unit is used to project the second three-dimensional road surface point cloud onto the corresponding road image to obtain the projected point cloud in the road image. A coloring unit is used to color the target three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the target projection point with a target color to highlight the road edge of the three-dimensional road surface. The target projection point includes projection points located within the target area in the projection point cloud of each frame of the road image. The target area includes a region composed of pixels whose horizontal coordinates are the same as those of the perceived static road edge line in the road image and whose vertical coordinates differ within a preset range. The target three-dimensional road surface points include three-dimensional road surface points that have a direct mapping relationship with the target projection point, or three-dimensional road surface points obtained by raising the elevation of three-dimensional road surface points that have a direct mapping relationship with the target projection point by a preset height.
7. The apparatus according to claim 6, characterized in that, The device further includes: The coordinate system transformation unit is used to transform the second three-dimensional road surface point cloud from the station center coordinate system to the camera coordinate system before projecting the second three-dimensional road surface point cloud onto the corresponding road image to obtain the projected point cloud in the road image; The first filtering unit is used to filter invalid three-dimensional road points in the second three-dimensional road point cloud under the camera coordinate system, wherein the invalid three-dimensional road points include three-dimensional road points whose elevation is outside the target elevation range and three-dimensional road points whose forward coordinate is less than or equal to 0. The projection unit is used to project the filtered second three-dimensional road surface point cloud in the camera coordinate system onto the corresponding road image to obtain the projected point cloud in the road image.
8. The apparatus according to claim 7, characterized in that, The device further includes: The second filtering unit is used to filter invalid projection points in the projection point cloud before coloring the target three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the target projection point with the target color to highlight the road edge of the three-dimensional road surface. The invalid projection points include projection points located outside the image coordinate system, projection points located below the hood line of the acquisition vehicle, projection points located above the static road edge line, and projection points located above the dynamic road edge line.
9. The apparatus according to claim 8, characterized in that, The coloring unit is used to determine the three-dimensional road points in the first three-dimensional road point cloud that meet the preset conditions as the target three-dimensional road points, and to color the target three-dimensional road points with the target color. The preset conditions include any one of the following: The three-dimensional road surface points in the first three-dimensional road surface point cloud that have a direct mapping relationship with the target projection point; The three-dimensional road surface points obtained after raising the elevation of the three-dimensional road surface points with direct mapping relationship by the preset height; The three-dimensional road surface points with direct mapping relationship are filtered based on the drivable area point cloud. The drivable area point cloud includes the three-dimensional road surface points in the first three-dimensional road surface point cloud corresponding to the projection points in the area enclosed by the hood line, the static road edge line and the dynamic road edge line in the filtered projection point cloud. After filtering the three-dimensional road surface points with direct mapping relationships based on the drivable area point cloud, the elevation of the remaining three-dimensional road surface points is increased by the preset height to obtain the three-dimensional road surface points.
10. The apparatus according to claim 8 or 9, characterized in that, The device further includes: The calculation unit is used to calculate the weight value corresponding to each three-dimensional road surface point to be processed in the second three-dimensional road surface point cloud under the camera coordinate system after filtering out invalid three-dimensional road surface points in the second three-dimensional road surface point cloud under the camera coordinate system, based on the target distance of each three-dimensional road surface point to be processed in the filtered second three-dimensional road surface point cloud under the camera coordinate system, wherein the target distance is the target distance between the lateral coordinate and the forward coordinate of the three-dimensional road surface point to be processed, and the target distance is negatively correlated with the weight value; The coloring unit is further configured to, after filtering invalid projection points in the projection point cloud, color the three-dimensional road surface point in the first three-dimensional road surface point cloud corresponding to the valid projection point as the color value of the valid projection point when there is only one valid projection point corresponding to a three-dimensional road surface point in the first three-dimensional road surface point cloud, wherein the valid projection point is a projection point included in the filtered projection point cloud; and when there are multiple valid projection points corresponding to the same three-dimensional road surface point in the first three-dimensional road surface point cloud, perform a weighted calculation on the color value and the weight value of the multiple valid projection points, and color the three-dimensional road surface point corresponding to the multiple valid projection points as the weighted color value, wherein each of the multiple valid projection points is located in a different road image.
11. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.
12. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the electronic device performs the method as described in any one of claims 1-5.
13. A vehicle, characterized in that, The vehicle includes the device according to any one of claims 6-10, or includes the electronic device according to claim 12.