High-precision map rendering method, device and equipment and storage medium

Abstract

The invention provides a high-precision map rendering method and device, equipment and a storage medium, and relates to the technical field of computers, in particular to the technical fields of automatic driving, computer vision, high-precision maps and the like. According to the specific implementation scheme, a camera observation space is determined based on at least one of a main vehicle position, camera parameters, preset parameters and road parameters; and determining at least one road element bounding box; determining at least one target bounding box from the at least one road element bounding box; adjusting the camera based on the position relationship between the camera observation space and the at least one target bounding box; and performing high-precision map rendering based on an image acquired by the adjusted camera. According to the invention, the use experience of the user on the high-precision map can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of autonomous driving, computer vision, and high-precision mapping. Background Technology

[0002] With the development of autonomous driving technology, the demand for displaying driving scenarios is increasing. High-precision maps, relying on centimeter-level positioning accuracy and rich road semantic information, can accurately replicate the three-dimensional structure and dynamic elements of the physical world, constructing a digital simulation environment for displaying driving scenarios. However, when the vehicle is in a complex multi-layered road structure area, the upper road can obscure information about the vehicle and its surrounding environment below, preventing this information from being displayed in the high-precision map. Therefore, how to display relevant information about the vehicle and its surrounding environment in the lower road layer in real time and accurately, in order to improve the user experience of high-precision maps, has become an urgent problem to be solved. Summary of the Invention

[0003] This disclosure provides a high-precision map rendering method, apparatus, device, and storage medium.

[0004] According to one aspect of this disclosure, a high-precision map rendering method is provided, comprising:

[0005] The camera observation space is determined based on at least one of the following: the vehicle position, camera parameters, preset parameters, and road parameters; and at least one road element bounding box is determined.

[0006] Determine at least one target bounding box from at least one road element bounding box;

[0007] The camera is adjusted based on the positional relationship between the camera's observation space and at least one target bounding box;

[0008] High-precision map rendering is performed based on images captured by the adjusted camera.

[0009] According to another aspect of this disclosure, a high-precision map rendering apparatus is provided, comprising:

[0010] The first determining module is used to determine the camera observation space based on at least one of the following: the main vehicle position, camera parameters, preset parameters, and road parameters; and to determine at least one road element bounding box.

[0011] The second determining module is used to determine at least one target bounding box from at least one road element bounding box;

[0012] A camera adjustment module is used to adjust the camera based on the positional relationship between the camera's observation space and at least one target bounding box;

[0013] The map rendering module is used to render high-precision maps based on images captured by the adjusted camera.

[0014] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0015] At least one processor; and

[0016] The memory is communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0018] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0019] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0020] This disclosure detects the positional relationship between the camera's observation space and the target bounding box, and then adjusts the camera to reduce the obstruction of the camera's field of view by the upper road. Based on this, images are acquired using the adjusted camera and rendered into a high-definition map. This allows for the mapping of clear and minimally disturbed road scene information onto the high-definition map, thereby improving the rendering quality. This provides reliable environmental perception support for autonomous driving systems, enabling users to intuitively and accurately grasp the current road conditions using high-definition maps, thus enhancing the user experience of using high-definition maps.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0022] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0023] Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this disclosure;

[0024] Figure 2 This is a flowchart illustrating the implementation of a high-precision map rendering method according to an embodiment of the present disclosure;

[0025] Figure 3This is a schematic diagram of a camera adjustment process according to an embodiment of the present disclosure;

[0026] Figure 4A This is a top view of the camera observation space according to an embodiment of the present disclosure;

[0027] Figure 4B This is a side view of the camera observation space according to an embodiment of the present disclosure;

[0028] Figure 5 This is a schematic diagram illustrating the determination of a target height according to an embodiment of the present disclosure;

[0029] Figure 6 This is a schematic diagram of the structure of a high-precision map rendering apparatus 600 according to an embodiment of the present disclosure;

[0030] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0031] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0032] The term "and / or" in this disclosure indicates that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The term "at least one" in this document means any combination of at least two of a plurality of options, such as including at least one of A, B, and C, which can mean including any one or more elements selected from the set of A, B, and C. The terms "first" and "second" in this document refer to and distinguish multiple similar technical terms, and do not imply a specific order or a limitation to only two. For example, "first feature" and "second feature" refer to two types / two features; the first feature can be one or more, and the second feature can also be one or more.

[0033] When demonstrating autonomous vehicle driving scenarios, high-definition map rendering can often be used to recreate the realistic physical world. However, when the vehicle travels through complex multi-layered road structures, the upper layers of the road can obscure dynamic environmental information such as the main vehicle and obstacles below. This can make users feel uneasy, thus affecting their experience using high-definition maps.

[0034] Currently, there are two main solutions to this problem:

[0035] (1) Display the main vehicle and obstacle vehicles and other dynamic elements at the top of the multi-level road at all times, and use the blurring effect to indicate that these vehicles are actually located on the lower level road, so that users can understand the spatial relationship in the scene.

[0036] The solution performs reasonably well when dealing with roads traveling in the opposite direction to the driver's current lane. However, when the obscured road is traveling in the same direction as the driver's current lane, the upper and lower road elements remain superimposed for an extended period, causing them to blend together and resulting in poor visual quality. This negatively impacts the user's accurate perception of the driver's actual driving environment.

[0037] (2) The upper road that obscures dynamic elements such as the main vehicle will be semi-transparent, so that the dynamic elements below that were originally obscured can be seen through, allowing users to see these dynamic elements clearly.

[0038] This solution presents a dilemma regarding transparency settings. If the transparency is not low enough, overlapping road elements will appear blurry, resulting in a poor overall visual effect and hindering the user's accurate judgment of the road scene. If the transparency is set low enough, while it can improve the visual clutter to some extent, structural information of the upper road will be lost, making it difficult to clearly present complex elevated structures and hindering the expression of complex elevated scene.

[0039] To address the aforementioned issues, this disclosure proposes a high-precision map rendering method. This method adaptively adjusts the camera's shooting perspective to resolve the problem of upper-level roads obscuring dynamic environmental information in high-precision maps. Specifically, this method detects the camera's observation space and target bounding boxes to smoothly adjust the camera to a suitable position and angle, thereby reducing the obstruction range of upper-level roads on the camera's field of view. This method enables the camera's field of view to focus on the driving scene of the main vehicle, reducing the problem of information mixing between upper and lower road layers due to prolonged overlap, which negatively impacts the user's perception of the driving environment. Simultaneously, this method, through smooth camera movement, helps users anticipate the structure of the road ahead, improving the user experience of high-precision maps.

[0040] Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this disclosure, such as... Figure 1As shown, the application scenario diagram of this disclosure embodiment may include, but is not limited to, an image acquisition device 110 and a map rendering device 120. The image acquisition device 110 and the map rendering device 120 can communicate with each other through any type of wired or wireless network. Specifically, the image acquisition device 110 can be used to acquire images of the environment around the host vehicle. The image acquisition device 110 can be installed in the host vehicle, such as an autonomous vehicle. The map rendering device 120 can be used to receive the images and render them to obtain a high-precision map corresponding to the environment around the host vehicle. Furthermore, this disclosure embodiment does not impose a specific limitation on the number of image acquisition devices 110. For example, the application scenario diagram of this disclosure embodiment may include one or more image acquisition devices 110.

[0041] Figure 2 This is a flowchart illustrating the implementation of a high-precision map rendering method according to an embodiment of the present disclosure, including:

[0042] S210. Based on at least one of the following: the main vehicle position, camera parameters, preset parameters, and road parameters, determine the camera observation space; and determine at least one road element bounding box.

[0043] S220. Determine at least one target bounding box from at least one road element bounding box;

[0044] S230. Adjust the camera based on the positional relationship between the camera's observation space and at least one target bounding box;

[0045] S240: Based on the images acquired by the adjusted camera, perform high-precision map rendering.

[0046] In this embodiment of the disclosure, the camera observation space may include the three-dimensional spatial range of the surrounding environment that the camera can capture. This camera observation space may be determined based on at least one of the following: the vehicle's position, camera parameters, preset parameters, and road parameters.

[0047] In this embodiment of the disclosure, the method for determining at least one road element bounding box may include:

[0048] Get the geometric shape points corresponding to the road element;

[0049] Based on geometric points, the bounding box algorithm is used to determine the bounding box of at least one road element.

[0050] In one example, a camera mounted on the main vehicle can be used to capture images containing road elements in real time. Through image processing algorithms, such as edge detection and corner detection, feature points of the road elements can be extracted from the images. These feature points can be used as geometric shape points corresponding to the road elements.

[0051] In another example, a LiDAR installed on the main vehicle can be used to acquire three-dimensional point cloud data of road elements. Each point in the point cloud data contains the spatial coordinate information of the road element and can be directly used as the geometric shape point of the road element.

[0052] In another example, image data captured by a camera and point cloud data captured by a LiDAR can be fused together. The color and texture information of the image can be used to assist in the segmentation and feature extraction of the point cloud data, thereby obtaining more accurate geometric shape points of road elements.

[0053] Here, road elements can include lanes, overpasses, and other roads that allow vehicles to travel.

[0054] Furthermore, based on the geometric shape points, bounding box algorithms can be used to determine the bounding box of at least one road element corresponding to the geometric shape points. Here, bounding box algorithms can include the Axis-Aligned Bounding Box (AABB) algorithm and the Oriented Bounding Box (OBB) algorithm.

[0055] In one example, the AABB algorithm can traverse the coordinates of the points of the geometric shape, find the maximum (e.g., x_max, y_max, and z_max) and minimum (e.g., x_min, y_min, and z_min) values ​​on the x-axis, y-axis, and z-axis, respectively, and then construct diagonal vertices (e.g., (x_max, y_max, z_max) and (x_min, y_min, z_min)) using the maximum and minimum values ​​on the x-axis, y-axis, and z-axis to generate a cuboid bounding box, i.e., a road element bounding box.

[0056] In another example, the OBB algorithm obtains eigenvectors and eigenvalues ​​by performing Principal Component Analysis (PCA) on the geometric points. Further, the principal orientation of the road elements is determined based on the eigenvectors, and a cuboid bounding box that tightly encloses the road elements, i.e., the road element bounding box, is generated based on this principal orientation.

[0057] Further, from the determined at least one road element bounding box, at least one target bounding box is determined. In other words, this step requires filtering out target bounding boxes that meet preset conditions from at least one road element bounding box.

[0058] In this embodiment of the disclosure, the camera can be adjusted based on the positional relationship between the camera's observation space and the target bounding box. Here, the positional relationship between the camera's observation space and the target bounding box can be an intersection relationship, and the camera can be adjusted by adjusting its position and / or angle, etc.

[0059] Furthermore, high-precision map rendering is performed based on the adjusted images captured by the camera. Specifically, the images can be preprocessed, including image scaling and normalization, to make the images meet the input requirements for subsequent rendering.

[0060] Next, feature extraction algorithms can be used to extract key feature points from the image and match these feature points with an existing high-precision (or initial high-precision) map. These key feature points represent road features in the image. Through feature matching, the position and orientation of the image in the existing high-precision map are determined, providing localization information for subsequent high-precision map rendering. Here, feature extraction algorithms can include Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), etc.

[0061] Furthermore, deep learning models can be used to identify and classify road features in images, such as determining the types of elements like lane lines and traffic signs. Then, based on the image's localization information and camera parameters (including intrinsic and extrinsic parameters), the coordinates of the road features in the image in the real world can be calculated.

[0062] Finally, based on the identified and located features, a high-precision map rendering engine (such as Cesium, OpenSceneGraph, etc.) is used to draw the road features onto a high-precision map. In one example, for an existing high-precision map, new road features can be fused and updated with existing road features.

[0063] By employing the aforementioned method, the positional relationship between the camera's observation space and the target bounding box is detected, and the camera is then adjusted to reduce the obstruction of the camera's field of view by the upper road. Based on this, images are acquired using the adjusted camera and rendered into a high-definition map, mapping the current clear and minimally disturbed road scene information onto the high-definition map. This provides reliable environmental perception support for autonomous driving systems, enabling users to intuitively and accurately grasp the current road conditions using high-definition maps, thereby improving the user experience of using high-definition maps.

[0064] Figure 3 This is a schematic diagram of a camera adjustment process according to an embodiment of the present disclosure. Figure 3 As shown, the camera adjustment process includes the following steps:

[0065] S301, Obtain the location of the main vehicle.

[0066] In one example, the location of the main vehicle in the initial high-precision map can be obtained by combining the road topology, geographic coordinate information, and preset positioning matching algorithm in the initial high-precision map with real-time data collected by positioning sensors (such as Global Navigation Satellite System, GNSS) on the main vehicle, through data fusion and comparison analysis.

[0067] S302. Determine the camera observation space.

[0068] In some implementations, camera parameters include initial camera height and lower limit camera height; preset parameters include camera movement time; road parameters include lane width and speed limit.

[0069] The camera observation space is determined based on at least one of the following: vehicle position, camera parameters, preset parameters, and road parameters, including:

[0070] Determine the lateral distance based on the lane width;

[0071] Determine the forward distance based on the speed limit and camera movement time;

[0072] Determine the longitudinal distance based on the camera's initial height and lower limit height.

[0073] The camera observation space is determined based on the lateral distance, forward distance, and longitudinal distance.

[0074] In this embodiment of the disclosure, the initial camera height can be the vertical distance from the center of the camera lens to a reference plane (such as the ground) when the camera starts working or is in its default state. In one example, the initial camera height can be a base value set when the camera is installed, which is related to the camera's installation location and method. For example, in an autonomous vehicle, the camera may be installed above the roof, in which case the initial camera height can be the vertical distance from the ground to the center of the camera lens.

[0075] The lower limit height of a camera can be defined as the lowest vertical position that a camera is allowed to reach during operation, which is the minimum vertical distance between the center of the camera lens and the reference plane. By setting the lower limit height, the lower limit of the camera's shooting angle can be restricted, enabling the camera to acquire effective information within a certain height range and maintaining the stability and reliability of image acquisition.

[0076] In this embodiment of the disclosure, the camera movement time can be the time required for the camera to complete a specific movement (such as adjusting position or angle), and this time can be preset according to actual needs and system performance.

[0077] In this embodiment of the disclosure, lane width can be the horizontal distance between two adjacent lane lines on a road. Lane width is typically determined based on factors such as road grade, traffic flow, and intended use.

[0078] Speed ​​limits are the maximum speed vehicles are allowed to travel on a specific lane. Speed ​​limits are typically set based on factors such as lane design standards, traffic conditions, and the surrounding environment.

[0079] In this embodiment of the disclosure, the camera observation space can be defined as a cuboid, which includes width, length and height, corresponding to the lateral distance, forward distance and longitudinal distance of the camera observation space.

[0080] Specifically, the lateral distance of the camera's observation space can be the width of the lane.

[0081] The forward distance of the camera's observation space can be the product of the limiting speed and the camera's movement time. To further improve the safety and stability of the system, a preset forward offset is also required. This preset forward offset can be reasonably set based on factors such as actual road conditions and vehicle movement.

[0082] The longitudinal distance of the camera observation space can be the difference between the initial height of the camera and the lower limit height of the camera. In addition, to compensate for the uncertainties caused by environmental factors and measurement errors, a preset longitudinal offset needs to be added. This preset longitudinal offset needs to be analyzed and calibrated according to the actual application scenario.

[0083] In one example, the lane width and speed limit are preset.

[0084] In this example, lane width and speed limit can be preset based on statistical analysis of a large number of regular road traffic conditions. Specifically, the lane width setting comprehensively considers common design standards for different levels of roads (such as highways, urban arterial roads, and secondary roads), as well as the lateral safety clearance requirements for vehicles. The speed limit can be determined based on factors such as the road's functional positioning, traffic flow, and surrounding environment, with reference to vehicle speed limit standards.

[0085] In this way, based on the pre-set lane width, the lateral distance of the camera observation space can be determined; at the same time, combined with the pre-set speed limit and camera movement time, the forward distance can be further determined, thus providing a data basis for the subsequent construction of the camera observation space.

[0086] In another example, determining lane width and speed limit includes:

[0087] Based on the location of the main vehicle, the lane width and speed limit are determined using an initial high-precision map; the lane width and speed limit are related to the road where the main vehicle is located.

[0088] In this example, the vehicle's position information can be obtained in real time through the fusion processing of multiple sensors on the vehicle. These sensors continuously collect the vehicle's motion state and environmental data, and transmit them to the vehicle's central processing unit for calculation and analysis, ultimately obtaining the vehicle's accurate position coordinates at the current moment.

[0089] After obtaining the location of the main vehicle, it is matched with the initial high-precision map. Specifically, the approximate area where the main vehicle is located can be determined first, and then a detailed search can be performed within that approximate area to find the road where the main vehicle is located. Once the road where the main vehicle is located is determined, relevant information about that road, including lane width and speed limit, can be retrieved from the database of the initial high-precision map.

[0090] In this way, by using the initial high-precision map based on the location of the main vehicle, the accuracy of determining lane width and speed limit can be improved, thereby improving the accuracy of determining lateral and longitudinal distances, and providing an accurate data foundation for the subsequent determination of camera observation space.

[0091] Furthermore, based on the lateral distance, forward distance, and longitudinal distance obtained above, the camera observation space is determined.

[0092] Using the above method, the camera observation space, determined by combining lateral, forward, and longitudinal distances, provides the main vehicle with a three-dimensional environmental perception range. Based on the camera observation space, the positional relationship with the target bounding box can be determined, thus providing data support for camera adjustment strategies.

[0093] S303, Determine the target bounding box.

[0094] In some implementations, determining at least one target bounding box from at least one road element bounding box includes:

[0095] The filtering range is determined based on the location of the main vehicle. Any point within the filtering range is less than or equal to the distance from the location of the main vehicle.

[0096] The bounding box of at least one road element that overlaps with the filtering range is determined as the target bounding box.

[0097] In this embodiment, the preset threshold can be a distance value pre-set according to actual needs and application scenarios. This preset threshold needs to comprehensively consider various factors, such as the vehicle's speed, road type, and traffic conditions. For example, on highways, due to higher speeds, the preset threshold can be set relatively large to allow the vehicle sufficient reaction time and safe distance; while in congested urban areas, where speeds are slower, the preset threshold can be set relatively small.

[0098] In one example, a hemispherical region is defined in three-dimensional space with the location of the main vehicle as the center point and a preset threshold as the radius. This region can be considered as the filtering range. The distance between any point within this filtering range and the location of the main vehicle will not exceed the preset threshold.

[0099] In this embodiment of the disclosure, the spatial relationship between the bounding box of each road element and the filtering range is determined to check whether there is an overlap between the bounding box of the road element and the filtering range. In one example, if the bounding box of the road element and the filtering range have at least one common point, it can be considered that there is an overlap.

[0100] Furthermore, all road element bounding boxes that overlap with the filtering range are filtered out, and these road element bounding boxes are identified as target bounding boxes.

[0101] Using the above method, the filtering range is determined based on the location of the main vehicle and a preset threshold, and at least one target bounding box is determined from at least one road element bounding box based on the filtering range. In this way, the camera observation space only needs to determine the positional relationship with the target bounding box, without having to make invalid judgments on road element bounding boxes far away from the main vehicle, thus narrowing the judgment range and improving the judgment efficiency.

[0102] S304. Determine whether the camera observation space and the target bounding box intersect. If the camera observation space and the target bounding box intersect, execute S305; if the camera observation space and the target bounding box do not intersect, execute S301 to reacquire the master vehicle position.

[0103] S305. Determine the target position and / or target angle.

[0104] S306. Adjust the camera based on the target position and / or target angle.

[0105] After adjusting the camera, the position of the main vehicle can be obtained again to verify whether the adjusted camera will be obstructed by the upper road (i.e., to verify whether the camera's observation space intersects with the target bounding box).

[0106] In some implementations, the camera is adjusted based on the positional relationship between the camera's observation space and at least one target bounding box, including:

[0107] Determine the target position and / or target angle when the camera's observation space intersects with the bounding box of any target;

[0108] Adjust the camera based on the target position and / or target angle.

[0109] In one example, the Separating Axis Theorem (SAT) can be used to detect whether the camera's observation space and the target's bounding box intersect.

[0110] Specifically, for the camera observation space and the target bounding box, the separation axes that need to be checked include the normal axes corresponding to the camera observation space and the target bounding box respectively (there are 6 different faces, so there are 6 normal axes in total), as well as the axes obtained by the cross product of the edge vectors between the camera observation space and the target bounding box (the camera observation space and the target bounding box each have 3 different edge vectors, and the 3 edge vectors in the camera observation space are cross-producted with the 3 edge vectors in the target bounding box respectively, so there are 9 axes in total).

[0111] Furthermore, the camera observation space and the target bounding box are projected onto each separation axis, and the projection intervals are checked for overlap. If the projection intervals of the camera observation space and the target bounding box overlap on all separation axes, then the camera observation space and the target bounding box intersect; otherwise, the camera observation space and the target bounding box do not intersect.

[0112] Determine the target position and / or target angle when the camera's observation space intersects with the bounding box of any target.

[0113] In some implementations, determining the target position and / or target angle includes:

[0114] The minimum occlusion height is determined based on the target bounding box that intersects with the camera's observation space;

[0115] Determine the target position and / or target angle based on the camera's lower limit height and minimum obstruction height.

[0116] In one example, the minimum occlusion height is determined based on the target bounding box intersecting the camera's observation space, including:

[0117] Determine the lowest point of the intersection between the camera's observation space and the target's bounding box;

[0118] The height of the lowest point is determined as the minimum shading height.

[0119] In this example, after determining that the camera's view space intersects with the target bounding box, it is necessary to find the intersection points between the camera's view space and the target bounding box. It is understandable that if the target bounding box is completely embedded in the camera's view space, then all vertices of the target bounding box can be considered as intersection points with the camera's view space.

[0120] Furthermore, after obtaining the intersection points, by comparing the coordinate values ​​of these intersection points in the vertical direction, the intersection point with the smallest vertical coordinate is selected as the lowest point of the intersection. For example, if the coordinates of the intersection points between the camera observation space and the target bounding box are (x1, y1, z1), (x2, y2, z2), ..., (xn, yn, zn), the minimum value, such as zmin, is determined by comparing the values ​​of z1, z2, ..., zn. The intersection point corresponding to this minimum value can be considered the lowest point.

[0121] Furthermore, the lowest point can be directly determined as the lowest occlusion height. For example, if the coordinates of the lowest point are (xmin, ymin, zmin), then zmin can be considered the lowest occlusion height.

[0122] By using this method, the lowest point of the intersection between the camera's observation space and the target's bounding box is determined, and the height corresponding to the lowest point is determined as the minimum occlusion height, providing data reference for determining the target's position and / or target angle.

[0123] In some implementations, the target position and / or target angle are determined based on the camera's lower limit height and minimum obstruction height, including:

[0124] The target height is determined by comparing the lower limit of the camera height with the lowest occlusion height.

[0125] Based on the target height, determine the target position and / or target angle.

[0126] In this embodiment of the disclosure, the target height is determined based on a comparison between the camera's lower limit height and the lowest occlusion height. Further, the target position and / or target angle are determined based on the target height.

[0127] Here, the target location can be the location where the camera captures the best image; that is, at this target location, the road surrounding the vehicle has the least impact on the camera's field of view. In one example, the location corresponding to the target height can be determined as the target location.

[0128] The target angle can be the angle between the camera's optical axis and the horizontal plane (or the pitch angle). When the target height changes, adjusting the pitch angle can reduce the problem of the captured image being blocked due to the viewing angle.

[0129] In one example, if the target height is h, the horizontal distance between the camera and the target to be acquired is d, and the initial camera height is H, the pitch angle θ can be calculated using trigonometric relationships. The formula for calculating the pitch angle θ is:

[0130]

[0131] By using the above method, the target position and / or target angle can be determined based on the minimum occlusion angle and the lower limit height of the camera, which can reduce the occlusion effect of road elements on the camera and improve the effectiveness and quality of the images acquired by the camera.

[0132] In some implementations, the lower limit height of the camera and the lowest occlusion height are compared to determine the target height, including:

[0133] Determine the maximum value between the lower limit of camera height and the minimum occlusion height;

[0134] The height corresponding to the maximum value is determined as the target height.

[0135] In this embodiment of the disclosure, the maximum value between the lower limit height of the camera and the lowest occlusion height is determined by comparing the two. Further, after determining the maximum value of the lower limit height of the camera and the lowest occlusion height, the height corresponding to this maximum value is determined as the target height.

[0136] By using the above method, the target height is determined by the maximum value between the lower limit height of the camera and the lowest occlusion height. This can provide a reference for camera adjustment strategies and reduce the chances of the camera being obstructed.

[0137] In the embodiments disclosed herein, it should be noted that the camera adjustment strategy can be implemented based on the target position, or based on the target angle, or both the target position and the target angle.

[0138] In one example, if the road elements still obstruct the camera's field of view after the camera is adjusted to a target position corresponding to the camera's lower limit height, then it is necessary to calculate the target angle that will not obstruct the camera's field of view, and then adjust the camera according to the target angle.

[0139] By adopting the above method, after determining the appropriate target location and / or target angle, the camera can be adjusted to a suitable position and / or attitude. This reduces the interference of road elements on camera observation, thereby obtaining clear and accurate road condition information and surrounding environment information, providing reliable environmental perception support for the autonomous driving system.

[0140] like Figure 3 As shown, it is understandable that if the main vehicle is in the starting state, steps S301 to S306 need to be executed repeatedly (i.e., Figure 3 (Steps within the dashed box); If the main vehicle is stopped (or not started), the camera adjustment process ends.

[0141] In some implementations, adjusting the camera based on the camera observation space and at least one target bounding box further includes:

[0142] If the camera's observation space does not intersect with any target bounding box, adjust the camera's position and / or angle to the default state.

[0143] In this embodiment, the default state can refer to a preset default state. During the observation of a road scene using a camera, when the camera does not detect any road elements, the camera's observation space and the target bounding box will not intersect. This means that no road elements are currently within the camera's field of view, and therefore will not interfere with the camera's observation view. Based on this, the camera can automatically return to a preset position and / or angle, which can be considered the camera's default state.

[0144] By adopting this approach, since the default state is a relatively stable state, the possibility of interference from false detection signals can be reduced when the camera is in the default state, thereby reducing unnecessary adjustments to the camera and improving the stability of camera observation.

[0145] Figure 4A This is a top view of the camera observation space according to an embodiment of the present disclosure.

[0146] like Figure 4A As shown, the camera can be located behind the main vehicle, and the camera's observation space can be determined based on the lane width. Figure 4A The lateral distance (x) of the area within the dashed box shown; the forward distance (y) of the camera's observation space can be determined based on the limited speed and camera movement time. A specific point can be determined along the camera's line of sight, using the camera's focal length, angle of view, and other parameters, based on the camera's position. This point can be considered as the location from which image information is obtained in three-dimensional space, i.e., the viewpoint position.

[0147] Figure 4B This is a side view of the camera observation space according to an embodiment of the present disclosure.

[0148] like Figure 4B As shown, based on the camera's initial height and lower limit height, the camera's observation space can be determined. Figure 4B The vertical distance (z) of the area within the dashed box shown.

[0149] In one example, the center point of the camera's observation space is determined by calculating half of the lateral distance, forward distance, and longitudinal distance (i.e., lateral distance / 2, forward distance / 2, and longitudinal distance / 2), respectively.

[0150] Since the area for determining the positional relationship between the camera's observation space and the target bounding box is in front of the camera, the center point of the camera's observation space is not at the camera's position, but rather the camera's position is offset by a distance along the line of sight, placing the camera in the middle to rear section of the camera's observation space.

[0151] Figure 5 This is a schematic diagram of determining the target height according to an embodiment of the present disclosure.

[0152] like Figure 5 As shown, there is a camera observation area ( Figure 5 The dashed box shown) and the target bounding box ( Figure 5 The intersecting portion of the solid-lined box shown. Further, based on the intersecting portion, the minimum occlusion height is determined ( Figure 5 (The height of the dotted line shown). Also... Figure 5 The document also demonstrates the lower limit height of the camera. By determining the maximum value between the lower limit height of the camera and the lowest occlusion height, the height corresponding to the maximum value is determined as the target height.

[0153] Furthermore, based on the target height, the camera is adjusted to the target position and / or target angle through camera movement, for example... Figure 5 The state of the dashed line camera is adjusted to that of the solid line camera.

[0154] It should be noted that, Figure 4A , Figure 4B and Figure 5 The positional relationship between the camera and the host vehicle is shown schematically. In implementing this disclosure, the camera needs to be mounted on the host vehicle. Generally, the camera can be mounted directly behind the host vehicle. However, when the host vehicle is turning or at certain specific angles, a corresponding offset will be set for the camera. For example, when the host vehicle is moving forward, the camera is located directly behind the host vehicle. In special situations such as turning, the camera may observe from the left or right side of the host vehicle. In this case, the camera position will be offset to ensure that the forward direction of the camera's observation space is consistent with the direction of the camera's line of sight.

[0155] This disclosure proposes a method to address the problem of upper-level roads obstructing the camera's view. This method utilizes camera movement to ensure that the images captured by the camera are not obstructed by the upper-level road, thus enabling high-precision map rendering. This allows users to clearly observe the environment around the vehicle while also fully presenting the structure of the elevated road, thereby improving the display effect of the driving scene and the user experience of the high-precision map.

[0156] This disclosure also proposes a high-precision map rendering device. Figure 6 This is a schematic diagram of the structure of a high-precision map rendering apparatus 600 according to an embodiment of the present disclosure, comprising:

[0157] The first determining module 610 is used to determine the camera observation space based on at least one of the following: the main vehicle position, camera parameters, preset parameters, and road parameters; and to determine at least one road element bounding box.

[0158] The second determining module 620 is used to determine at least one target bounding box from at least one road element bounding box;

[0159] The camera adjustment module 630 is used to adjust the camera based on the positional relationship between the camera's observation space and at least one target bounding box;

[0160] The map rendering module 640 is used for high-precision map rendering based on images acquired by the adjusted camera.

[0161] In some implementations, camera parameters include initial camera height and lower limit camera height; preset parameters include camera movement time; road parameters include lane width and speed limit.

[0162] The first determining module 610 is used for:

[0163] Determine the lateral distance based on the lane width;

[0164] Determine the forward distance based on the speed limit and camera movement time;

[0165] Determine the longitudinal distance based on the camera's initial height and lower limit height.

[0166] The camera observation space is determined based on the lateral distance, forward distance, and longitudinal distance.

[0167] In some implementations, the lane width and speed limit are preset.

[0168] In some implementations, the first determining module 610 is used for:

[0169] Based on the location of the main vehicle, the lane width and speed limit are determined using an initial high-precision map; wherein, the lane width and speed limit are associated with the road where the main vehicle is located.

[0170] In some implementations, the second determining module 620 is used for:

[0171] The filtering range is determined based on the location of the main vehicle. Any point within the filtering range is less than or equal to the distance from the location of the main vehicle.

[0172] The bounding box of at least one road element that overlaps with the filtering range is determined as the target bounding box.

[0173] In some implementations, the camera adjustment module 630 is used for:

[0174] Determine the target position and / or target angle when the camera's observation space intersects with the bounding box of any target;

[0175] Adjust the camera based on the target position and / or target angle.

[0176] In some implementations, the camera adjustment module 630 is used for:

[0177] The minimum occlusion height is determined based on the target bounding box that intersects with the camera's observation space;

[0178] Determine the target position and / or target angle based on the camera's lower limit height and minimum obstruction height.

[0179] In some implementations, the camera adjustment module 630 is used for:

[0180] The target height is determined by comparing the lower limit of the camera height with the lowest occlusion height.

[0181] Based on the target height, determine the target position and / or target angle.

[0182] In some implementations, the camera adjustment module 630 is used for:

[0183] Determine the lowest point of the intersection between the camera's observation space and the target's bounding box;

[0184] The height of the lowest point is determined as the minimum shading height.

[0185] In some implementations, the camera adjustment module 630 is used for:

[0186] Determine the maximum value between the lower limit of camera height and the minimum occlusion height;

[0187] The height corresponding to this maximum value is determined as the target height.

[0188] In some embodiments, the camera adjustment module 630 is also used for:

[0189] If the camera's observation space does not intersect with any target bounding box, adjust the camera's position and / or angle to the default state.

[0190] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0191] The acquisition, storage, and application of personal information by users involved in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0192] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0193] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0194] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0195] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0196] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as high-precision map rendering methods. For example, in some embodiments, the high-precision map rendering method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the high-precision map rendering method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the high-precision map rendering method by any other suitable means (e.g., by means of firmware).

[0197] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0198] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0199] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0200] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0201] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0202] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0203] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0204] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A high-precision map rendering method, comprising: The camera observation space is determined based on at least one of the following: vehicle position, camera parameters, preset parameters, and road parameters. Furthermore, determine the bounding box of at least one road element; Determine at least one target bounding box from the bounding boxes of the at least one road element; The camera is adjusted based on the positional relationship between the camera's observation space and the at least one target bounding box; High-precision map rendering is performed based on the images captured by the adjusted camera.

2. The method according to claim 1, wherein, The camera parameters include the initial camera height and the lower limit of the camera height; the preset parameters include the camera movement time; the road parameters include the lane width and the speed limit. The process of determining the camera observation space based on at least one of the following: vehicle position, camera parameters, preset parameters, and road parameters, includes: Based on the lane width, determine the lateral distance; The forward distance is determined based on the speed limit and the camera movement time. The longitudinal distance is determined based on the initial height of the camera and the lower limit height of the camera; The camera observation space is determined based on the lateral distance, the forward distance, and the longitudinal distance.

3. The method according to claim 2, wherein, The lane width and the speed limit are preset.

4. The method according to claim 2, wherein, Determining the lane width and the speed limit includes: Based on the location of the main vehicle, the lane width and the speed limit are determined using an initial high-precision map; wherein the lane width and the speed limit are associated with the road where the main vehicle is located.

5. The method according to claim 4, wherein, Determining at least one target bounding box from the at least one road element bounding box includes: The filtering range is determined based on the location of the main vehicle, and the distance between any point within the filtering range and the location of the main vehicle is less than or equal to a preset threshold. The bounding boxes of at least one road element that overlap with the filtering range are determined as the target bounding box.

6. The method according to any one of claims 2-5, wherein, The adjustment of the camera based on the positional relationship between the camera's observation space and the at least one target bounding box includes: When the camera observation space intersects with any of the target bounding boxes, the target position and / or target angle are determined; The camera is adjusted based on the target position and / or the target angle.

7. The method according to claim 6, wherein, Determining the target position and / or target angle includes: The minimum occlusion height is determined based on the target bounding box that intersects with the camera's observation space; The target position and / or the target angle are determined based on the lower limit height of the camera and the minimum occlusion height.

8. The method according to claim 7, wherein, Determining the target position and / or the target angle based on the camera's lower limit height and the lowest occlusion height includes: The target height is determined by comparing the lower limit height of the camera with the lowest occlusion height. Based on the target height, determine the target position and / or the target angle.

9. The method according to claim 7, wherein, Determining the minimum occlusion height based on the target bounding box intersecting the camera's observation space includes: Determine the lowest point of the intersection between the camera observation space and the target bounding box; The height of the lowest point is determined as the lowest occlusion height.

10. The method according to claim 8, wherein, The step of comparing the lower limit height of the camera with the lowest occlusion height to determine the target height includes: Determine the maximum value between the lower limit height of the camera and the minimum occlusion height; The height corresponding to the maximum value is determined as the target height.

11. The method according to any one of claims 1-10, wherein adjusting the camera based on the camera observation space and the at least one target bounding box further comprises: If the camera's observation space does not intersect with any of the target bounding boxes, the position and / or angle of the camera are adjusted to the default state.

12. A high-precision map rendering device, comprising: The first determining module is used to determine the camera observation space based on at least one of the following: the vehicle position, camera parameters, preset parameters, and road parameters; Furthermore, determine the bounding box of at least one road element; The second determining module is used to determine at least one target bounding box from the at least one road element bounding box; A camera adjustment module is used to adjust the camera based on the positional relationship between the camera's observation space and the at least one target bounding box; The map rendering module is used to perform high-precision map rendering based on the adjusted images captured by the camera.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-11.

15. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-11.

Images