Map construction method for vehicle auxiliary driving, electronic equipment and program product

Abstract

The invention provides a map construction method for vehicle aided driving, electronic equipment and a program product, and relates to the technical field of intelligent driving. The method comprises the steps that front environment information of a driving road where a vehicle is located and surrounding environment information of the vehicle are obtained, the front road condition information is obtained through an unmanned aerial vehicle, and the surrounding environment information is obtained through a vehicle-mounted sensor; fusing the front road condition information and the surrounding environment information to construct a multi-layer rasterized map; the multi-layer rasterized map comprises a rasterized ground layer map, a rasterized barrier layer map and a rasterized high-altitude layer map. The method and the device are used for solving the problem that the accuracy of an intelligent auxiliary driving decision is not high due to insufficient map precision of existing intelligent auxiliary driving.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent driving technology, and in particular to a map construction method, electronic equipment, and program product for vehicle assisted driving. Background Art

[0002] Advanced Driver Assistance Systems (ADAS) are a collection of intelligent technologies that utilize sensors, algorithms, and control systems to monitor the vehicle's environment in real time and assist the driver in making decisions. Their core goals are to improve safety, reduce driving burden, and prevent accidents.

[0003] Existing in-vehicle assisted driving systems rely on a single sensor or a simple combination of sensors, such as a camera, or a combination of a camera and millimeter-wave radar, resulting in low accuracy in assisted driving decisions. Summary of the Invention

[0004] In view of this, the embodiments of the present disclosure provide a map construction method, electronic device and program product for vehicle assisted driving to solve the problem that the map accuracy of existing intelligent assisted driving is insufficient, resulting in low accuracy of intelligent assisted driving decisions.

[0005] In a first aspect, the present disclosure provides a map construction method for vehicle assisted driving, comprising:

[0006] Acquiring forward environmental information of the road on which the vehicle is traveling and information about the vehicle's surrounding environment, wherein the forward road condition information is obtained by a drone, and the surrounding environment information is obtained by vehicle-mounted sensors;

[0007] The forward road condition information and the surrounding environment information are integrated to construct a multi-layer rasterized map; the multi-layer rasterized map includes a rasterized ground layer map, a rasterized obstacle layer map and a rasterized high-altitude layer map.

[0008] In a second aspect, the present disclosure provides an electronic device, including:

[0009] at least one processor; and

[0010] a memory communicatively connected to the at least one processor; wherein,

[0011] The memory stores at least one computer program that can be executed by the at least one processor, and the at least one computer program is executed by the at least one processor so that the at least one processor can execute the map construction method for vehicle assisted driving as described in the first aspect.

[0012] In a third aspect, the present disclosure provides a computer program product, which includes a computer program. When the computer program is executed in a processor, it implements the map construction method for vehicle assisted driving described in the first aspect.

[0013] Optionally, the computer program may be stored in a readable storage medium of a computer device or in the cloud; the processor of the computer device reads the computer program from the readable storage medium or the cloud.

[0014] The embodiments provided by the present disclosure obtain information about the front environment of the road on which a vehicle is traveling and information about the surrounding environment of the vehicle, wherein the front road condition information is obtained by a drone, and the surrounding environment information is obtained by on-board sensors; the front road condition information and the surrounding environment information are integrated to construct a multi-layer rasterized map; the multi-layer rasterized map includes a rasterized ground layer map, a rasterized obstacle layer map, and a rasterized high-altitude layer map. By integrating the front road condition information detected by the drone and the surrounding environment information collected by on-board sensors, a multi-layer rasterized map including the ground layer, obstacle layer, and high-altitude layer is generated, thereby improving map accuracy, providing data support for intelligent assisted driving, and thus ensuring the accuracy and safety of intelligent assisted driving decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0016] Figure 1 The figure shows a schematic diagram of the architecture of the vehicle assisted driving system in an embodiment of the present disclosure;

[0017] Figure 2 FIG2 is a flow chart of a map construction method for vehicle assisted driving according to an embodiment of the present disclosure;

[0018] Figure 3 FIG2 is a block diagram of a map construction device for vehicle assisted driving according to an embodiment of the present disclosure;

[0019] Figure 4 FIG. 1 is a schematic structural diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present disclosure in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0021] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0022] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0023] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. Similar words such as "connected" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0024] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly defined as such herein.

[0025] Overview

[0026] In related technologies, in complex driving scenarios such as off-road driving and narrow roads, existing in-vehicle assisted driving systems rely on single or simply fused sensors (such as cameras and millimeter-wave radars), which present significant technical bottlenecks: they can only detect two-dimensional planar obstacles and lack the ability to assess the three-dimensional risk of ground slopes, curvatures, and high-altitude suspended obstacles. This leads to missed detection of dangerous road conditions such as cliffs, deep pits, and low-altitude height-restricted obstacles. This makes it difficult to guarantee the accuracy and safety of vehicle assisted driving.

[0027] Exemplary Systems

[0028] Based on the above analysis, the embodiment of the present disclosure provides a map construction method for vehicle assisted driving, which improves the accuracy of vehicle assisted driving decisions by improving the accuracy of the map. Figure 1 The vehicle assisted driving system shown is implemented, and the system mainly includes:

[0029] The drone module, equipped with a lidar and visual camera, is responsible for scanning the terrain within a certain distance in front of the vehicle (for example, scanning the terrain within a range of 0-50 meters in front of the vehicle), identifying obstacles, obtaining road condition data streams and obstacle information, and transmitting the road condition data streams and obstacle information to the data processing unit (DHU) in real time. In addition, the drone also transmits information such as battery level to the DHU for timely monitoring of the drone's status.

[0030] Surround view camera module. Generally, a vehicle includes four surround view cameras, which collect video streams of the vehicle's surrounding environment and transmit them to the DHU to assist in the perception of the vehicle's surrounding environment. After being integrated with the data output by the drone module, a more comprehensive understanding of the vehicle's environment can be obtained.

[0031] Corner radar module. Generally, a vehicle includes four corner radar modules, which detect obstacle information around the vehicle and transmit the obstacle information to the DHU. The obstacle information is integrated with the data output by the drone module and the surround-view camera module to improve the perception accuracy of obstacles and other targets around the vehicle.

[0032] The DHU module transmits the road condition data stream and obstacle information output by the drone module and the surrounding environment video stream output by the surround-view camera module to the Advanced Driver Assistance Data Control Unit (ADCU) for processing, and is responsible for real-time image display (or three-dimensional terrain display) and receiving ADCU feedback information for related operations.

[0033] The ADCU module integrates the multi-source data provided by the drone module, surround view camera module and corner radar module, performs map construction and risk quantification results, and feeds the constructed map and risk quantification results back to the DHU module to provide decision support for vehicle driving.

[0034] The display unit (CSD) receives the map with risk quantification results from the DHU and displays the three-dimensional environmental risk quantification results on the vehicle interface.

[0035] Exemplary Methods

[0036] The map construction method for vehicle assisted driving provided by the embodiments of the present disclosure can be applied to controllers such as ADCU and DHU of the vehicle, and can also be applied to other terminals, servers, etc. that can communicate with vehicles and drones.

[0037] The map construction method for vehicle assisted driving provided by the embodiment of the present disclosure is as follows: Figure 2 As shown, it mainly includes the following steps:

[0038] Step 201: Acquire the front environment information of the road on which the vehicle is traveling and the surrounding environment information of the vehicle. The front road condition information is obtained by a drone, and the surrounding environment information is obtained by a vehicle-mounted sensor.

[0039] In some embodiments, the surrounding environment information includes surround view image information collected by a surround view camera and the corresponding collection time, and first obstacle information collected by a corner radar and the corresponding collection time, wherein the first obstacle information includes position description information and motion description information;

[0040] The forward road condition information includes a real-time road condition data stream and second obstacle information corresponding to the real-time road condition data stream; the second obstacle information includes a two-dimensional bounding box of the obstacle, a pixel-level mask, and a corresponding acquisition time, and the two-dimensional bounding box carries a semantic label of the obstacle; the real-time road condition data stream includes three-dimensional point cloud data at each acquisition time.

[0041] In an exemplary embodiment, a drone equipped with a lidar emits pulses at a 10Hz frequency, scanning the road within a 50-meter radius in front of the vehicle and generating a real-time traffic data stream containing 3D point cloud data corresponding to each acquisition moment. The drone transmits this real-time traffic data back to the vehicle's DHU via millimeter-wave communication in real time. This 3D point cloud data includes spatial coordinates (X / Y / X coordinates) and reflection intensity, which are used to calculate terrain height, slope, and curvature.

[0042] In the exemplary embodiment, the visual camera carried by the drone is a wide-angle camera, which captures video streams at 30fps and uses an improved lightweight object detection model (YOLOv5s model) to identify obstacle types (such as rocks, vegetation, potholes, etc.) in real time. The improved YOLOv5s model uses the TensorRT acceleration engine and significantly improves the inference speed of the YOLOv5s model through optimization methods such as layer fusion and precision calibration. At the same time, the CIoU (Complete Intersection over Union) loss function is introduced to optimize the bounding box regression, as shown in formula (1):

[0043]

[0044] Among them, IoU is the intersection over union ratio, ρ 2 (b,b gt) is the square of the Euclidean distance between the center points of the predicted box and the ground-truth box, c is the diagonal length of the minimum bounding rectangle of the two boxes, v measures the difference in aspect ratio between the predicted box and the ground-truth box, and α is the weight coefficient. By optimizing the loss function, the model's detection accuracy is effectively improved.

[0045] The improved YOLOv5s model outputs a 2D bounding box and pixel-level mask of the obstacle, which carries the semantic label of the obstacle. This 2D bounding box and pixel-level mask are then synchronized with the 3D point cloud data output by the lidar using the PTP (Precision Time Protocol) clock protocol, minimizing time errors. Coordinate transformation is then used to unify the data into the vehicle coordinate system, achieving precise fusion of multi-source data and providing a data foundation for environmental modeling and path planning.

[0046] Step 202 : Fusing the road condition information ahead and the surrounding environment information to construct a multi-layer rasterized map; the multi-layer rasterized map includes a rasterized ground layer map, a rasterized obstacle layer map, and a rasterized high-altitude layer map.

[0047] In some embodiments, before fusing the road condition information ahead and the surrounding environment information to construct a multi-layer rasterized map, the method further includes: converting the surround view image information and the first obstacle information to a vehicle coordinate system based on the relative positions of the surround view camera and the corner radar and the vehicle; obtaining the attitude information of the drone and the relative position information of the drone and the vehicle; determining a rotation matrix based on the attitude information, and determining a translation vector based on the relative position information; and converting the road condition information ahead to the vehicle coordinate system based on the rotation matrix and the translation vector.

[0048] By converting the data output by the surround-view camera, corner radar, and drone into the vehicle coordinate system respectively, spatial alignment of multi-source data is achieved.

[0049] In an exemplary embodiment, the three-dimensional point cloud data obtained by the drone's laser radar scanning is converted to the vehicle coordinate system through a rotation matrix and a translation vector based on the coordinate origin of the drone's coordinate system. The rotation matrix is ​​calculated based on the attitude angle provided in real time by the drone's inertial measurement unit (IMU), and the translation vector is obtained by estimating the relative position of the drone and the vehicle using visual simultaneous localization and mapping (SLAM) technology. As shown in formula (2):

[0050]

[0051] Where R is the rotation matrix and T is the translation vector. Using Lie group algebra theory, the 3D point cloud Pu = {Xu, Yu, Zu} obtained by the drone’s lidar scan is transformed into the vehicle coordinate system Pv = {Xv, Yv, Zv}.

[0052] In some embodiments, before fusing the forward road condition information and the surrounding environment information to construct a multi-layer rasterized map, the method further includes: performing time alignment on the forward road condition information and the surrounding environment information.

[0053] In an exemplary embodiment, hardware timestamps are used to align forward traffic information and surrounding environment information. Hardware timestamp alignment. Hardware timestamp alignment refers to the process of unifying data collected by multiple devices or sensors to the same time base using precise hardware-level time stamps, ensuring strict temporal consistency across all systems.

[0054] In some embodiments, the fusing of the road condition information ahead and the surrounding environment information to construct a multi-layer rasterized map includes: performing a perspective transformation based on the surround image information to obtain a corresponding bird's-eye view image; fusing the first obstacle information and the bird's-eye view image at the same acquisition time to obtain an environmental model; and constructing the multi-layer rasterized map based on the road condition information ahead and the environmental model.

[0055] The environment model includes a bird's-eye view image (BEV), and obstacles in the bird's-eye view image are associated with position description information and motion description information of the obstacles. For example, obstacles in the bird's-eye view image are associated with speed and angle information of the obstacles.

[0056] In some embodiments, the fusing of the first obstacle information and the bird's-eye view image at the same acquisition time to obtain the environment model includes: obtaining the bird's-eye view image at the same acquisition time as the first obstacle information, projecting the obstacle in the bird's-eye view image based on the position description information in the first obstacle information to obtain the projection position coordinates of the obstacle in the bird's-eye view image; and fusing the first obstacle information with the bird's-eye view image based on the projection position coordinates and the center coordinates of the obstacle detection box in the bird's-eye view image to obtain the environment model.

[0057] In some embodiments, based on the projection position coordinates and the center coordinates of the obstacle detection frame in the bird's-eye view image, the first obstacle information is fused with the bird's-eye view image, including: determining a Mahalanobis distance based on the projection position coordinates and the center coordinates of the obstacle detection frame in the bird's-eye view image; and associating the first obstacle information with the obstacle detection frame in the bird's-eye view image when the Mahalanobis distance is less than a distance threshold.

[0058] In an exemplary embodiment, the first obstacle information and the bird's-eye view image are associated using the Hungarian algorithm, and the Mahalanobis distance is used as the association metric. The Mahalanobis distance is expressed as formula (3):

[0059] d 2 =(zh(x)) T S -1 (zh(x)) (3)

[0060] Where z is the center coordinate of the obstacle detection frame in the bird's-eye view image (i.e., the coordinates of the center of the detection frame), h(x) is the projected coordinates of the obstacle detected by the corner radar in the BEV image, and S is the pre-configured covariance matrix. By minimizing the Mahalanobis distance using the Hungarian algorithm, a one-to-one association is achieved between the obstacle detection frame in the bird's-eye view image and the obstacle in the first obstacle information. Once the two are successfully associated, the obstacle's position, velocity, and other information from the bird's-eye view image and the first obstacle information are integrated to construct a globally consistent environment model.

[0061] In some embodiments, constructing the multi-layer rasterized map based on the road condition information ahead and the environment model includes: extracting first target points having a height value less than or equal to a first preset value from the three-dimensional point cloud data, fitting each first target point to obtain a continuous ground layer, and rasterizing the ground layer to obtain a rasterized ground layer map; extracting second target points having a height value greater than the first preset value and less than a second preset value from the three-dimensional point cloud data, clustering each second target point to obtain an obstacle point cloud cluster, associating the second obstacle information with the obstacle point cloud cluster according to the acquisition time, and obtaining a rasterized obstacle layer map corresponding to the raster of the ground layer; extracting third target points having a height value greater than or equal to the second preset value from the three-dimensional point cloud data, identifying suspended obstacle point clouds based on each third target point, determining the height and lateral span of the suspended obstacle, and obtaining a rasterized high-altitude layer map corresponding to the raster of the ground layer.

[0062] Assuming that the grid resolution is 0.5m*0.5m, after the ground layer map is divided into grids according to this resolution, the obstacle layer map and the upper air layer map correspond to the same grid boundary, that is, one grid in this paper includes the ground layer, obstacle layer and upper air layer at the same time.

[0063] In the exemplary embodiment, the Z-axis coordinate in the 3D point cloud data represents the height direction, while the X-axis direction points forward from the ground plane. The Z-axis coordinate value of the 3D point cloud data represents the height value of the 3D point cloud. Points within the 3D point cloud data with a Z axis between 0 and 0.2 meters are designated as the first target point, points within the Z axis between 0.2 and 2 meters as the second target point, and points with a Z axis greater than 2 meters as the third target point.

[0064] In the exemplary embodiment, each point in the 3D point cloud data includes coordinate values ​​(X / Y / Z coordinates) and a reflection intensity value. The coordinate values ​​represent the spatial properties of the object, while the reflection intensity value is used to characterize the surface characteristics of the ground or obstacles. The reflection intensity value can be used to identify the ground and obstacles.

[0065] In some embodiments, the multi-layer gridded map further includes a traffic risk level for each grid; and the method further includes: performing the following processing on each grid:

[0066] Determine a slope value and a roughness value based on a ground layer map corresponding to the grid; wherein the slope value is used to indicate the inclination of the terrain within the grid; and the roughness value is used to indicate the flatness of the terrain within the grid; determine an obstacle density value based on an obstacle layer map corresponding to the grid; wherein the obstacle density value is used to indicate the probability that the grid is an obstacle; perform a weighted operation based on the respective weights of the slope value, the roughness value, and the obstacle density value to obtain a quantified value of the traffic risk of the grid; and determine the traffic risk level of the grid based on the value range corresponding to the communication risk quantification value.

[0067] In the exemplary embodiment, the slope value (expressed as α), roughness value (expressed as σ), and obstacle density value (expressed as ρ) of each grid are calculated to obtain a comprehensive risk coefficient: R = 0.5α + 0.3σ + 0.2ρ. The weight values ​​0.5, 0.3, and 0.2 are only examples and can be configured to other values ​​as needed. This risk coefficient is the quantitative value of the traffic risk of the grid. Among them, α is the slope value of the grid, normalized to the value range of [0, 1], 15° corresponds to 0.5, and 30° corresponds to 1; σ is the roughness value, determined based on the height variance of the highest and lowest points of the three-dimensional point cloud data of the lidar within the grid, and σ = 0 indicates complete flatness; ρ is the obstacle density value, which is used to indicate the probability that the grid is an obstacle, and also represents the proportion of obstacle grids per unit area, with a value range of greater than or equal to 0 and less than or equal to -1.

[0068] In an exemplary embodiment, when the quantified value of the traffic risk is greater than or equal to the first quantified threshold, the traffic risk level is determined to be high risk, and the corresponding grid in the multi-layer rasterized map is marked red, prohibiting passage; when the quantified value of the traffic risk is greater than or equal to the second quantified threshold, and less than the first quantified threshold, the traffic risk level is determined to be medium risk, and the corresponding grid in the multi-layer rasterized map is marked yellow, requiring careful driving; when the quantified value of the traffic risk is less than the second quantified threshold, the traffic risk level is determined to be low risk, and the corresponding grid in the multi-layer rasterized map is marked green, indicating safe passage. Among them, the first quantified threshold is greater than the second quantified threshold, for example, the first quantified threshold is 0.6, and the second quantified threshold is 0.3; green (R<0.3): safe passage; yellow (0.3≤R<0.6): careful driving; red (R≥0.6): prohibited passage. The risk levels in the map are visualized, and the risk levels are adapted to the vehicle characteristics. The three-level risk level division of green / yellow / red can fit the vehicle characteristics and avoid assessment deviations such as "misjudgment of danger when the vehicle can pass" or "safety when the danger is missed".

[0069] In an exemplary embodiment, the traffic risk level of the grid may be updated at a certain frequency, for example, at a frequency of 10 Hz.

[0070] In some embodiments, the method further includes: determining the slope value and curvature radius of each grid in the ground layer map; in the ground layer map, marking the grids whose slope value is less than the slope threshold and whose curvature radius is greater than the curvature threshold as drivable areas; in the ground layer map, marking the grids whose slope value is greater than or equal to the slope threshold, or whose curvature radius is less than or equal to the curvature threshold, as prohibited driving areas.

[0071] In some embodiments, the method further includes: when the height of the suspended obstacle is less than a height threshold and the lateral span is greater than a lateral threshold, marking the grid corresponding to the suspended obstacle as a high-altitude danger area in the high-altitude layer map.

[0072] In the exemplary embodiment, when constructing the ground layer map, the RANSAC algorithm is used to fit the first target point to obtain the ground plane, and the slope and curvature of each grid (resolution 0.5m×0.5m) are calculated. The slope calculation is shown in formula (4), and the curvature calculation is shown in formula (5):

[0073] α = arctan(ΔZ / ΔX) (4);

[0074]

[0075] Wherein, α represents the slope value, ΔZ represents the Z-axis difference between the highest and lowest points of each first target point in the grid, and ΔX represents the X-axis difference between the highest and lowest points of each first target point in the grid.

[0076] The radius of curvature is expressed as r = 1 / k and can measure the flatness of the terrain.

[0077] The drivable areas in the ground layer map are screened by slope value and curvature radius. Grids that meet the conditions α < 15° and r > 5m are marked as drivable areas, and grids that do not meet these conditions are marked as prohibited driving areas, that is, steep slopes or high curvature areas, so that they can be avoided during intelligent assisted driving to prevent vehicle chassis collisions or vehicle loss of control.

[0078] In an exemplary embodiment, the data basis for constructing the obstacle layer map includes each second target point in the three-dimensional point cloud data and the second obstacle information output by the drone. The second target points are used for obstacle set contour segmentation, and the second obstacle information is used for obstacle type classification, such as rocks, tree trunks, etc.

[0079] The process of constructing the obstacle layer map includes: using the Euclidean clustering algorithm to segment each second target point into an independent obstacle point cloud cluster, and the Euclidean clustering algorithm to segment the second target points with a Euclidean distance less than a threshold of 0.5m into the same obstacle point cloud cluster. These obstacle point cloud clusters constitute the outline of the obstacle.

[0080] The semantic labels (such as "rock" and "vegetation") identified by the drone through the improved YOLOv5s in the second obstacle information are associated with the obstacle point cloud cluster to complete the obstacle type labeling.

[0081] Based on the Bayesian filtering algorithm, the dynamic probability of the grid is updated based on multiple frames of 3D point cloud data to determine the probability of each grid belonging to an obstacle area. This probability value corresponds to the obstacle density. If the probability after multiple frames of continuous updates is greater than 0.7, the grid is determined to be an obstacle area. Otherwise, it is a free area (i.e., a ground area without obstacles).

[0082] The process of dynamic probability update is as follows:

[0083] First, the grid probability is initialized based on the first frame of 3D point cloud data collected by the lidar. Pinit = min(1, max(0, number of point clouds in the grid / empirical threshold)). The number of point clouds in the grid varies with the scene and is not a fixed value. For example, rock obstacles have more point clouds, while flat roads have fewer point clouds. The empirical threshold is set to 5. That is, if the number of point clouds in the grid is ≥5, the initial probability is close to 1.

[0084] Pre-configure the initial probability of the obstacle grid. For example, the initial probability value of the grid of the vehicle, pedestrian, rock and other obstacle classes is Pinit = 0.9. Different initial probability values ​​can also be set for different types of obstacles. The initial probability value of the grid of the road without obstacles, that is, the free class, is Pinit = 0.9. init =0.1.

[0085] When the grid contains both 3D point cloud data and obstacle categories identified by visual semantics, the initial probability value of the grid is determined by weighted calculation, for example, Pinit = 0.6 * number of point clouds in the grid / empirical threshold + 0.4 * Pcls, where Pcls is the initial probability value of the obstacle category.

[0086] Secondly, the Bayesian iterative algorithm is used to update the probability value of the grid in each frame of 3D point cloud data. The update rules are as follows:

[0087] Ppost=min(1,Pprior+ΔPocc) when an obstacle is detected;

[0088] Ppost=max(0,Pprior-ΔPfree) when no obstacle is detected;

[0089] For example,

[0090] Pprior is the probability value of the grid in the previous frame of point cloud data. ΔPocc = 0.3 is the probability increment when the grid detects an obstacle in the current frame. This value is a pre-configured fixed value. The value of 0.3 is only an example. Other values ​​can be configured according to the situation. Similarly, Δpfree = 0.2 is the probability increment when the grid does not detect an obstacle in the current frame. This value is a pre-configured fixed value. The value of 0.2 is only an example. Other values ​​can be configured according to the situation.

[0091] Then, a multi-frame iteration process is performed, using the posterior probability Ppost(t-1) of the previous frame as the prior probability Pprior(t) of the current frame. Continuous iteration is performed to achieve "evidence accumulation". For example, if an obstacle is detected in three consecutive frames, the probability increases from 0.5 to 0.8 to 1.0 (saturation).

[0092] Finally, the decision output is: if Ppost ≥ 0.7, the grid is considered an obstacle area; otherwise, it is a free area. 0.7 is a pre-configured empirical threshold that balances false detections and missed detections, and can be replaced with other values ​​as needed.

[0093] In the disclosed embodiment, by fusing the road condition information ahead detected by the drone and the surrounding environment information collected by the vehicle-mounted sensors, a multi-layer rasterized map including the ground layer, obstacle layer and high-altitude layer is generated, thereby improving the map accuracy, providing data support for intelligent assisted driving, and thus ensuring the accuracy and safety of intelligent assisted driving decisions.

[0094] The disclosed embodiment adopts a method of fusing drone detection and vehicle-mounted sensor detection to ensure the comprehensiveness and accuracy of the constructed multi-layer rasterized map, and can solve the problems of the existing technology using a single sensor or simple data superposition, which leads to fuzzy obstacle type identification and a high false detection rate in dynamic scenes.

[0095] Furthermore, the multi-layered rasterized map in this disclosed embodiment integrates data from drone high-altitude detection and vehicle-mounted sensor data, enabling a full-dimensional quantitative assessment of ground terrain, static obstacles, and high-altitude threats, filling the blind spot in three-dimensional risk assessment. Using multi-layered rasterized maps as the basis for intelligent assisted decision-making ensures the refinement of the maps upon which intelligent assisted decision-making relies. Risk assessments are conducted based on the dynamic characteristics of each grid in the map, such as slope and roughness, providing a basis for refined intelligent assisted driving decisions. This allows drivers to identify potential hazards such as cliff edges, deep pits, and low-altitude height-restricted obstacles in advance, ensuring driving safety and efficiency.

[0096] In addition, experimental verification has shown that the multi-layer rasterized map can cover the three-dimensional environment of 0-50 meters in front of the vehicle, accurately distinguish more than 20 types of obstacles, and reduce the false detection rate of obstacles from 30% to 5%. It fills the high-altitude areas and terrain blind spots, improves the decision-making accuracy of intelligent assisted driving, avoids misjudgments of slopes and height limits, shortens the driver's response time, and deeply couples risk assessment with vehicle performance.

[0097] It is understood that the above-mentioned various method embodiments mentioned in this disclosure can be combined with each other to form combined embodiments without violating the principle logic. Due to space limitations, this disclosure will not go into details. It is understood by those skilled in the art that in the above-mentioned methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic, and the execution order between steps is not limited to being implemented according to the step number.

[0098] Exemplary Systems

[0099] Based on the same concept, the present disclosure also provides an intelligent assisted driving system, which mainly includes a drone and a vehicle:

[0100] The drone is used to collect real-time environmental information about the road ahead of the vehicle;

[0101] The vehicle is used to obtain forward environmental information of the road on which the vehicle is traveling and information about the vehicle's surrounding environment, the forward road condition information being obtained via a drone, and the surrounding environment information being obtained via on-board sensors; the forward road condition information and the surrounding environment information being integrated to construct a multi-layer rasterized map; the multi-layer rasterized map comprising a rasterized ground layer map, a rasterized obstacle layer map, and a rasterized high-altitude layer map.

[0102] In some embodiments, the drone is equipped with a lidar and a wide-angle camera, and the wide-angle camera is pre-configured with an improved lightweight target detection model; the drone obtains three-dimensional point cloud data through real-time acquisition by the lidar, and the wide-angle camera processes the collected video through the target detection model to obtain a two-dimensional bounding box and pixel-level mask that mark the obstacle type.

[0103] In some embodiments, the vehicle includes a surround-view camera, a corner camera, and a controller. The surround-view camera collects images of the vehicle's surroundings, while the corner camera collects information such as the speed and angle of obstacles around the vehicle. The vehicle controller fuses the forward road condition information with the surrounding environment information to construct a multi-layer rasterized map.

[0104] In addition, the present disclosure also provides devices, electronic devices, and computer program products, all of which can be used to implement any of the map construction methods for vehicle assisted driving provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method section and will not be repeated here.

[0105] Exemplary devices

[0106] Figure 3 This is a block diagram of a map construction device for vehicle assisted driving provided by an embodiment of the present disclosure. The map construction device for vehicle assisted driving mainly includes:

[0107] An acquisition module 301 is configured to acquire information about the front environment of the road on which the vehicle is traveling and information about the surrounding environment of the vehicle, wherein the front road condition information is acquired by a drone, and the surrounding environment information is acquired by vehicle-mounted sensors;

[0108] The construction module 302 is used to fuse the forward road condition information and the surrounding environment information to construct a multi-layer rasterized map; the multi-layer rasterized map includes a rasterized ground layer map, a rasterized obstacle layer map and a rasterized high-altitude layer map.

[0109] Figure 4 A block diagram of an electronic device provided in an embodiment of the present disclosure.

[0110] Reference Figure 4An embodiment of the present disclosure provides an electronic device, which includes: at least one processor 401; at least one memory 402, and one or more I / O interfaces 403, connected between the processor 401 and the memory 402; wherein the memory 402 stores one or more computer programs that can be executed by the at least one processor 401, and the one or more computer programs are executed by the at least one processor 401 to enable the at least one processor 401 to execute the above-mentioned map construction method for vehicle assisted driving.

[0111] Each module in the above-mentioned electronic device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0112] Exemplary computer program products and storage media

[0113] The embodiments of the present disclosure further provide a computer program product, including a computer program, which implements the above-mentioned map construction method for vehicle assisted driving when executed in a processor.

[0114] The computer program may be stored in a readable storage medium of a computer device or in the cloud; the processor of the computer device reads the computer program from the readable storage medium or the cloud.

[0115] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0116] It will be understood by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium).

[0117] As is well known to those skilled in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those skilled in the art, communication media typically contains computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0118] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0119] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0120] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0121] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0122] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0123] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0124] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0125] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A map construction method for vehicle assisted driving, characterized in that: include: Acquiring forward environmental information of the road on which the vehicle is traveling and information about the vehicle's surrounding environment, wherein the forward road condition information is obtained by a drone, and the surrounding environment information is obtained by onboard sensors; Fusing the forward road condition information and the surrounding environment information to construct a multi-layer rasterized map; The multi-layer rasterized map includes a rasterized ground layer map, a rasterized obstacle layer map and a rasterized high-altitude layer map.

2. The method according to claim 1, characterized in that The surrounding environment information includes surround view image information collected by the surround view camera and the corresponding collection time, and first obstacle information collected by the corner radar and the corresponding collection time, the first obstacle information including position description information and motion description information; The forward road condition information includes a real-time road condition data stream and second obstacle information corresponding to the real-time road condition data stream; the second obstacle information includes a two-dimensional bounding box of the obstacle, a pixel-level mask, and a corresponding acquisition time, and the two-dimensional bounding box carries a semantic label of the obstacle; the real-time road condition data stream includes three-dimensional point cloud data at each acquisition time.

3. The method according to claim 2, characterized in that The fusing of the front road condition information and the surrounding environment information to construct a multi-layer rasterized map includes: Performing perspective transformation based on the surround view image information to obtain a corresponding bird's-eye view image; fusing the first obstacle information and the bird's-eye view image acquired at the same time to obtain an environment model; The multi-layer rasterized map is constructed based on the front road condition information and the environment model.

4. The method according to claim 3, characterized in that The fusing of the first obstacle information and the bird's-eye view image at the same acquisition time to obtain an environment model includes: Acquire the bird's-eye view image collected at the same time as the first obstacle information, and project the position description information in the first obstacle information onto the bird's-eye view image to obtain the projection position coordinates of the obstacle in the bird's-eye view image; Based on the projection position coordinates and the center coordinates of the obstacle detection frame in the bird's-eye view image, the first obstacle information is fused with the bird's-eye view image to obtain an environment model.

5. The method according to claim 4, characterized in that The method further comprises: fusing the first obstacle information with the bird's-eye view image based on the projection position coordinates and the center coordinates of the obstacle detection frame in the bird's-eye view image, including: determining a Mahalanobis distance based on the projection position coordinates and the center coordinates of the obstacle detection frame in the bird's-eye view image; When the Mahalanobis distance is less than a distance threshold, the first obstacle information is associated with the obstacle detection frame in the bird's-eye view image.

6. The method according to claim 3, characterized in that The constructing of the multi-layer rasterized map based on the front road condition information and the environment model includes: Extracting first target points having a height value less than or equal to a first preset value from the three-dimensional point cloud data, fitting the first target points to obtain a continuous ground layer, and rasterizing the ground layer to obtain a rasterized ground layer map; extracting from the three-dimensional point cloud data each second target point having a height greater than the first preset value and less than a second preset value, clustering and segmenting each second target point to obtain an obstacle point cloud cluster, associating the second obstacle information with the obstacle point cloud cluster according to the acquisition time, and obtaining a rasterized obstacle layer map corresponding to the grid of the ground layer; Extract each third target point whose height value is greater than or equal to the second preset value from the three-dimensional point cloud data, identify the suspended obstacle point cloud based on each third target point, determine the height and lateral span of the suspended obstacle, and obtain a rasterized high-altitude layer map corresponding to the grid of the ground layer.

7. The method according to claim 6, characterized in that The multi-layer rasterized map also includes the traffic risk level of each grid; The method further comprises: Perform the following processing on each raster separately: Determining a slope value and a roughness value based on a ground layer map corresponding to the grid; wherein the slope value is used to indicate the inclination of the terrain within the grid; and the roughness value is used to indicate the flatness of the terrain within the grid; Determining an obstacle density value based on an obstacle layer map corresponding to the grid; wherein the obstacle density value is used to indicate a probability that the grid is an obstacle; Performing a weighted operation according to the respective weights of the slope value, the roughness value, and the obstacle density value to obtain a quantified value of the traffic risk of the grid; The traffic risk level of the grid is determined according to the value range corresponding to the communication risk quantization value.

8. The method according to claim 6, characterized in that The method further comprises: Determining the slope value and curvature radius of each grid in the ground layer map; In the ground layer map, a grid having a slope value less than a slope threshold and a curvature radius greater than a curvature threshold is marked as a drivable area; In the ground layer map, grids whose slope values ​​are greater than or equal to the slope threshold, or whose curvature radii are less than or equal to the curvature threshold, are marked as prohibited driving areas.

9. The method according to claim 6, characterized in that The method further comprises: When the height of the suspended obstacle is less than the height threshold and the lateral span is greater than the lateral threshold, the grid corresponding to the suspended obstacle is marked as a high-altitude danger area in the high-altitude layer map.

10. The method according to claim 2, characterized in that Before fusing the front road condition information and the surrounding environment information to construct a multi-layer rasterized map, the method further includes: Based on the relative positions of the surround view camera, the corner radar and the vehicle, converting the surround view image information and the first obstacle information into a vehicle coordinate system; Acquiring the attitude information of the UAV and the relative position information of the UAV and the vehicle; determining a rotation matrix based on the posture information, and determining a translation vector based on the relative position information; The forward road information is converted to the vehicle coordinate system based on the rotation matrix and the translation vector.

11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores at least one computer program executable by the at least one processor, and the at least one computer program is executed by the at least one processor so as to enable the at least one processor to execute the map construction method for vehicle assisted driving according to any one of claims 1 to 10.

12. A computer program product, characterized in that The computer program product includes a computer program, which implements the map construction method for vehicle assisted driving according to any one of claims 1 to 10 when executed in a processor.

Images