Road condition perception method, device, equipment, storage medium and product

Abstract

This application discloses a road condition perception method, device, equipment, storage medium, and product, relating to the field of artificial intelligence technology. The method includes: constructing a risk cost map based on aircraft perception data and vehicle perception data; planning a global movement trajectory based on the risk cost map; locally optimizing the global movement trajectory to generate an optimized movement trajectory; and controlling the aircraft and / or vehicle to move according to the optimized movement trajectory to perform road condition perception. By aligning and fusing the macroscopic global field-of-view perception data provided by the aircraft with the local high-precision information obtained by vehicle sensors in a unified representation space during road condition perception, both global foresight and local refinement are preserved in autonomous driving path planning, improving recognition performance. Furthermore, the risk cost map constructed based on the fusion enables task-oriented path optimization, allowing vehicles to achieve more robust perception and decision-making in unexpected scenarios.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to road condition perception methods, devices, equipment, storage media, and products. Background Technology

[0002] Traditional intelligent driving relies on onboard sensors (cameras, radar) for near-field perception. Although it has high accuracy in local environments, it has obvious limitations in field of view and blind spots. It is difficult to detect construction in the distance, stationary obstacles, traffic flow after curves, and the behavior patterns of vehicles behind in a timely manner. In multi-vehicle lateral tests, the inability to deal with suddenly appearing dynamic obstacles (such as animals) and the inability to identify static obstacles in front (such as stationary or slow-moving vehicles) are the major problems of intelligent driving. Summary of the Invention

[0003] The main purpose of this application is to provide a road condition sensing method, device, equipment, storage medium and product, which aims to solve the technical problems that related technologies cannot cope with the sudden appearance of animals and cannot identify stationary or slow-moving vehicles in front.

[0004] To achieve the above objectives, this application proposes a road condition perception method, the method comprising: A risk cost map is constructed based on aircraft perception data and vehicle perception data. Plan the global movement trajectory based on the aforementioned risk cost map; The global movement trajectory is locally optimized to generate an optimized movement trajectory; The aircraft and / or vehicle are controlled to move according to the optimized movement trajectory in order to achieve road condition perception.

[0005] Optionally, constructing the risk cost map based on aircraft perception data and vehicle perception data includes: The aircraft's sensing data is projected and transformed to generate projected sensing data; An environmental map is generated based on the projection perception data and the vehicle perception data; The environmental map is marked with occupancy probability to generate a risk cost map.

[0006] Optionally, the projection transformation of the aircraft sensing data to generate projected sensing data includes: The aircraft perception data is divided into flat terrain perception data and non-flat terrain perception data. The flat terrain sensing data is projected onto the ground to generate first projection data; The non-flat terrain sensing data is back-projected to generate second projection data; Projection perception data is constructed based on the first projection data and the second projection data.

[0007] Optionally, generating an environmental map based on the projected perception data and the vehicle perception data includes: The projection sensing data is spatiotemporally aligned to generate aligned sensing data. An environment map is generated based on the alignment perception data and the vehicle perception data.

[0008] Optionally, the step of marking the occupancy probability of the environment map to generate a risk cost map includes: Static element modeling is performed based on the projection perception data and the vehicle perception data to generate the static occupancy probability of each grid in the environment map. Based on the projection perception data and the vehicle perception data, dynamic element monitoring is performed to generate the dynamic occupancy probability of each grid in the environment map. Based on the static occupancy probability and the dynamic occupancy probability, construct the occupancy probability parameters corresponding to each grid in the environment map; The environmental map is marked based on the occupancy probability parameter to generate a risk cost map.

[0009] Optionally, the step of performing static element modeling based on the projected perception data and the vehicle perception data to generate the static occupancy probability corresponding to each grid in the environment map includes: Based on the projection perception data and the vehicle perception data, a probability occupancy calculation is performed on the environmental map to generate occupancy assessment parameters. The occupancy assessment parameters are normalized to generate the static occupancy probability corresponding to each grid cell in the environment map.

[0010] Optionally, the step of dynamically monitoring elements based on the projected perception data and the vehicle perception data to generate the dynamic occupancy probability corresponding to each grid in the environmental map includes: The projection perception data and the vehicle perception data are fused to generate fused features; Dynamic element observation data is constructed based on the aforementioned fusion features; Based on the dynamic element observation data, the dynamic occupancy probability of each grid cell in the environment map is constructed.

[0011] Optionally, the step of planning the global movement trajectory based on the risk cost map includes: Based on the path planning algorithm, at least one path to be detected is constructed according to the path start point and the path end point; The path cost weight corresponding to each path to be detected is determined based on the risk cost map. A global movement trajectory is selected from the at least one path to be detected based on the path cost weight.

[0012] Optionally, the step of performing local optimization on the global movement trajectory to generate an optimized movement trajectory includes: Generate a sequence of control commands based on the global movement trajectory; Construct a predicted trajectory based on the control command sequence; Select optimizable points from the predicted trajectory; Local optimization is performed on the control instructions corresponding to the optimizable points in the control instruction sequence to generate an optimized instruction sequence; An optimized movement trajectory is generated based on the optimized instruction sequence.

[0013] Optionally, selecting optimizable points in the predicted trajectory includes: The cost information corresponding to the predicted trajectory is calculated using a preset cost function; Points in the predicted trajectory whose corresponding cost information is greater than a preset cost threshold are selected as points that can be optimized.

[0014] Optionally, the aircraft sensing data is sent by the aircraft when it determines that the increase in the amount of observation information is greater than a preset threshold.

[0015] Furthermore, to achieve the above objectives, this application also provides a road condition sensing device, the road condition sensing device comprising: The module is used to build a risk cost map based on aircraft perception data and vehicle perception data. The planning module is used to plan the global movement trajectory based on the risk cost map; The optimization module is used to perform local optimization on the global movement trajectory to generate an optimized movement trajectory; The control module is used to control the movement of the aircraft and / or vehicle according to the optimized movement trajectory in order to perform road condition perception.

[0016] Optionally, the construction module is further configured to perform projection transformation on the aircraft perception data to generate projected perception data; generate an environmental map based on the projected perception data and the vehicle perception data; and mark the environmental map with occupancy probability to generate a risk cost map.

[0017] Optionally, the construction module is further configured to divide the aircraft perception data into flat terrain perception data and non-flat terrain perception data; project the flat terrain perception data onto the ground to generate first projection data; perform back-projection processing on the non-flat terrain perception data to generate second projection data; and construct projection perception data based on the first projection data and the second projection data.

[0018] Optionally, the construction module is further configured to perform spatiotemporal alignment processing on the projected perception data to generate aligned perception data; and generate an environmental map based on the aligned perception data and the vehicle perception data.

[0019] Optionally, the construction module is further configured to perform static element modeling based on the projection perception data and the vehicle perception data to generate static occupancy probabilities corresponding to each grid in the environment map; perform dynamic element monitoring based on the projection perception data and the vehicle perception data to generate dynamic occupancy probabilities corresponding to each grid in the environment map; construct occupancy probability parameters corresponding to each grid in the environment map based on the static occupancy probabilities and the dynamic occupancy probabilities; and mark the environment map based on the occupancy probability parameters to generate a risk cost map.

[0020] Optionally, the construction module is further configured to perform probability occupancy calculation on the environmental map based on the projection perception data and the vehicle perception data, and generate occupancy assessment parameters; and to normalize the occupancy assessment parameters to generate the static occupancy probability corresponding to each grid in the environmental map.

[0021] In addition, to achieve the above objectives, this application also provides a road condition sensing device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the road condition sensing method as described above.

[0022] In addition, to achieve the above objectives, this application also provides a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the road condition perception method as described above.

[0023] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the road condition perception method as described above.

[0024] One or more technical solutions proposed in this application have at least the following technical effects: By aligning and fusing the macroscopic global field of vision perception data provided by the aircraft with the local high-precision information obtained by the vehicle sensors in a unified representation space during road condition perception, the autonomous driving path planning can achieve both global foresight and local precision, thereby improving the recognition effect. Furthermore, the risk cost map constructed based on the fusion enables task-oriented path optimization, allowing the vehicle to achieve more robust perception and decision-making in unexpected scenarios. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart illustrating an embodiment of the road condition perception method of this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the road condition perception method of this application; Figure 3 This is a flowchart illustrating Embodiment 3 of the road condition perception method of this application; Figure 4 This is a schematic diagram of the module structure of the road condition sensing device according to an embodiment of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the road condition perception method in the embodiments of this application.

[0028] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0029] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0030] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0031] Based on this, embodiments of this application provide a road condition perception method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the road condition perception method of this application.

[0032] In this embodiment, the road condition perception method includes steps S10 to S40: Step S10: Construct a risk cost map based on aircraft perception data and vehicle perception data.

[0033] It should be noted that the execution subject of this embodiment can be a road condition sensing device. The road condition sensing device can be an electronic device such as a personal computer, server, or controller, or other devices that can achieve the same or similar functions. For example, in order to facilitate air-ground collaborative road condition sensing, the road condition sensing device can be a controller installed in the vehicle, such as an intelligent driving controller. Of course, depending on actual needs, the road condition sensing device can also be other devices that are independent of the vehicle but can control the vehicle. This embodiment does not limit this. In this embodiment and the following embodiments, the road condition sensing device is used as an example to describe the road condition sensing method of this application.

[0034] It's important to note that traditional autonomous driving relies on onboard sensors (cameras, radar) for near-field perception. While this provides high accuracy in localized environments, it suffers from significant limitations in field of view and blind spots. It struggles to detect distant construction sites, stationary obstacles, traffic flow after curves, and the behavior patterns of vehicles behind. Its inability to respond to suddenly appearing animals and its inability to recognize stationary or slow-moving vehicles ahead are two major areas of concern for autonomous driving. While aircraft (such as drones) can provide high-altitude, top-down global situational information, their low perception resolution, communication latency, and bandwidth limitations make it difficult to independently perform high-precision planning.

[0035] Based on this, we can explore fusing air-to-ground multimodal information. Through unified bird's-eye view (BEV) representation, cross-modal attention mechanisms, and dynamic confidence weighting, we can complementaryly integrate the macroscopic traffic flow and road structure information provided by UAVs with the microscopic high-precision environmental perception of vehicles, forming a road environment cognition that combines a global perspective with local details. Simultaneously, through uncertainty modeling and task-driven optimization, we can ensure that perception confidence and potential risks are explicitly considered in path planning, avoiding misjudgments caused by relying on single-modal information. Based on this, we can construct a risk cost map by combining aircraft perception data and vehicle perception data.

[0036] In practical applications, aircraft perception data can be perception data collected by sensors installed in the aircraft, vehicle perception data can be perception data collected by sensors installed in the vehicle, and risk cost map can represent the driving cost of a vehicle when driving on the map. The higher the driving cost, the higher the driving risk.

[0037] In this specific implementation, in order to save communication bandwidth, the aircraft perception data in this embodiment is sent by the aircraft when it determines that the increase in the amount of observation information is greater than a preset threshold.

[0038] In practical applications, the aircraft can calculate the increase in observation information using the following formula:

[0039] In the formula, It is the information entropy of the prior probability of grid m. It is after receiving observation data from the drone z u Then, update the information entropy of the obtained posterior probability. This measures how much new information the drone's observations bring to the system, i.e., how much uncertainty is reduced.

[0040] Understandably, if the drone discovers something the system did not anticipate at all, the posterior probability will change dramatically, the uncertainty will decrease, and the information gain will be very high. By only uploading data from grids with information gain exceeding the threshold to the vehicle, redundant and invalid information is avoided. Only the key information that truly changes the vehicle's perception and avoids danger is used to transmit, which greatly improves the system's efficiency.

[0041] Step S20: Plan the global movement trajectory based on the risk cost map.

[0042] In practical applications, a global movement trajectory with the minimum total cost can be planned.

[0043] In a specific implementation, in order to reasonably construct the global movement trajectory, step S20 of this embodiment may include: Based on the path planning algorithm, at least one path to be detected is constructed according to the path start point and the path end point; The path cost weight corresponding to each path to be detected is determined based on the risk cost map. A global movement trajectory is selected from the at least one path to be detected based on the path cost weight.

[0044] It should be noted that the path planning algorithm can be pre-set by the management personnel of the road condition sensing equipment, such as setting simulated annealing algorithm, artificial potential field method, fuzzy logic algorithm, etc. as the path planning algorithm.

[0045] In practical applications, at least one path to be detected can be planned in the environment map based on the path starting point and the path ending point according to the path planning algorithm. Then, the path cost weight corresponding to each path to be detected can be calculated according to the risk cost map. Finally, the path to be detected with the smallest path cost weight among the at least one path to be detected is taken as the global movement trajectory.

[0046] In practical applications, global movement trajectories can also be planned directly using graph search algorithms based on risk cost maps, for example, using A-Star (abbreviated as A). Graph search algorithm or Dynamic A (abbreviated as D) The graph search algorithm plans the global movement trajectory.

[0047] The specific execution flow of the global movement trajectory at this time is as follows: Converting the risk cost map from a grid to a graph format, and integrating the edge weights of each grid according to the cost, we find the path with the lowest static cost from the starting point to the ending point, which can be represented as:

[0048] Here, G(V, E) is generated by converting the previously obtained risk cost map c into a graph. V represents the center point of each cell in the risk cost map, E represents the edge connecting adjacent cells, and w(e) is the weight of the edge, representing the cost of traversing this edge. The path integral can be approximated as the average cost of the starting and ending cells of the path. A higher weight means a greater cost and less cost-effectiveness in traversing this edge. The search is performed on the weighted graph G to find the path Π with the minimum total weight from the starting point to the ending point. This path is then taken as the globally optimal path.

[0049] Step S30: Perform local optimization on the global movement trajectory to generate an optimized movement trajectory.

[0050] Step S40: Control the aircraft and / or vehicle to move according to the optimized movement trajectory in order to perform road condition perception.

[0051] In practical applications, since the actual environment is constantly changing, the global movement trajectory can generally only indicate one direction of travel. Therefore, the global movement trajectory can be locally optimized according to the actual situation, and then an optimized movement trajectory can be generated. After that, the aircraft and / or vehicle can be controlled to move according to the optimized movement trajectory to perform road condition perception. During road condition perception, new aircraft perception data and vehicle perception data can be generated. After that, the execution step S10 can be returned until the overall road condition perception is completed (such as moving from the preset starting point to the destination).

[0052] In practical use, control commands corresponding to the vehicle can be generated based on the optimized movement trajectory. The vehicle can be controlled to move based on the control commands, and the aircraft can select the corresponding flight trajectory according to the optimized movement trajectory to ensure that the vehicle is within the observation range of the aircraft and that the perception data collected by the aircraft can complement the vehicle.

[0053] This embodiment provides a road condition perception method. By aligning and fusing the macroscopic global field of view perception data provided by the aircraft with the local high-precision information obtained by the vehicle sensors in a unified representation space during road condition perception, it can achieve both global foresight and local precision in autonomous driving path planning, thereby improving the recognition effect. Furthermore, the risk cost map constructed based on the fusion realizes task-oriented path optimization, enabling the vehicle to achieve more robust perception and decision-making in sudden scenarios.

[0054] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Step S10 includes steps S101 to S103: Step S101: Perform projection conversion on the aircraft sensing data to generate projected sensing data.

[0055] It should be noted that the aircraft perception data is collected by the aircraft itself. Since the aircraft is in the air, its dimensions differ from those of the vehicle perception data. In order to ensure that the two can be correctly integrated in the future, the aircraft perception data needs to be projected and transformed from three-dimensional spatial data into two-dimensional data in the overhead view (BEV), thereby generating the corresponding projected perception data.

[0056] In a specific implementation, in order to perform projection transformation reasonably, step S101 in this embodiment may include: The aircraft perception data is divided into flat terrain perception data and non-flat terrain perception data. The flat terrain sensing data is projected onto the ground to generate first projection data; The non-flat terrain sensing data is back-projected to generate second projection data; Projection perception data is constructed based on the first projection data and the second projection data.

[0057] It should be noted that when classifying aircraft perception data, the classification can be based on the sparse or dense depth information corresponding to each data point. For example, if the sparse or dense depth information corresponding to part A of the aircraft perception data does not exist or the confidence level is lower than a preset confidence threshold, then this part can be classified as flat terrain perception data. Conversely, if the sparse or dense depth information corresponding to part B of the aircraft perception data exists and the confidence level is higher than the preset confidence threshold, then this part of the data can be classified as non-flat terrain perception data.

[0058] For ease of understanding, examples are provided below, but these are not intended to limit the scope of this solution: To facilitate calculation, a model can be created: Coordinate System and Transformation: The system defines a unified world coordinate system W. The rigid body transformation matrices from the vehicle and aircraft body coordinate systems to the world coordinate system are as follows: and SE(3) is a three-dimensional special Euclidean group, representing all possible rotational and translational transformations; Vehicle state: The vehicle's state vector at time t is represented by x. t (X)t Y t Let ψ be the two-dimensional position of the vehicle in the world coordinate system. t Let v be the heading angle. t For speed; UAV State: The state vector of the UAV at time t is represented by yt. This represents the three-dimensional position of the aircraft in the world coordinate system. For heading angle; Then at this point:

[0059] The vehicle perception data is z ɡ t The aircraft's sensing data is z u t The perception data can come from sensors such as vision cameras, LiDAR, and millimeter-wave radar.

[0060] For the alignment of the open-ground viewpoint under an approximate planar road z=0, where n is the ground plane normal vector, d is the distance from the camera to the plane, and K is the intrinsic parameter describing the camera's imaging geometry, the identity matrix H∈R is used. 3 3 The top-view image P captured by the UAV is projected onto the ground-based BEV, and the result is... That is, the first projection data:

[0061] Here, R (rotation matrix) and t (translation vector) are extrinsic parameters of the UAV sensor (such as a camera), which together describe the attitude and position of the UAV sensor relative to the world coordinate system.

[0062] For non-flat terrain, sparse or dense depth information D(p) can be used to project point-by-point the image onto each point:

[0063] In the formula, π-1 is the inverse function of π in the UAV sensor projection model, which can perform back projection operation on each pixel P, and the output X is the 3D world coordinate, i.e. the second projection data.

[0064] In practical use, the obtained first projection data and second projection data can be combined into projection sensing data.

[0065] Step S102: Generate an environmental map based on the projection perception data and the vehicle perception data.

[0066] It should be noted that after acquiring projection perception data and vehicle perception data, the preset map can be updated based on both to generate an environmental map.

[0067] In a specific implementation, to ensure a reasonable construction of the environment map, step S102 in this embodiment may include: The projection sensing data is spatiotemporally aligned to generate aligned sensing data. An environment map is generated based on the alignment perception data and the vehicle perception data.

[0068] It should be noted that it takes a certain amount of time for the drone to transmit the perception data to the vehicle, and there may be some delay in the process. In order to ensure the validity of the data and to construct an environmental map reasonably, the projected perception data needs to be spatiotemporally aligned to generate aligned perception data. Then, the environmental map is generated based on the aligned perception data and the vehicle perception data.

[0069] For ease of understanding, examples are provided below, but these are not intended to limit the scope of this solution: Based on the examples above, we can further assume: The initial environment map M consists of a set of grids {mi} with a resolution of △. This map can also be extended with semantic layers (such as categories like roads, vehicles, and pedestrians) and dynamic layers (such as movement speed and intent). In the actual constraints of collaborative communication between vehicles and drones, the communication delay is Δt. c The data packet loss rate is p loss ; The poses of the vehicle and the drone can be estimated using error states EKF and UKF, respectively, at which point:

[0070] Where f, g is the state transition function, which is based on the state (x) at the previous time step. t y t ) and the current control input (u t η t Predict the state at the current moment, w t v t It is process noise; h ɡ h u It is the observation function, which describes the theoretical mapping relationship from the system state to the observed value. , It is observation noise; Based on this, spatiotemporal alignment can be performed to align timestamps:

[0071] In the formula, This is the original BEV feature map of the aircraft at time t, i.e., projected sensing data. Each location has a feature value representing the scene information at that location. It is a BEV feature map after time compensation, that is, aligned sensing data, which simulates the BEV feature map captured by the aircraft at time t through spatiotemporal alignment processing. vx and vy are the velocity fields of the BEV mesh, estimating the motion velocity vector at each position (x, y) on the BEV feature map, which can be obtained through optical flow or target tracking; (x-vx△t, y-vy△t) represents the most likely position before time △t, therefore it is only necessary to... The backtracking position is used to obtain the feature value, which will be filled into (x,y) at the current time t, assuming that the object moves in uniform linear motion within time Δt.

[0072] Step S103: Mark the occupancy probability on the environmental map to generate a risk cost map.

[0073] In practical use, in order to ensure that reasonable paths can be planned in the future, the occupancy probability can be marked on the environmental map, and the marked environmental map can be used as a risk cost map.

[0074] In a specific implementation, in order to reasonably construct a risk cost map, step S103 of this embodiment may include: Static element modeling is performed based on the projection perception data and the vehicle perception data to generate the static occupancy probability of each grid in the environment map. Based on the projection perception data and the vehicle perception data, dynamic element monitoring is performed to generate the dynamic occupancy probability of each grid in the environment map. Based on the static occupancy probability and the dynamic occupancy probability, construct the occupancy probability parameters corresponding to each grid in the environment map; The environmental map is marked based on the occupancy probability parameter to generate a risk cost map.

[0075] It should be noted that static element modeling refers to modeling static objects (such as roadblocks) to determine the probability that a static object occupies a grid, thereby generating a static occupancy probability. Similarly, dynamic element monitoring involves monitoring potentially moving targets (such as pedestrians, animals, etc.) to determine the probability that a dynamic target may occupy a grid, thereby generating a dynamic occupancy probability.

[0076] In practical use, after obtaining the static occupancy probability and the dynamic occupancy probability, the two can be merged to generate the probability that each grid may be occupied at the current moment, i.e., the occupancy probability parameter. Then, the occupancy probability parameter is marked in the environment map, and the marked environment map is used as the risk cost map.

[0077] The static occupancy probability and dynamic occupancy probability can be obtained using the following methods: Pf t (m i )=1 (1 p t (m i )) (1 p t (m i occ |z t )) In the formula, p t (m i () represents the current time t, and the grid m. i The static occupancy probability, p t (m i occ |z t () represents the current time t, and the grid m. i The dynamic occupancy probability, Pf t (m i ) can be the occupancy probability parameter of grid mi at the current time t.

[0078] Of course, other methods of fusion can also be used, and this embodiment does not limit this.

[0079] In practical implementation, in order to maximize the role of the risk cost map, in addition to the occupancy probability parameter, other levels of cost can be fused to generate the passage cost. Then, the environmental map is marked based on the passage cost to generate the risk cost map.

[0080] For example: Suppose the passage cost is c(m); Then we can have:

[0081] In the formula, pf(m) represents the occupancy probability parameter corresponding to the grid in the environment map. The semantic category representing the raster. Represents the speed of movement of objects within the grid. This represents the uncertainty of the road condition perception results, and can be generated based on perception errors, noise data, etc. α occ αT sem α v α σ These are all weighting coefficients, which can be preset by the management personnel of the road condition sensing equipment. If the road condition sensing method of this embodiment is executed using a model, the weighting coefficients can be determined during the model training process.

[0082] It is understandable that the above-mentioned information is integrated into a single index, the passage cost c(m), through a weighted summation formula. The four weighted terms represent physical collision risk, rule and comfort cost, interaction risk cost, and unknown risk cost, respectively.

[0083] In a specific implementation, in order to reasonably generate static occupancy probabilities, the step of modeling static elements based on the projected perception data and the vehicle perception data to generate the static occupancy probability corresponding to each grid in the environment map, as described in this embodiment, may include: Based on the projection perception data and the vehicle perception data, a probability occupancy calculation is performed on the environmental map to generate occupancy assessment parameters. The occupancy assessment parameters are normalized to generate the static occupancy probability corresponding to each grid cell in the environment map.

[0084] In practical applications, the occupancy assessment parameters for each grid cell can be determined by updating the log-posterior probability. Then, in order to convert them into easily understandable probability values, they can be normalized to generate the static occupancy probability corresponding to each grid cell in the environment map.

[0085] For ease of understanding, examples are provided below, but these are not intended to limit the scope of this solution: We can first update the occupancy probability of each grid using the log-posterior probability to determine the occupancy assessment parameters. This parameter represents how much new information the new observations from vehicle and drone sensors bring to the grid's occupancy status. Then, we have:

[0086] Among them, the vehicle perception data is z ɡ t The aircraft's sensing data is z u t w ɡ (m) and w u (m) represent the information weights corresponding to the vehicle and the aircraft, respectively, p(m) i |z ɡ t ) for vehicle to grid m i The amount of information brought by the occupancy state, p(m i |z ut L0 represents the information brought by the aircraft's occupancy status of the grid mi. The inverse sensing model can be set based on a piecewise function or a lightweight neural network. L0 is the initial state maintained when there are no new observations. It is usually assumed that when the environment is initialized, L0=0, so this term can be ignored.

[0087] Once the occupancy evaluation parameters are obtained, they are represented by logarithmic probabilities. At this point, the logarithmic probabilities need to be converted into probability values ​​that are easily understood by humans. σ() is the sigmoid function, the inverse function of log-odds, ensuring that the output probability value is always within the reasonable range of [0,1]. Therefore, we have:

[0088] In the formula, p t (m i () represents the current time t, and the grid m. i The static occupancy probability, where e is the natural logarithm.

[0089] In practical applications, in order to reasonably determine w ɡ (m) and w u For each grid cell that needs to be evaluated, a feature vector h(m) is calculated to comprehensively describe its state. Then, the feature vector h(m) is input into the neural network to obtain the final weight w that determines the importance of information fusion. ɡ (m) and w u (m), at this time we have:

[0090] In the formula, conf u conf ɡ Δt represents the confidence score of the perception algorithm for aircraft and vehicles. u , △t ɡ IG represents the timeliness of information for aircraft and vehicles. u IG ɡ The information gain of aircraft and vehicles measures the useful information that observations bring to the system. u var ɡ The variance of the data for aircraft and vehicles represents the degree of fluctuation across multiple observations of the grid. Then at this point:

[0091] MLP u and MLP ɡ This represents a neural network model, where σ() is the sigmoid function.

[0092] Furthermore, for blind spots caused by occlusion, a visibility weight can be introduced to further modulate the weighting of information fusion, resulting in:

[0093] Here, vis indicates the occlusion state. This represents the amount of new information added to the text above. It is a weight between [0,1]. By simulating the line of sight in the physical world, geometric analysis is performed on the 3D digital map built by the system in real time. It simulates a virtual ray from the mi sensor to the center of the grid. If the ray reaches mi without any other occupied grid, it means that the line of sight is unobstructed, and the weight is set to close to 1. If the ray passes through other occupied grids before reaching mi, it means that mi is located in the blind zone of the sensor, so the weight is set to close to 0. It dynamically evaluates how reliable the vehicle or aircraft is in seeing the current grid mi.

[0094] In a specific implementation, in order to reasonably set the dynamic occupancy probability, the step of generating the dynamic occupancy probability corresponding to each grid in the environment map based on the projection perception data and the vehicle perception data, as described in this embodiment, may include: The projection perception data and the vehicle perception data are fused to generate fused features; Dynamic element observation data is constructed based on the aforementioned fusion features; Based on the dynamic element observation data, the dynamic occupancy probability of each grid cell in the environment map is constructed.

[0095] It should be noted that dynamic elements may move, and during this movement, the probability of each grid cell being occluded will change. Therefore, it is necessary to consider the movement and observation of dynamic elements (also known as dynamic targets), in which case:

[0096] in, That is, the grid m at time t. i The dynamic occupancy probability, z t It can be element observation data, or it can be constructed based on fusion features. For example, the fusion features can be output as the target tracking model, which can then output element observation data. It represents the probability that the j-th grid cell was occupied at the previous time step. , is the state transition probability. As given by the target tracking model, summation implies that the system believes the current grid's occupancy may have been caused by an object moving from any grid in the previous time step; UAV's top-down detection improves the trackability of distant textured targets. The latter describes incorporating new observations from the current time step into the system to correct and update the prediction results, resulting in a more accurate and reliable estimate. The updated posterior probability aligns the features of the two different modalities in the same world coordinate system, preparing for subsequent fusion.

[0097] In practical applications, the methods for obtaining fused features can be as follows:

[0098] In the formula, This represents the attention score, which measures the relevance of features at each location of the vehicle. It's a large negative value, forcibly reducing attention to invisible areas and preventing the system from speculating about information in blind spots. The transmission delay of the j-th feature information of the aircraft is used to assign lower weight to outdated information. Information with large delays can no longer accurately reflect the current environmental state, resulting in lower reliability during fusion. It is an enhanced vehicle feature that incorporates aircraft information (AV), i.e., a fused feature.

[0099] This embodiment provides a road condition perception method. This embodiment takes into account the modal differences between aircraft perception data and vehicle perception data, and performs projection processing and spatiotemporal alignment on them to ensure the rationality of the risk cost map generated by fusion.

[0100] Based on the first embodiment of this application, in the third embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 Step S30 includes steps S301 to S305: Step S301: Generate a control command sequence based on the global movement trajectory.

[0101] Step S302: Construct a predicted trajectory based on the control command sequence.

[0102] In practical use, the global movement trajectory can be used as a directional reference to generate a batch of control commands required when the vehicle travels in that direction, and these control commands can be assembled into a control command sequence in sequence.

[0103] Then, the control command sequence is analyzed using a dynamic model to generate a predicted trajectory.

[0104] In constructing the control command sequence, a global movement trajectory can be used as a directional guide to construct the control commands required when the vehicle travels along this global movement trajectory for a preset duration (e.g., 1.5 seconds). Step S303: Select optimizable points from the predicted trajectory.

[0105] In practical applications, only the trajectory points in the predicted trajectory whose costs exceed a preset value can be selected as optimizable points. Of course, depending on actual needs, all trajectory points in the entire preset trajectory can also be selected as optimizable points.

[0106] In a specific implementation, in order to reasonably select optimizable points, step S303 in this embodiment may include: The cost information corresponding to the predicted trajectory is calculated using a preset cost function; Points in the predicted trajectory whose corresponding cost information is greater than a preset cost threshold are selected as points that can be optimized.

[0107] It should be noted that the preset cost function can be set in advance by the administrators of the road condition sensing equipment.

[0108] For example: if a pre-defined cost function is constructed using tracking deviation cost, control cost, and collision risk cost, then:

[0109] In the formula, To track the cost of deviation, the vehicle's predicted state at time k is represented by the reference state extracted from the global path at time k. Q is the weight matrix, which determines the penalty for tracking error. The larger Q is, the more strictly the vehicle follows the global path. To control costs, uk is the control input at time k, and R determines the penalty for the magnitude of the control action, avoiding excessive or abrupt steering and acceleration / deceleration, resulting in a smoother riding experience. For the collision risk cost, λc is the weighting coefficient of the risk term, which determines the priority of safety in the optimization. C(x) k ) is the vehicle in state x k The collision risks faced at that time. Specifically:

[0110] p i (k) is the probability that the i-th grid cell is occupied at time k. It's a collision function, which calculates the collision response when the vehicle is in state x. k When the vehicle body overlaps with grid i, the ratio of the overlapping area is such that if the vehicle completely covers grid i, then... If the car is far from the grid, If the vehicle coincides with only a small portion of grid i, then θ i It is a relatively small value.

[0111] C(x k Summing all the grids physically means calculating the sum of the values ​​of the grid cells when the vehicle is in state x. k The expected collision area at that time quantifies the risk level at that location. If a vehicle enters a high-risk area (high p... i (k) ), and the vehicle body overlaps significantly with this area (height θ) i ), then C(x) k If the value is very large, the optimizer will severely punish the state, thus forcing the vehicle to move away from the area; Based on this, its constraints can be easily changed, and it can be written as:

[0112] This means that at every future time k, the vehicle's expected collision area must be less than a safety threshold. , k indicates that this constraint applies to all states throughout the entire prediction time domain. This is equivalent to drawing an absolute boundary for the vehicle's future trajectory, ensuring that the planned trajectory will never "take a risk".

[0113] Step S304: Perform local optimization on the control instructions corresponding to the optimizable points in the control instruction sequence to generate an optimized instruction sequence.

[0114] In practical use, the control commands corresponding to the optimizable points in the control command sequence can be adjusted (such as reducing the steering wheel angle, making the steering wheel angle smaller, increasing or decreasing the speed, etc.) in order to try to reduce the cost corresponding to the optimizable point and generate an optimized command sequence.

[0115] Step S305: Generate an optimized movement trajectory based on the optimized instruction sequence.

[0116] In practical use, after determining the optimized instruction sequence, trajectory prediction can be performed according to the optimized execution sequence to generate an optimized movement trajectory.

[0117] In practical implementation, the weights or parameters, models (such as the model that determines the information fusion weights, the inverse projection model, etc.) involved in the road condition perception method of this embodiment can be updated during multi-task joint training, and the loss function for multi-task joint training can be:

[0118] In the formula, LBCE(p, y) occ ) represents the binary cross-entropy loss, yocc It indicates whether a grid cell is actually occupied, and is responsible for optimizing the accuracy of the occupied grid cell map; For multi-class cross-entropy loss, y sem It represents the actual semantic category of the grid, which is responsible for enabling the model to identify what it is and understand traffic rules; This allows the model to estimate the speed and direction of motion of other objects in the environment, enabling predictions about the future and decision-making. and The predicted motion vectors are simulated separately, with the latter being the true label representing the actual motion state;

[0119] , These represent features extracted from sensors in aircraft and vehicles, respectively. Through H-space transformation, the motion field is utilized... Distort the drone's features to the vehicle's viewpoint coordinate system. It is the L1 norm, used to calculate the absolute difference between the mapped features and the original vehicle features. The optimization goal of cons is to make the feature representation of the same scene as consistent as possible under different views, which is the basis of multi-sensor fusion. To achieve planning awareness, planning failure events (crashes, running off the road) are treated as sparse monitoring signals in the simulation environment.

[0120] And λ occ , λ sem , λ flow , λ cons , λ plan These are all weighting coefficients, which can be preset by the managers of the road condition sensing equipment.

[0121] In practical applications, by treating the entire system as a policy network and fine-tuning it using reinforcement learning, with the ultimate reward being the successful arrival at the destination, a reward model is designed and optimized. For example, the behavior of making dangerous decisions due to missing obstacles is punished, forcing the perception module to pay more attention to areas that are crucial to driving safety, thus achieving end-to-end optimization of perception and decision-making.

[0122] This embodiment provides a road condition perception method. Based on global path planning, this embodiment achieves local optimization, ensuring that the specific movement of the vehicle can change in real time according to the changes of dynamic elements in the actual scene, enabling the vehicle to achieve more robust perception and decision-making in complex traffic and sudden scenarios.

[0123] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the road condition perception method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0124] This application also provides a road condition sensing device, please refer to... Figure 4 The road condition sensing device includes: Module 10 is used to build a risk cost map based on aircraft perception data and vehicle perception data; Planning module 20 is used to plan a global movement trajectory based on the risk cost map; Optimization module 30 is used to perform local optimization on the global movement trajectory to generate an optimized movement trajectory; The control module 40 is used to control the movement of the aircraft and / or vehicle according to the optimized movement trajectory in order to perform road condition perception.

[0125] The road condition sensing device provided in this application, employing the road condition sensing method in the above embodiments, can solve the technical problems of related technologies being unable to cope with suddenly appearing animals and unable to identify stationary or slow-moving vehicles ahead. Compared with the prior art, the beneficial effects of the road condition sensing device provided in this application are the same as those of the road condition sensing method provided in the above embodiments, and other technical features in the road condition sensing device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0126] This application provides a road condition sensing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the road condition sensing method in the first embodiment described above.

[0127] The following is for reference. Figure 5 The diagram illustrates a structural schematic suitable for implementing the road condition sensing device in the embodiments of this application. The road condition sensing device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The road condition sensing device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0128] like Figure 5 As shown, the road condition sensing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the road condition sensing device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the road condition sensing device to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show road condition sensing devices with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0129] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0130] The road condition sensing device provided in this application, employing the road condition sensing method in the above embodiments, can solve the technical problems of related technologies being unable to cope with suddenly appearing animals and unable to identify stationary or slow-moving vehicles ahead. Compared with the prior art, the beneficial effects of the road condition sensing device provided in this application are the same as those of the road condition sensing method provided in the above embodiments, and other technical features in this road condition sensing device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0131] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0132] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0133] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the road condition perception method in the above embodiments.

[0134] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0135] The aforementioned computer-readable storage medium may be included in the road condition sensing device; or it may exist independently and not be assembled into the road condition sensing device.

[0136] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the road condition sensing device, cause the road condition sensing device to: construct a risk cost map based on aircraft perception data and vehicle perception data; plan a global movement trajectory based on the risk cost map; perform local optimization on the global movement trajectory to generate an optimized movement trajectory; and control the aircraft and / or vehicle to move according to the optimized movement trajectory in order to perform road condition sensing.

[0137] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Python, Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0138] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0139] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0140] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described road condition perception method. This solves the technical problems of related technologies being unable to cope with suddenly appearing animals and unable to identify stationary or slow-moving vehicles ahead. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the road condition perception method provided in the above embodiments, and will not be repeated here.

[0141] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the road condition perception method described above.

[0142] The computer program product provided in this application can solve the technical problems of related technologies being unable to cope with suddenly appearing animals and unable to identify stationary or slow-moving vehicles ahead. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the road condition perception method provided in the above embodiments, and will not be repeated here.

[0143] All user-related data involved in this application (such as user privacy data, user behavior data, etc.) were obtained with the user's permission or consent; that is to say, when this application is used in a specific product or technology, user permission is required to obtain and process the relevant data, and the processing of the relevant data must comply with the relevant laws, regulations and regulatory standards of the relevant countries and regions.

[0144] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

[0145] This application also discloses A1, a road condition perception method, the road condition perception method comprising: A risk cost map is constructed based on aircraft perception data and vehicle perception data. Plan the global movement trajectory based on the aforementioned risk cost map; The global movement trajectory is locally optimized to generate an optimized movement trajectory; The aircraft and / or vehicle are controlled to move according to the optimized movement trajectory in order to achieve road condition perception.

[0146] A2. The road condition perception method as described in A1, wherein constructing a risk cost map based on aircraft perception data and vehicle perception data includes: The aircraft's sensing data is projected and transformed to generate projected sensing data; An environmental map is generated based on the projection perception data and the vehicle perception data; The environmental map is marked with occupancy probability to generate a risk cost map.

[0147] A3. The road condition perception method as described in A2, wherein the projection transformation of the aircraft perception data to generate projected perception data includes: The aircraft perception data is divided into flat terrain perception data and non-flat terrain perception data. The flat terrain sensing data is projected onto the ground to generate first projection data; The non-flat terrain sensing data is back-projected to generate second projection data; Projection perception data is constructed based on the first projection data and the second projection data.

[0148] A4. The road condition perception method as described in A2, wherein generating an environmental map based on the projected perception data and the vehicle perception data includes: The projection sensing data is spatiotemporally aligned to generate aligned sensing data. An environment map is generated based on the alignment perception data and the vehicle perception data.

[0149] A5. The road condition perception method as described in A2, wherein the step of marking the occupancy probability of the environmental map and generating a risk cost map includes: Static element modeling is performed based on the projection perception data and the vehicle perception data to generate the static occupancy probability of each grid in the environment map. Based on the projection perception data and the vehicle perception data, dynamic element monitoring is performed to generate the dynamic occupancy probability of each grid in the environment map. Based on the static occupancy probability and the dynamic occupancy probability, construct the occupancy probability parameters corresponding to each grid in the environment map; The environmental map is marked based on the occupancy probability parameter to generate a risk cost map.

[0150] A6. The road condition perception method as described in A5, wherein the step of performing static element modeling based on the projected perception data and the vehicle perception data to generate the static occupancy probability corresponding to each grid in the environment map includes: Based on the projection perception data and the vehicle perception data, a probability occupancy calculation is performed on the environmental map to generate occupancy assessment parameters. The occupancy assessment parameters are normalized to generate the static occupancy probability corresponding to each grid cell in the environment map.

[0151] A7. The road condition perception method as described in A6, wherein the step of dynamically monitoring elements based on the projection perception data and the vehicle perception data to generate the dynamic occupancy probability corresponding to each grid in the environmental map includes: The projection perception data and the vehicle perception data are fused to generate fused features; Dynamic element observation data is constructed based on the aforementioned fusion features; Based on the dynamic element observation data, the dynamic occupancy probability of each grid cell in the environment map is constructed.

[0152] A8. The road condition perception method as described in A1, wherein planning the global movement trajectory based on the risk cost map includes: Based on the path planning algorithm, at least one path to be detected is constructed according to the path start point and the path end point; The path cost weight corresponding to each path to be detected is determined based on the risk cost map. A global movement trajectory is selected from the at least one path to be detected based on the path cost weight.

[0153] A9. The road condition perception method as described in A1, wherein the step of locally optimizing the global movement trajectory to generate an optimized movement trajectory includes: Generate a sequence of control commands based on the global movement trajectory; Construct a predicted trajectory based on the control command sequence; Select optimizable points from the predicted trajectory; Local optimization is performed on the control instructions corresponding to the optimizable points in the control instruction sequence to generate an optimized instruction sequence; An optimized movement trajectory is generated based on the optimized instruction sequence.

[0154] A10. The road condition perception method as described in A9, wherein selecting optimizable points in the predicted trajectory includes: The cost information corresponding to the predicted trajectory is calculated using a preset cost function; Points in the predicted trajectory whose corresponding cost information is greater than a preset cost threshold are selected as points that can be optimized.

[0155] A11. The road condition perception method as described in any one of A1-A10, wherein the aircraft perception data is sent by the aircraft when it determines that the increase in the amount of observation information is greater than a preset threshold.

[0156] This application also discloses B12, a road condition sensing device, the road condition sensing device comprising: The module is used to build a risk cost map based on aircraft perception data and vehicle perception data. The planning module is used to plan the global movement trajectory based on the risk cost map; The optimization module is used to perform local optimization on the global movement trajectory to generate an optimized movement trajectory; The control module is used to control the movement of the aircraft and / or vehicle according to the optimized movement trajectory in order to perform road condition perception.

[0157] B13. The road condition perception device as described in B12, wherein the construction module is further configured to perform projection conversion on the aircraft perception data to generate projection perception data; generate an environmental map based on the projection perception data and the vehicle perception data; and mark the environmental map with occupancy probability to generate a risk cost map.

[0158] B14. The road condition sensing device as described in B13, wherein the construction module is further configured to divide the aircraft sensing data into flat terrain sensing data and non-flat terrain sensing data; project the flat terrain sensing data onto the ground to generate first projection data; perform back-projection processing on the non-flat terrain sensing data to generate second projection data; and construct projection sensing data based on the first projection data and the second projection data.

[0159] B15. The road condition sensing device as described in B13, wherein the construction module is further configured to perform spatiotemporal alignment processing on the projected sensing data to generate aligned sensing data; and generate an environmental map based on the aligned sensing data and the vehicle sensing data.

[0160] B16. The road condition perception device as described in B13, wherein the construction module is further configured to: perform static element modeling based on the projection perception data and the vehicle perception data to generate static occupancy probabilities corresponding to each grid in the environmental map; perform dynamic element monitoring based on the projection perception data and the vehicle perception data to generate dynamic occupancy probabilities corresponding to each grid in the environmental map; construct occupancy probability parameters corresponding to each grid in the environmental map based on the static occupancy probabilities and the dynamic occupancy probabilities; and mark the environmental map based on the occupancy probability parameters to generate a risk cost map.

[0161] B17. The road condition sensing device as described in B16, wherein the construction module is further configured to perform probability occupancy calculation on the environmental map based on the projection sensing data and the vehicle sensing data, and generate occupancy assessment parameters; and to normalize the occupancy assessment parameters to generate static occupancy probabilities corresponding to each grid in the environmental map.

[0162] This application also discloses C18, a road condition sensing device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the road condition sensing method as described above.

[0163] This application also discloses D19, a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the road condition perception method as described above.

[0164] This application also discloses E20, a computer program product comprising a computer program that, when executed by a processor, implements the steps of the road condition perception method as described above.

Claims

1. A road condition sensing method, characterized in that, The road condition perception method includes: A risk cost map is constructed based on aircraft perception data and vehicle perception data. Plan the global movement trajectory based on the aforementioned risk cost map; The global movement trajectory is locally optimized to generate an optimized movement trajectory; The aircraft and / or vehicle are controlled to move according to the optimized movement trajectory in order to achieve road condition perception.

2. The road condition perception method as described in claim 1, characterized in that, The construction of the risk cost map based on aircraft perception data and vehicle perception data includes: The aircraft's sensing data is projected and transformed to generate projected sensing data; An environmental map is generated based on the projection perception data and the vehicle perception data; The environmental map is marked with occupancy probability to generate a risk cost map.

3. The road condition perception method as described in claim 2, characterized in that, The projection transformation of the aircraft's sensing data to generate projected sensing data includes: The aircraft perception data is divided into flat terrain perception data and non-flat terrain perception data. The flat terrain sensing data is projected onto the ground to generate first projection data; The non-flat terrain sensing data is back-projected to generate second projection data; Projection perception data is constructed based on the first projection data and the second projection data.

4. The road condition perception method as described in claim 2, characterized in that, The step of generating an environmental map based on the projected perception data and the vehicle perception data includes: The projection sensing data is spatiotemporally aligned to generate aligned sensing data. An environment map is generated based on the alignment perception data and the vehicle perception data.

5. The road condition perception method as described in claim 2, characterized in that, The step of marking the occupancy probability of the environment map and generating a risk cost map includes: Static element modeling is performed based on the projection perception data and the vehicle perception data to generate the static occupancy probability of each grid in the environment map. Based on the projection perception data and the vehicle perception data, dynamic element monitoring is performed to generate the dynamic occupancy probability of each grid in the environment map. Based on the static occupancy probability and the dynamic occupancy probability, construct the occupancy probability parameters corresponding to each grid in the environment map; The environmental map is marked based on the occupancy probability parameter to generate a risk cost map.

6. The road condition perception method as described in claim 5, characterized in that, The step of modeling static elements based on the projection perception data and the vehicle perception data to generate the static occupancy probability corresponding to each grid in the environment map includes: Based on the projection perception data and the vehicle perception data, a probability occupancy calculation is performed on the environmental map to generate occupancy assessment parameters. The occupancy assessment parameters are normalized to generate the static occupancy probability corresponding to each grid cell in the environment map.

7. A road condition sensing device, characterized in that, The road condition sensing device includes: The module is used to build a risk cost map based on aircraft perception data and vehicle perception data. The planning module is used to plan the global movement trajectory based on the risk cost map; The optimization module is used to perform local optimization on the global movement trajectory to generate an optimized movement trajectory; The control module is used to control the movement of the aircraft and / or vehicle according to the optimized movement trajectory in order to perform road condition perception.

8. A road condition sensing device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the road condition perception method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the road condition perception method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the road condition perception method as described in any one of claims 1 to 6.

Images