A method for obstacle avoidance trajectory optimization based on flight embodied intelligence

Abstract

This invention relates to the field of path planning and decision-making technology, specifically to a method for optimizing obstacle avoidance trajectories in flight embodied intelligence. The specific implementation steps include: first, acquiring perception data, identifying obstacle information, and assigning weights to sensors based on a pre-constructed physical characteristic-sensor confidence table; then, generating fused perception data using a Kalman filter and conflict resolution mechanism; next, generating an obstacle trajectory probability map and a semantic segmentation map of movable regions by constructing a multi-task learning model and sharing an encoder, and using a predictive segmentation collaborative optimization mechanism; obtaining an initial obstacle avoidance path based on gradient descent; generating a risk cost by calculating the spatiotemporal collision probability of the initial obstacle avoidance path overlapping with the obstacle trajectory probability map, and assigning collision weights to obstacles; finally, constructing a comprehensive risk based on the spatiotemporal collision probability, risk cost, and collision weights, and inputting it into the RRT* algorithm to generate the optimal obstacle avoidance path.

Description

Technical Field

[0001] This invention relates to the field of path planning and decision-making technology, specifically to a method for optimizing obstacle avoidance trajectories using flight embodied intelligence. Background Technology

[0002] Obstacle avoidance is a critical and challenging problem when flying in complex environments. Traditional obstacle avoidance methods typically rely on single sensor data, such as lidar or visual sensors, for obstacle detection and path planning. However, single sensor data often has limitations. For example, lidar is susceptible to adverse weather conditions, while visual sensors perform poorly in environments with low light or lack of texture, resulting in incomplete perception information.

[0003] To overcome the limitations of single sensors, some methods fuse data from multiple sensors. However, these methods lack dynamic fusion weighting mechanisms and cannot adaptively adjust based on different scenarios and sensor confidence levels, thus affecting the robustness and accuracy of the fused perception data. Furthermore, existing obstacle avoidance methods treat semantic segmentation and obstacle trajectory prediction as independent tasks or process them sequentially, resulting in low computational efficiency and difficulty in fully utilizing the potential correlation between the two, thereby limiting the accuracy of movable area boundary correction and trajectory prediction.

[0004] During the path planning phase, some methods only consider the current position of obstacles while ignoring their future trajectories, which can lead to collisions between the planned path and future moving obstacles. The severity of the consequences of collisions varies depending on the type of obstacle, but existing methods do not adequately consider the impact of obstacle type on collision risk assessment, resulting in suboptimal obstacle avoidance path selection. Therefore, existing methods still have shortcomings in handling the correlation between multimodal fusion, semantic understanding, and behavior prediction, as well as the comprehensiveness of risk assessment.

[0005] To address this, a method for optimizing obstacle avoidance trajectories based on flight embodied intelligence is proposed. Summary of the Invention

[0006] The purpose of this invention is to provide a method for optimizing obstacle avoidance trajectories in flight-integrated intelligence. To address the problems existing in the prior art, this invention first acquires perception data, detects obstacle information using an obstacle recognition model, and assigns weights to sensors based on a pre-constructed physical characteristic-sensor confidence table. Then, using a Kalman filter and conflict resolution mechanism, fused perception data is generated. Next, a multi-task learning model is constructed and an encoder is shared. A predictive segmentation collaborative optimization mechanism is used to generate an obstacle trajectory probability map and a semantic segmentation map of the movable region, and gradient descent is used to obtain an initial obstacle avoidance path. Finally, the spatiotemporal collision probability of the initial obstacle avoidance path overlapping with the obstacle trajectory probability map is calculated, generating a risk cost. Collision weights are determined based on the obstacle category recognition results. A comprehensive risk is constructed based on the spatiotemporal collision probability, risk cost, and collision weights, and input into the RRT* algorithm to generate the optimal obstacle avoidance path.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for optimizing obstacle avoidance trajectory in flight-integrated intelligence, the method comprising:

[0008] Obstacle perception data is acquired through multiple sensors; a physical characteristic-sensor confidence table is constructed based on the impact of physical characteristics on sensor performance, and weights are assigned to multiple sensors through the physical characteristic-sensor confidence table; at the same time, a conflict resolution mechanism is constructed to adjust the weights of sensors with the same physical characteristics but different recognition results; the weights of multiple sensors and perception data are input into a Kalman filter to generate fused perception data.

[0009] A multi-task learning model with an embedded prediction and segmentation collaborative optimization mechanism is constructed. Based on fused perception data, an obstacle trajectory probability map and a semantic segmentation map of the movable region are generated, and gradient descent is performed to obtain the initial obstacle avoidance path.

[0010] Calculate the spatiotemporal collision probability of the initial obstacle avoidance path overlapping with the obstacle trajectory probability map, and generate the risk cost; assign collision weights to each obstacle based on the consequences of the flight avatar intelligence colliding with the obstacle; calculate the weighted average of the spatiotemporal collision probability, risk cost, and collision weights to obtain the comprehensive risk, and input it into the RRT* algorithm to generate the optimal obstacle avoidance path.

[0011] Preferably, the specific implementation process of acquiring obstacle perception data through multiple sensors includes: acquiring raw perception data through the flight-integrated intelligent LiDAR, visual sensor, and IMU attitude sensor; and processing the raw perception data in parallel by a lightweight, real-time optimized obstacle recognition model to detect the physical characteristics of the obstacle.

[0012] Preferably, a physical characteristic-sensor confidence table is constructed based on the impact of physical characteristics on sensor performance, and weights are assigned to each sensor through the physical characteristic-sensor confidence table. The specific implementation process includes: listing physical characteristics affecting sensor performance, including materials such as metal, glass, wood, plastic, and fabric; regular and irregular geometric shapes; and weather conditions affecting sensor performance, including sunny days, rainy days, and foggy days; setting evaluation values ​​between 1 and 10 for the confidence of LiDAR, vision sensors, and IMU attitude sensors in recognizing the physical characteristics; establishing the physical characteristic-sensor confidence table; using a machine learning model to learn the physical characteristic-sensor confidence table; inputting the physical characteristics of obstacles into the machine learning model; and generating weights for each sensor in real time.

[0013] Preferably, a conflict resolution mechanism is constructed to adjust the weights of sensors with the same physical characteristics but different recognition results. The specific implementation process of inputting the weights of each sensor and the perception data into a Kalman filter to generate fused perception data includes: when the difference between the recognition results of the same physical characteristics by the lidar and the vision sensor exceeds a threshold, the conflict resolution mechanism is dynamically used. The conflict resolution mechanism is verified using an IMU attitude sensor. At the same time, priority is assigned based on confidence level, and the weights are adjusted through priority. The sensor weights and perception data are input into a Kalman filter to generate fused perception data.

[0014] Preferably, the specific implementation process of constructing a multi-task learning model with an embedded prediction and segmentation collaborative optimization mechanism includes: receiving fused perception data from multiple sensors through an input layer; constructing a shared encoder to extract features from the input fused perception data, and simultaneously inputting them into a semantic segmentation decoder and an obstacle trajectory prediction decoder in a parallel decoder; the semantic segmentation decoder uses the features to output a semantic segmentation map of the movable region; and the obstacle trajectory prediction decoder uses the features to predict the motion trajectory of the obstacle and outputs an obstacle trajectory probability map.

[0015] Preferably, the specific implementation process of generating an obstacle trajectory probability map and a movable region semantic segmentation map based on fused sensing data, and obtaining an initial obstacle avoidance path through gradient descent includes: the fused sensing data is processed by a shared encoder and two parallel decoders to generate a movable region semantic segmentation map and an obstacle trajectory probability map; the obstacle trajectory probability map output by the obstacle trajectory prediction decoder is fed back to the semantic segmentation decoder, which uses the feedback information to adjust the segmentation of the obstacle region and the movable region; the loss of semantic segmentation and obstacle trajectory prediction is adjusted through a collaborative loss function to generate an optimized obstacle trajectory probability map and a movable region semantic segmentation map; and an initial obstacle avoidance path is obtained based on the obstacle trajectory probability map and the movable region semantic segmentation map using the gradient descent method.

[0016] Preferably, the loss of semantic segmentation and obstacle trajectory prediction is adjusted by a collaborative loss function to generate an optimized obstacle trajectory probability map and a movable region semantic segmentation map. An initial obstacle avoidance path is obtained based on the obstacle trajectory probability map and the movable region semantic segmentation map using a gradient descent method. The specific implementation process includes: the judgment rules of the collaborative loss function are as follows: increase the loss when there is a conflict between the movable region and the trajectory prediction; decrease the loss when the future trajectory prediction region overlaps with the movable region semantic segmentation map; after the collaborative loss function and the multi-task learning model, a movable region segmentation map and an obstacle trajectory probability map are obtained; and the initial obstacle avoidance path is calculated using the gradient descent method based on the movable region segmentation map and the obstacle trajectory probability map.

[0017] Preferably, the specific implementation process for calculating the spatiotemporal collision probability of the initial obstacle avoidance path overlapping with the obstacle trajectory probability map and generating the risk cost includes: decomposing the initial obstacle avoidance path into a series of discrete path points and discretizing the time; for each point on the path and each future time step, determining the spatiotemporal region that the flight vehicle will occupy; for each spatiotemporal point on the planned path, querying the probability value of the obstacle's future trajectory probability map in the spatiotemporal region, performing numerical integration on the probability map of the region to obtain the collision probability; accumulating the probabilities of all spatiotemporal points along the initial obstacle avoidance path that overlap with the obstacle trajectory to obtain the total spatiotemporal collision probability of the initial obstacle avoidance path; and generating the risk cost based on the total spatiotemporal collision probability and the dispersion of the obstacle trajectory probability map.

[0018] Preferably, obstacle collision weights are assigned based on the consequences of collisions between the flight vehicle's intelligence and obstacles; a comprehensive risk is obtained by weighted averaging of the spatiotemporal collision probability, risk cost, and collision weights, and then input into the RRT* algorithm to generate the optimal obstacle avoidance path. The specific implementation process includes: based on prior knowledge of the severity of collisions between the flight vehicle and different types of obstacles, the consequences of collisions with different obstacle categories are predefined; based on the obstacle categories identified by the obstacle recognition model and the consequences of collisions with those obstacle categories, corresponding collision weights are assigned to obstacles in the current scene; a comprehensive risk value is obtained by weighted averaging of the spatiotemporal collision probability, risk cost, and collision weights, and then input into the RRT* algorithm to generate the optimal obstacle avoidance path that minimizes the comprehensive risk.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0020] 1. The sensor data fusion method proposed in this invention improves the robustness and accuracy of sensor data. Existing technologies lack a dynamic weighting mechanism when processing multi-sensor data fusion, failing to adaptively adjust weights based on different scenarios and sensor confidence levels, thus affecting the robustness and accuracy of the fused sensor data. This invention dynamically assigns fusion weights to sensors by combining a pre-constructed physical characteristic-sensor confidence table, and utilizes a conflict resolution mechanism to adaptively adjust the contribution of different sensors and resolve conflicts, generating more robust and accurate fused sensor data through a Kalman filter.

[0021] 2. The predictive segmentation co-optimization mechanism proposed in this invention improves the efficiency and accuracy of environmental understanding and trajectory prediction. Existing technologies treat semantic segmentation and trajectory prediction as independent tasks or process them sequentially, resulting in low efficiency and difficulty in leveraging their correlation. This invention constructs a multi-task learning model and shares an encoder to achieve parallel processing of semantic segmentation and obstacle trajectory prediction. This not only avoids redundant computation and promotes feature generality, but also uses the predictive segmentation co-optimization mechanism to correct the semantic segmentation map of the movable region using the trajectory prediction result, ensuring logical consistency between the two, thereby improving the accuracy of environmental understanding and the reliability of trajectory prediction.

[0022] 3. The obstacle avoidance path method proposed in this invention optimizes the safety and reliability of obstacle avoidance paths. Existing technologies ignore the uncertainty of future obstacle trajectories and collision consequences, failing to fully quantify the uncertainty of obstacle trajectories and consider the impact of obstacle categories on collision risk assessment. This invention calculates the spatiotemporal collision probability where the initial obstacle avoidance path overlaps with the obstacle trajectory probability map, generating a risk cost and comprehensively considering future collision risks. Simultaneously, it determines collision weights based on obstacle category identification results, enabling obstacle avoidance decisions to distinguish the degree of harm of different obstacles. Finally, an optimal obstacle avoidance path is generated based on the comprehensive risk constituted by spatiotemporal collision probability, risk cost, and collision weights, making the planned path safer and more optimized. Attached Figure Description

[0023] Figure 1 A flowchart illustrating a method for optimizing obstacle avoidance trajectory using flight-integrated intelligence;

[0024] Figure 2 This is a flowchart illustrating the parallel processing of semantic segmentation of fused sensing data and obstacle trajectory prediction proposed in an embodiment of this invention.

[0025] Figure 3 This is a design diagram of the conflict penalty for the collaborative loss function proposed in an embodiment of this invention application;

[0026] Figure 4 This is a design diagram of the consensus incentive for the collaborative loss function proposed in an embodiment of this invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Please see Figures 1-4 This invention proposes a method for optimizing obstacle avoidance trajectory based on flight-integrated intelligence, the technical solution of which is as follows:

[0029] A method for obstacle avoidance trajectory optimization in flight embodied intelligence, referencing Figure 1 The specific implementation steps of the method proposed in this invention include:

[0030] S1. Acquire perception data through sensors of flight-integrated intelligence, and detect obstacle information in the perception data by obstacle recognition model; combine with the pre-built physical characteristics-sensor confidence table to assign weights to the sensors;

[0031] S2. Input the sensing data and weights into the Kalman filter and use the conflict resolution mechanism to generate fused sensing data.

[0032] S3. Construct a multi-task learning model and share the encoder to achieve parallel processing of semantic segmentation of fused perception data and obstacle trajectory prediction;

[0033] S4. Utilize the predictive segmentation collaborative optimization mechanism to correct the semantic segmentation map of the movable area based on the trajectory prediction results, while ensuring that the two are logically consistent, to generate the obstacle trajectory probability map and the semantic segmentation map of the movable area, and obtain the initial obstacle avoidance path.

[0034] S5. Based on the obstacle trajectory probability map, calculate the spatiotemporal collision probability of the initial obstacle avoidance path overlapping with the obstacle trajectory probability map, and generate the risk cost;

[0035] S6. Combine the obstacle category identification results to determine the collision weight, and construct a comprehensive risk based on the spatiotemporal collision probability, risk cost and collision weight, and input it into the RRT* algorithm to generate the optimal obstacle avoidance path.

[0036] Example 1

[0037] This embodiment provides a specific application of a flight-integrated intelligence obstacle avoidance trajectory optimization method. Its typical application scenario is for quadcopter drones used for urban express delivery, which optimizes obstacle avoidance path planning to ensure flight safety.

[0038] Furthermore, sensory data is acquired through the sensors of the flight-embedded intelligence, and obstacle information in the sensory data is detected by the obstacle recognition model; weights are assigned to the sensors by combining the pre-built physical characteristics-sensor confidence table; corresponding to step S1 above, the specific process is as follows:

[0039] During flight, the UAV uses a lidar scanner to scan the environment at a frequency of 10Hz to acquire 3D point cloud data. An RGB-D camera acquires RGB images and depth maps at a frequency of 30Hz. Simultaneously, an IMU outputs attitude, angular velocity, and acceleration data at a frequency of 100Hz. The raw perception data is input into a pre-trained, lightweight, real-time optimized obstacle recognition model. Utilizing multi-scale feature fusion, it can process data from different sensors in parallel. By collecting multi-sensor data on the physical characteristics of various obstacles such as birds, trees, and buildings (e.g., metal, glass, pedestrians) under different weather conditions and combining this data with prior knowledge, a detailed physical characteristic-sensor confidence table is established. A machine learning model based on a Transformer architecture is used to learn the mapping relationship. This model takes the preliminary physical characteristic recognition results of the obstacles as input and outputs dynamic weights for each sensor mode. The identified obstacle physical characteristics are input into the machine learning model, and the output dynamic weights adjust the contribution of different sensors to specific physical characteristics in real time.

[0040] By using a lightweight, real-time optimized obstacle recognition model to process raw perception data in parallel, the system can quickly and accurately detect the type, location, and size of obstacles, while simultaneously extracting multi-scale, semantically rich feature representations, providing high-quality input for subsequent perception fusion and decision-making.

[0041] Specifically, a pre-built physical characteristic-sensor confidence table is used to dynamically assign fusion weights to sensors, and the process is as follows:

[0042] Traditional Kalman filter weights are based on the covariance of sensor measurement noise, which is static and preset. Sensor weights are dynamically adjusted according to the physical characteristics of obstacles. Based on semantic judgment, radar is more reliable than camera when detecting metal, while camera has a higher weight when detecting glass.

[0043] Based on semantic understanding, dynamic, and context-aware weight adjustment, it surpasses the relatively static and statistically noise-based weight allocation methods in traditional Kalman filters.

[0044] By constructing a physical property-sensor confidence table and using a machine learning model, dynamic weights of each sensor modality can be generated in real time based on the preliminary physical properties of obstacles. This enables the model to adaptively learn the optimal contribution of different sensors to different physical properties in different scenarios, thereby improving the adaptability and accuracy of fusion perception.

[0045] Furthermore, the perceived data and weights are input into a Kalman filter, and fused perceived data is generated using a conflict resolution mechanism. Corresponding to step S2 above, the specific process includes:

[0046] When conflicting identification results from different modal sensors occur—that is, when the differences in judgments by different sensors regarding the category, location, size, or motion state of the same obstacle exceed a threshold—a dynamic conflict resolution strategy is employed. This strategy utilizes other sensors for verification. Simultaneously, high-confidence sensors have higher priority in physical characteristic identification. Their position in the conflict is determined by the accuracy of each sensor's judgment of a particular physical characteristic, and their weights are adjusted accordingly. For example, if the radar's confidence in a stationary object exceeds a certain threshold, and the visual sensor's confidence in classifying the same object as moving is below a certain threshold, the radar's "stationary" judgment is forcibly adopted, and the visual sensor's relevant weight is reduced to zero or a very low value. These weights are then input into a Kalman filter. In the Kalman filter, each sensor's measurement has an associated noise covariance matrix. Higher-weighted sensor measurements are more reliable, and their corresponding noise covariance matrix elements are reduced; lower-weighted measurements are less reliable, and their corresponding noise covariance matrix elements are amplified. Thus, the higher the weight, the more the Kalman filter "trusts" the sensor's data during state updates.

[0047] The Kalman gain determines the weighting ratio between the predicted state and the current measurement. When the sensor weight is increased, the elements of its measurement noise covariance matrix decrease accordingly, leading to an increase in the Kalman gain and thus enhancing the sensor's ability to correct the final state estimate. Conversely, a low-weight sensor has a smaller impact on the state estimate.

[0048] By utilizing conflict resolution mechanisms and dynamically employing conflict resolution strategies, and allowing some sensors to overwhelmingly reject low-confidence detections from other sensors in certain physical characteristic identifications, the differences between identification results from different sensors can be effectively handled. Weights can be adjusted to ensure the robustness and accuracy of fusion, thereby guiding the Kalman filter to generate more reliable fused sensing data.

[0049] Furthermore, a multi-task learning model is constructed and the encoder is shared to achieve parallel processing of semantic segmentation of fused perceptual data and obstacle trajectory prediction. Corresponding to step S3 above, the specific process includes:

[0050] Fusion-sensory data is fed into a multi-task learning model. The model's shared encoder extracts high-level, semantically rich feature representations from the fusion-sensory data. The shared features output by the encoder are branched into two independent decoder branches. The semantic segmentation decoder transforms the shared feature map into a pixel-level semantic segmentation map, clearly distinguishing obstacle regions, movable regions, and background categories. The trajectory prediction decoder utilizes the shared features and the model's understanding of the obstacle's current state to predict the obstacle's future trajectory, outputting a probability map of the obstacle's future trajectory.

[0051] By constructing a multi-task learning model and sharing the encoder, parallel processing of semantic segmentation and obstacle trajectory prediction based on fused perceptual data was achieved. This ensures that the two tasks can share underlying environmental understanding and object information, avoids redundant computation, promotes the generality of features, and improves the efficiency and performance of the overall system.

[0052] Furthermore, by utilizing a predictive segmentation collaborative optimization mechanism, the trajectory prediction results are used to correct the semantic segmentation map of the movable region, while ensuring logical consistency between the two. This generates an obstacle trajectory probability map and a semantic segmentation map of the movable region, resulting in an initial obstacle avoidance path. Corresponding to step S4 above, the specific process includes:

[0053] The fused perception data is processed by a shared encoder and two parallel decoders to initially generate a semantic segmentation map and a trajectory probability map. The output of the trajectory prediction decoder is fed back to the semantic segmentation decoder, which uses this feedback information to dynamically adjust the boundaries of the movable region. Then, a collaborative loss function is designed to penalize conflicts between the movable region and the trajectory prediction. Through deep collaborative training, the model obtains the movable region segmentation map and the obstacle trajectory probability map. Finally, based on the movable region segmentation map and the obstacle trajectory probability map, an initial obstacle avoidance path is obtained using the gradient descent method.

[0054] Specifically, the semantic segmentation decoder dynamically adjusts the boundaries of the movable region using the output of the trajectory prediction decoder. The processing procedure is as follows:

[0055] After receiving feedback information from trajectory prediction, the semantic segmentation decoder dynamically adjusts its classification decisions for obstacle regions and movable regions based on this feedback. When the trajectory prediction model predicts an extremely low probability of obstacle movement in a region initially identified as a movable region by semantic segmentation, it indicates that the region is not truly movable, and the semantic segmentation decoder tends to reclassify the pixels in that region as either an obstacle region or a non-movable region. Conversely, when semantic segmentation initially identifies a region as an obstacle region, and trajectory prediction displays a high probability of the obstacle's future trajectory within that region, it indicates that the obstacle is movable and the segmentation boundary is too conservative, and the semantic segmentation decoder tends to reclassify the pixels in that region as a movable region.

[0056] By dynamically adjusting the boundaries of the movable area using trajectory prediction, the semantic segmentation decoder is no longer an isolated module. It can dynamically and intelligently utilize future behavior information from trajectory prediction to generate more accurate and realistic movable area boundaries, significantly improving the flight embodied intelligence's ability to understand complex environments.

[0057] By feeding trajectory prediction results back to the semantic segmentation decoder, which can dynamically adjust the boundaries of the movable region, ambiguity caused by fuzziness and uncertainty in the fused sensing data is eliminated, thereby improving the accuracy and reliability of movable region segmentation.

[0058] Specifically, a collaborative loss function is designed, and its processing procedure is as follows:

[0059] A conflict penalty term is designed to penalize trajectory prediction for generating high-probability trajectories within semantically segmented regions marked as immovable or occupied by obstacles. A binary mask is obtained from the semantic segmentation map, and numerical integration is performed on the immovable regions on the trajectory probability map output by trajectory prediction. When the probability value within the region is high, indicating a high probability that the model predicts an obstacle will enter the region, a penalty is applied. A consistency incentive term is designed to extract high-probability regions that obstacles may occupy in the future from the trajectory prediction probability map, and the intersection-union ratio (IUU) is used to measure the degree of overlap between the high-probability regions and the semantically segmented movable regions. The higher the overlap, the lower the loss. An uncertainty alignment term is designed to penalize trajectory prediction for regions with high semantic segmentation uncertainty when the trajectory prediction gives a very definite trajectory.

[0060] The model considers the rationality of trajectory prediction when optimizing semantic segmentation, and vice versa. This allows the model to learn a deeper level of intrinsic correlation and synergy between the two tasks, rather than simply performing the two tasks independently.

[0061] The collaborative loss function is designed to penalize conflicts between the movable region and trajectory prediction, encouraging high-probability trajectory prediction regions to maintain a high degree of overlap with the semantic segmentation map of the movable region. This ensures the logical consistency between semantic segmentation and trajectory prediction, resulting in a more accurate movable region segmentation map and a more reliable obstacle trajectory probability map, thus providing a guarantee for initial path planning.

[0062] Further, based on the obstacle trajectory probability map, the spatiotemporal collision probability of the initial obstacle avoidance path overlapping with the obstacle trajectory probability map is calculated, and a risk cost is generated. Corresponding to step S5 above, the specific process includes:

[0063] The initial obstacle avoidance path is decomposed into a series of discrete path points, and time is discretized. For each point on the path and each future time step, the spatiotemporal region that the flight embodied intelligence will occupy is determined. The probability value of the obstacle trajectory probability map in that spatiotemporal region is queried, and the spatiotemporal collision probability of the predicted future trajectories overlapping the path is accumulated. The probabilities of all spatiotemporal points along the initial obstacle avoidance path that overlap with the obstacle trajectory are accumulated to obtain the total spatiotemporal collision probability of the initial obstacle avoidance path. Based on the dispersion of the obstacle future trajectory probability map, the risk cost is quantified and generated. The higher the entropy value, the more ambiguous the prediction, and the higher the risk cost.

[0064] By calculating the spatiotemporal collision probability of the initial obstacle avoidance path overlapping with the obstacle trajectory probability map, and generating risk costs based on the discrete quantification of the obstacle trajectory probability map, the spatiotemporal risks of the path and the uncertainty of the prediction can be comprehensively assessed, providing a more refined and comprehensive risk assessment indicator for subsequent optimal path selection.

[0065] Furthermore, collision weights are determined by combining obstacle category identification results. A comprehensive risk is constructed based on spatiotemporal collision probability, risk cost, and collision weights, and then input into the RRT* algorithm to generate the optimal obstacle avoidance path. Corresponding to step S6 above, the specific process includes:

[0066] The severity weights of collision consequences for different obstacle categories are predefined. Based on the obstacle categories identified by the obstacle recognition model, obstacles in the current scene are assigned corresponding collision weights. A weighted average of the spatiotemporal collision probability, risk cost, and collision weights is used to form a comprehensive risk value. The weighting coefficients are dimensionless values ​​between 0 and 1, used to balance the importance of each risk factor in the comprehensive risk assessment. This comprehensive risk function is used as the cost function and input into the RRT* algorithm. Each time the search tree is expanded, the RRT* algorithm evaluates the comprehensive risk of the newly generated path segment and prioritizes the branch with the lowest risk. Finally, the algorithm generates the optimal obstacle avoidance path that minimizes the comprehensive risk.

[0067] The collision weight is determined by combining the obstacle category identification results, and the spatiotemporal collision probability, risk cost and collision weight are weighted and averaged to form a comprehensive risk value. This value is then used as a cost function to input into the RRT* algorithm, thereby generating the optimal obstacle avoidance path that minimizes the comprehensive risk. This ensures that the planned path not only avoids obstacles but also prioritizes avoiding high-risk collisions, significantly improving the safety of flight-integrated intelligence.

[0068] Example 2

[0069] This embodiment provides a specific application scenario for obstacle avoidance by a highway patrol drone, mainly used for highway traffic monitoring and rapid accident response. Its specific implementation method can be as follows:

[0070] Drones patrol at 150 km / h, with high-resolution visible light cameras capturing vehicle images and thermal imaging cameras identifying vehicles parked on the roadside due to malfunctions or not clearly visible in visible light at night or in foggy conditions. Long-range LiDAR provides information on road topology and large obstacles within several kilometers ahead. The physical characteristics-sensor confidence table incorporates physical characteristics such as flight speed, environmental visibility, and target speed; at high speeds, the confidence weight for long-range sensors is higher. For high-speed scenarios, the "motion consistency" principle is introduced. When LiDAR detects a stationary object ahead, but visible light blurs due to rapid movement and misjudges it as a moving object, the LiDAR's stationary judgment receives higher weight, ensuring accurate identification.

[0071] In this multi-task learning model, semantic segmentation and obstacle trajectory prediction are processed in parallel, segmenting areas such as "highway surface," "vehicles," "median strip," and "guardrail." For high-speed moving vehicles, the model predicts their complex behaviors within the lane, including lane changes, overtaking, and deceleration, integrating a traffic flow model into its trajectory prediction model. The model identifies vehicles on the road and performs semantic segmentation within 0.1 seconds, while simultaneously predicting the trajectories of multiple vehicles within a 500-meter radius ahead for the next 3 seconds. For vehicles that brake suddenly, the trajectory prediction can quickly correct its predicted trajectory and increase the predicted collision probability to an acceptable threshold within 1 second, triggering obstacle avoidance decisions.

[0072] By constructing a physical characteristic-sensor confidence table and using a machine learning model to learn the table to generate dynamic fusion weights, the sensor fusion process becomes more intelligent and adaptive. This solves the problem that static or empirical weights cannot adapt to changing environments, enabling the system to maximize the advantages of each sensor in different scenarios and improve the accuracy of fused sensing data. Utilizing conflict resolution mechanisms and an "overwhelming denial" strategy, contradictory information between sensors can be effectively identified and processed, and weights can be dynamically adjusted to ensure the robustness and accuracy of fusion, avoiding errors in sensing results due to local sensor failures or misjudgments.

[0073] A multi-task learning model was constructed, enabling two key tasks to be processed in parallel and sharing features, significantly improving computational efficiency. Simultaneously, sharing underlying features also promoted collaborative learning between tasks, enhancing the depth of understanding of the environment and obstacle motion. By utilizing a predictive segmentation collaborative optimization mechanism, the trajectory prediction results can dynamically correct the semantic segmentation map of the movable region, effectively eliminating ambiguity and uncertainty in the perceptual data, making the division of the movable region more accurate and consistent with actual motion requirements.

[0074] By constructing a physical characteristic-sensor confidence table and adding characteristics such as flight speed and environmental visibility, long-range sensors receive higher weight during high-speed flight. Furthermore, the "motion state consistency" principle is introduced to handle conflicts between lidar and visible light data, ensuring accurate identification. At the perception and decision-making level, a multi-task learning model processes semantic segmentation and trajectory prediction in parallel, enabling it to predict complex vehicle behaviors and quickly correct trajectories.

[0075] Example 3

[0076] In this invention, collision weights are determined by combining obstacle category identification results. Based on the comprehensive risk constituted by spatiotemporal collision probability, risk cost, and collision weights, an optimal obstacle avoidance path is generated. The specific implementation method can be as follows:

[0077] During the system design phase, extensive experimental data, simulations, and expert experience are required to predefine the severity weights of collision consequences for different obstacle categories. For different obstacle types, the system assigns a numerical value reflecting their danger level; the more severe the collision consequences, the higher the weight. An obstacle recognition model is used to classify obstacles in the current scene. Once the obstacle category is identified, the system assigns corresponding collision weights to the obstacles according to a predefined mapping table. Then, a weighting coefficient is added to the spatiotemporal collision probability, risk cost, and collision weight to balance the importance of these three factors in the comprehensive risk assessment. This coefficient is optimized based on the safety requirements of the actual application scenario and the system characteristics. Finally, a weighted average is performed to obtain the comprehensive risk value.

[0078] The calculated comprehensive risk function is used as the cost function and input into the RRT* path planning algorithm. During each iteration or expansion of the RRT* algorithm, it evaluates the comprehensive risk value of the newly generated path segment or the entire path and tends to select path branches that minimize the comprehensive risk. This generates the optimal obstacle avoidance path that minimizes the comprehensive risk.

[0079] By determining collision weights, the problem of treating all obstacles equally in existing technologies for obstacle avoidance decision-making is solved. A weighted average of spatiotemporal collision probability, risk cost, and collision weights is used to form a comprehensive risk value. This multi-dimensional risk assessment allows for a more comprehensive and detailed judgment of potential hazards. Finally, the comprehensive risk function is used as a cost function input into the RRT* algorithm, which generates the optimal obstacle avoidance path that minimizes the comprehensive risk. This not only makes obstacle avoidance decision-making more intelligent and optimized, prioritizing the avoidance of highly hazardous obstacles, but also significantly improves the flight safety of the flight embodied intelligence in complex dynamic environments.

[0080] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent obstacle avoidance trajectory optimization of an aerial vehicle, characterized in that, The application relates to an obstacle avoidance method based on multi-sensor fusion and multi-task learning. The method comprises the following steps: Obstacle perception data is acquired through multiple sensors; a physical property-sensor confidence table is constructed according to the influence of physical properties on sensor performance, and the multiple sensors are given weights through the physical property-sensor confidence table; A conflict resolution mechanism is constructed to adjust the weights of the sensors whose identification results are different for the same physical property; the multiple sensor weights and the perception data are input into a Kalman filter to generate fused perception data; A multi-task learning model with an embedded prediction segmentation cooperative optimization mechanism is constructed, the fused perception data is used to generate an obstacle trajectory probability graph and a movable region semantic segmentation graph, and gradient descent is performed to obtain an initial obstacle avoidance path, the process being as follows: The fused perception data is input into a shared encoder and two parallel decoders to generate a movable region semantic segmentation graph and an obstacle trajectory probability graph; the obstacle trajectory probability graph output by the obstacle trajectory prediction decoder is fed back to the semantic segmentation decoder, and the semantic segmentation decoder adjusts the segmentation of the obstacle region and the movable region according to the feedback information; a cooperative loss function is used to adjust the loss of semantic segmentation and obstacle trajectory prediction, and an optimized obstacle trajectory probability graph and a movable region semantic segmentation graph are generated; the initial obstacle avoidance path is obtained based on the gradient descent method according to the obstacle trajectory probability graph and the movable region semantic segmentation graph; the judgment rule of the cooperative loss function is that the loss is increased when the movable region and the trajectory prediction conflict, and the loss is reduced when the future trajectory prediction region and the non-movable region semantic segmentation graph remain overlapped; The movable region segmentation graph and the obstacle trajectory probability graph are obtained through the cooperative loss function and the multi-task learning model, and the initial obstacle avoidance path is calculated according to the movable region segmentation graph and the obstacle trajectory probability graph by using the gradient descent method; The spatiotemporal collision probability of the initial obstacle avoidance path overlapping the obstacle trajectory probability graph is calculated to generate a risk cost; The multiple obstacle collision weights are given according to the consequences of the collision between the flight body intelligence and the obstacles; 2.The method of claim 1, wherein, The comprehensive risk is obtained by weighted averaging the spatiotemporal collision probability, the risk cost and the collision weight, and is input into an RRT* algorithm to generate an optimal obstacle avoidance path. The specific implementation process of acquiring the perception data of the obstacles through the multiple sensors comprises the following steps: 3.The method of claim 1, wherein, Raw perception data is acquired through a laser radar, a vision sensor and an IMU attitude sensor of the flight body intelligence, and the raw perception data is processed in parallel by a lightweight and real-time optimized obstacle identification model to detect the physical properties of the obstacles. The specific implementation process of constructing a physical property-sensor confidence table according to the influence of physical properties on sensor performance and giving weights to the multiple sensors through the physical property-sensor confidence table comprises the following steps: A physical property-sensor confidence table is established by setting an evaluation value between 1 and 10 for the confidence of the laser radar, the visual sensor and the IMU attitude sensor to identify the physical properties by enumerating the physical properties affecting the performance of the sensor, including metal, glass, wood, plastic and fabric materials, regular geometric shapes and irregular body shapes, and weather environments affecting the performance of the sensor, including sunny days, rainy days and heavy fog, a machine learning model is used to learn the physical property-sensor confidence table, and the physical properties of the obstacles are input into the machine learning model to generate multiple sensor weights in real time. 4.The method of claim 1, wherein, The conflict resolution mechanism is constructed to adjust the sensor weights of the identification results of the same physical properties that are different; the multiple sensor weights and the perception data are input into the Kalman filter to generate fused perception data, and the specific implementation process includes: When the difference between the identification results of the same physical properties by the laser radar and the visual sensor exceeds the threshold value, the conflict resolution mechanism is dynamically used, the conflict resolution mechanism uses the IMU attitude sensor for verification, at the same time, the priority is assigned based on the confidence, and the weight is adjusted through the priority; the sensor weight and the perception data are input into the Kalman filter to generate fused perception data.

5. The method of claim 1, wherein, The specific implementation process of the multi-task learning model with embedded prediction segmentation cooperative optimization mechanism includes: The input layer receives the perception data fused from multiple sensors; the shared encoder extracts features from the input fused perception data, which are input into the semantic segmentation decoder and the obstacle trajectory prediction decoder in the parallel decoder, the semantic segmentation decoder outputs the movable area semantic segmentation map using the features; the obstacle trajectory prediction decoder predicts the motion trajectory of the obstacle using the features, and outputs the obstacle trajectory probability map.

6. The method of claim 1, wherein, The specific implementation process of calculating the spatiotemporal collision probability of the initial obstacle avoidance path overlapping with the obstacle trajectory probability map to generate the risk cost includes: The initial obstacle avoidance path is decomposed into a series of discrete path points, and the time is discretized, for each point and each future time step on the path, the spatiotemporal region occupied by the flight body is determined; for each spatiotemporal point on the planned path, the probability value of the obstacle future trajectory probability map in the spatiotemporal region is queried, the probability map of the region is numerically integrated to obtain the collision probability, the probabilities of all spatiotemporal points overlapping with the obstacle trajectory along the initial obstacle avoidance path are accumulated to obtain the total spatiotemporal collision probability of the initial obstacle avoidance path; the risk cost is generated based on the total spatiotemporal collision probability and the dispersion of the obstacle trajectory probability map.

7. The method of claim 1, wherein, The specific implementation process of assigning a collision weight to the obstacle according to the consequences of the collision between the flight body intelligence and the obstacle, weighting the spatiotemporal collision probability, the risk cost and the collision weight to obtain the comprehensive risk, and inputting the comprehensive risk into the RRT* algorithm to generate the optimal obstacle avoidance path includes: According to prior knowledge of the severity of the collision between the flight body and different categories of obstacles, the consequences of the collision between different obstacle categories are defined in advance, the obstacle categories identified by the obstacle identification model and the consequences of the collision between the obstacle categories are used to assign corresponding collision weights to the obstacles in the current scene; the spatiotemporal collision probability, the risk cost and the collision weight are weighted and averaged to obtain a comprehensive risk value, which is input into the RRT* algorithm to generate an optimal obstacle avoidance path that minimizes the comprehensive risk.

Images