Flying car multi-sensor fusion path planning method, system and equipment and storage medium

Abstract

The invention discloses an aerocar multi-sensor fusion path planning method, system and device and a storage medium, and the method comprises the steps: S1, obtaining the surrounding environment information of an aerocar, constructing a three-dimensional simulation space, and building the starting point coordinates and target point coordinates of the aerocar; s2, improving the ant colony algorithm, including setting a multi-factor heuristic function, an adaptive volatilization coefficient, a dynamic weight coefficient and a multi-factor pheromone updating mechanism; s3, setting a safe distance function of an artificial potential field method and a track evaluation function of a dynamic window method; and S4, the adjusted ant colony algorithm performs global path planning based on the surrounding environment information and the starting point coordinates and the target point coordinates of the hovercar, a local path is evaluated and judged through a safety distance function and a trajectory evaluation function, and an optimal path of the hovercar is obtained. The efficiency and accuracy of path planning are remarkably improved, and the safety and stability of the flight process are ensured.

Description

Technical Field

[0001] The present invention belongs to the field of flying cars and relates to a flying car multi-sensor fusion path planning method, system, equipment and storage medium. Background Art

[0002] Path planning technology is a key guidance technology for planning the aerial flight path of a flying car from any take-off point to any landing point in the city. Unlike traditional ground transportation, the path planning of urban air traffic takes place in a three-dimensional space, and the complex three-dimensional environment and channel structure must be considered. This three-dimensional path planning involves selecting appropriate air routes, considering the restrictions on channel change nodes and the direction of route travel. The complexity of urban air traffic path planning lies in the fact that it needs to be carried out in a three-dimensional space, and it is difficult to plan the best path in real time and efficiently. At the same time, take-off and landing control technology is also an important challenge to the safety of flying cars, which requires effective take-off and priority to ensure landing safety. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a flying car multi-sensor fusion path planning method, system, device and storage medium, which significantly improves the efficiency and accuracy of path planning and ensures the safety and stability of the flight process.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions: A flying car multi-sensor fusion path planning method includes the following processes: S1, obtain the surrounding environment information of the flying car, construct a three-dimensional simulation space, and establish the starting point coordinates and target point coordinates of the flying car; S2, improved ant colony algorithm, including setting multi-factor heuristic function, adaptive volatility coefficient, dynamic weight coefficient and multi-factor pheromone update mechanism; S3, setting the safety distance function of the artificial potential field method and the trajectory evaluation function of the dynamic window method; S4, the adjusted ant colony algorithm performs global path planning based on the surrounding environment information and the starting point coordinates and target point coordinates of the flying car, where the local path is evaluated and judged by the safety distance function and the trajectory evaluation function to obtain the optimal path of the flying car.

[0005] Preferably, the specific process of S1 is: obtaining the three-dimensional point cloud of the lidar and the visual information of the camera; processing and fusing them in combination with MATLAB to create a three-dimensional space; when the flying car reaches a certain flight altitude, determining the grid map size of the flight stage, as well as the starting point coordinates and target point coordinates of the flying car.

[0006] Preferably, in S2, in the ant colony algorithm, a modified distance function from the next node to the end point and a turning point smoothing factor are added to the heuristic function, and a multi-factor heuristic function is set; an adaptive volatility coefficient that changes with the number of iterations is set; a pheromone weight coefficient and a heuristic function weight coefficient are set in the dynamic weight coefficient; and a multi-factor pheromone update mechanism is set.

[0007] Preferably, in S3, in the safety distance function, the position of the flying car is set as the origin, and the obstacle radius R, the safety distance L from the obstacle, the obstacle range radius M and the vertical distance S between the obstacle and the line connecting the flying car to the target point are obtained; when R<S<L, the flying car is affected by the repulsive force of the obstacle, and when L<S<M, the flying car is not affected by the repulsive force of the obstacle.

[0008] Preferably, in S3, in the trajectory evaluation function, an initial average speed is set, and the trajectory evaluation function is calculated, each generated predicted trajectory is quantitatively evaluated by the trajectory evaluation function to determine the best trajectory, and the trajectory evaluation function used by the dynamic window method is updated as the evaluation criterion for the next path planning.

[0009] Preferably, the specific process of S4 is as follows: S41, setting the number of ants, and placing the ants at the starting point, moving to the next node by selecting probability, and planning the subsequent paths in the same way, such as using the selection probability to make the ants move from the current node to the next node at time, and when the ants complete the path, updating the pheromone of each path passed until reaching the target point; S42, by judging the positional relationship between the current node and the interval node and the obstacle, infer whether the adjacent node can be discarded. If the positional relationship is horizontal, vertical, or the distance between the line and the obstacle is less than the range of the obstacle, the adjacent node is discarded. If the node cannot be discarded, the next node is selected as the current key node, the current key node is set as the local target point, and the key node is recorded and continuously updated iteratively; S43, generating multiple candidate trajectories near the local target point; each trajectory can be generated by a vehicle motion model, and the optimal speed control is selected as output through the safety distance of the artificial potential field method and the trajectory evaluation function of the dynamic window method; S44, using sensors to monitor the surrounding environment in real time, updating environmental information, checking whether the current local target point is blocked by a new obstacle, if the local target point is unreachable, selecting the node closest to the current position of the vehicle, and setting the closest node as the new starting point, and returning to S41 to re-plan the global path; S45, detecting whether the current position of the flying car has reached the local target point. If the distance between the current position of the flying car and the local target point is less than the set distance, it is considered to have reached the local target point. At this time, the next node is selected as the new local target point, and S43 is re-executed; if the local target point has not been reached, return to S42 to continue iteration; S46, detect whether the vehicle has reached the global target point. If the distance between the vehicle and the global target point is less than the set distance, it is considered to have reached the global target point; if it has not reached the global target point, return to S45 until it reaches the global target point.

[0010] Preferably, after obtaining the optimal path of the flying car, a B-spline curve is used to smooth the planned path and eliminate the relatively tortuous paths at the nodes.

[0011] A flying car multi-sensor fusion path planning method, comprising: A coordinate establishment module is used to obtain the surrounding environment information of the flying car, construct a three-dimensional simulation space, and establish the starting point coordinates and target point coordinates of the flying car; Ant colony algorithm improvement module, used to improve the ant colony algorithm, including setting multi-factor heuristic function, adaptive volatility coefficient, dynamic weight coefficient and multi-factor pheromone update mechanism; Function building module, used to set the safety distance function of the artificial potential field method and the trajectory evaluation function of the dynamic window method; The path planning module is used to adjust the ant colony algorithm to perform global path planning based on the surrounding environment information and the starting point coordinates and target point coordinates of the flying car. The local path is evaluated and judged by the safety distance function and the trajectory evaluation function to obtain the optimal path of the flying car.

[0012] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the flying car multi-sensor fusion path planning method when executing the computer program.

[0013] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the flying car multi-sensor fusion path planning method are implemented.

[0014] Compared with the prior art, the present invention has the following beneficial effects: The present invention effectively overcomes the limitations of a single algorithm by integrating the global search capability of the ant colony algorithm, the local obstacle avoidance advantage of the artificial potential field method, and the real-time trajectory optimization characteristics of the dynamic window method. The collaborative working mode of multiple algorithms significantly improves the efficiency and accuracy of path planning, ensuring that the flying car can complete the path planning task safely, quickly and efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is the overall framework diagram of the flying car of the present invention; Figure 2 The three views of the flying car structure of the present invention; Figure 3 This is a flow chart of the path planning system of the present invention; Figure 4 A schematic diagram of a safety distance model of the artificial potential field method of the present invention; Figure 5 It is a schematic diagram of the multi-fusion algorithm flow of the present invention; Figure 6 It is a schematic diagram of the integration of the multi-fusion algorithm and the sensor of the present invention; Figure 7 It is the global path planning diagram of the multi-fusion algorithm of the present invention; Figure 8 This is the dynamic obstacle path planning diagram of the multi-fusion algorithm of the present invention.

[0016] Among them: 1. Fixing device; 2. LiDAR; 3. Camera; 4. GPS; 5. Temperature and humidity sensor; 6. Data processor; 7. Thrust device; 8. Altitude control mechanism. DETAILED DESCRIPTION

[0017] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0018] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which the present invention belongs. The terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or can communicate with each other; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" used in this article includes any and all combinations of one or more related listed items. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0020] The disclosure below provides many different embodiments or examples to realize different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplicity and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides various specific examples of processes and materials, but those of ordinary skill in the art can be aware of the application of other processes and / or the use of other materials.

[0021] like Figure 1 As shown in FIG. 1 , the overall framework of the flying car of the present invention is provided, and the flying car is equipped with a fixing device 1, a laser radar 2, a camera 3, a GPS 4, a temperature and humidity sensor 5, a data processor 6, a thrust device 7, and a height control mechanism 8. These sensors are used to obtain spatial information of the flying environment and the distribution of obstacles and transmit data in a timely manner. The detailed layout is shown in FIG. Figure 2 shown.

[0022] The laser radar 2 is installed at the bottom of the flying car to identify obstacles, buildings, etc. and provide high-precision map data. Cameras 3 are installed around the flying car to obtain information about the surrounding environment. GPS 4 is installed in the flying car fixture 1 to provide real-time location information and determine the current location of the flying car or dynamic obstacles.

[0023] The temperature and humidity sensor 5 is installed at the rear of the flying car to measure the temperature and humidity data in the environment and predict weather changes. The data processor 6 is installed at the center of the flying car to process and manage the data collected by the sensor, calculate, store and analyze the data for subsequent application and decision-making.

[0024] The thrust device 7 is installed at the tail of the flying car to provide the thrust required by the flying car and control the speed and direction of the flying car. The height control mechanism 8 is installed at the center of the flying car wing to control the flying height of the flying car to ensure its stable flight in the air and avoid obstacles.

[0025] like Figure 3 and Figure 6 As shown in FIG. 1 , the flying car multi-sensor fusion path planning method of the present invention includes the following processes: Step 1: Initial system parameter setting and initial configuration: The surrounding environment information of the flying car is obtained by sensors, combined with MATLAB to build a three-dimensional simulation space, and establish the starting point coordinates and target point coordinates of the flying car.

[0026] Step 1: Generate high-precision 3D point clouds through LiDAR and visual information provided by the camera, and combine MATLAB to process and fuse sensor data to create a detailed 3D space with a size of .

[0027] Step 2: When the flying car reaches a certain altitude, the GPS coordinate system is used to determine the grid map size during the flight phase. , and the starting coordinates of the flying car and the target point coordinates .

[0028] Step 2: Adjust the basic parameter settings of the ant colony algorithm: through the detected environmental information, set parameters such as multi-factor heuristic function, adaptive volatility coefficient, dynamic weight coefficient and multi-factor pheromone update mechanism, update the path in real time to ensure flight efficiency.

[0029] Step 1, multi-factor heuristic function: In the early stage of iteration, the larger values ​​of the distance correction function and the turning point smoothing factor function reduce the number of required iterations and the number of path turns. In the later stage, the values ​​of these two functions are appropriately reduced to avoid the algorithm falling into the local optimal solution. Therefore, the corrected distance function from the next node to the end point is added to the heuristic function. and turning point smoothing factor , set the multi-factor heuristic function .

[0030] Step 2, adaptive volatility coefficient: In the early stage of iteration, the adaptive volatility coefficient value is relatively large, which can improve the path search ability of ants and increase the probability of finding the optimal path. In the later stage of iteration, the adaptive volatility coefficient value is reduced, and the pheromone volatility is reduced, which can improve the convergence speed of the optimal path. Therefore, the adaptive volatility coefficient is set to change with the number of iterations. .

[0031] Step 3, dynamic weight coefficient: the pheromone weight coefficient and heuristic function weight coefficient change with the number of iterations. In the early stage of algorithm iteration, the pheromone weight coefficient and heuristic function weight coefficient are assigned the minimum value to improve the global search ability and the probability of finding the global optimal solution. When the set iteration limit is reached, the pheromone weight coefficient and heuristic function weight coefficient gradually increase to improve the convergence ability of the algorithm. Therefore, the pheromone weight coefficient is set and the heuristic function weight coefficient .

[0032] Step 4: Multi-factor pheromone update mechanism: In the process of pheromone update, the path length and number of turns are considered to improve the quality of the planned path. Therefore, a multi-factor pheromone update mechanism is set up. .

[0033] Step 3: Design evaluation function: Design the safety distance of the artificial potential field method and the trajectory evaluation function of the dynamic window method, and make a weighted combination of these factors to determine the score of each path in order to select the best and safest path.

[0034] Step 1: Establish the safety distance function of the artificial potential field method: Figure 4 As shown in the figure, O is the location of the flying car, which is set as the origin; R is the radius of the circular obstacle; L is the safe distance from the obstacle; M is the radius of the obstacle's range; S is the vertical distance between the obstacle and the line connecting the flying car to the target point (black five-pointed star). When R<S<L, the flying car is affected by the repulsive force of the obstacle, and when L<S<M, the flying car is not affected by the repulsive force of the obstacle. Therefore, the flying car can avoid excessive obstacle avoidance, which leads to a too long planned path.

[0035] Step 2: Trajectory evaluation function: setting the initial average speed , and calculate the trajectory evaluation function Each generated predicted trajectory needs to be quantitatively evaluated by the trajectory evaluation function to determine the best trajectory. The flying car will It is used as a speed control instruction to drive and update the trajectory evaluation function used by the dynamic window method. , as the evaluation criterion for the next path planning.

[0036] Step 4: Figure 5 As shown in the figure, multiple algorithms are integrated to plan the path of the flying car: the global path planning of the improved ant colony algorithm is combined with the local obstacle avoidance of the dynamic window method to plan the optimal path of the flying car and perform smoothness processing.

[0037] Step 1: Set the number of ants m and place the ants at the starting point , by choosing the probability Move to the next node , subsequent paths are planned in the same way, such as using the selection probability Make the ant leave the current node at time t Move to the next node When the ant completes the path, it updates the pheromone of each path it passes through. Until you reach the destination .

[0038] Step 2: By judging the current node With interval nodes and the position relationship of obstacles to infer whether the adjacent nodes can be discarded If the position relationship is horizontal, vertical, or the distance between the line and the obstacle is less than the range of the obstacle, the adjacent node is discarded. If the node cannot be discarded, select the next node as the current key node , the current key node Set as local target point , and record key nodes, and continuously iterate and update.

[0039] Step 3: At the local target point Generate M candidate trajectories nearby. Each trajectory can be generated by the vehicle's motion model. The optimal speed is selected through the safety distance of the artificial potential field method and the trajectory evaluation function of the dynamic window method. Control as output.

[0040] Step 4: Use sensors to monitor the surrounding environment in real time, update environmental information, and check the current local target point Is it blocked by a new obstacle? , it means it is blocked by obstacles. If the local target point is unreachable, select the node closest to the vehicle's current position , and Set it as a new starting point and return to step 1 to replan the global path.

[0041] Step 5: Check whether the current position of the flying car has reached the local target point. If the distance between the current position of the flying car and the local target point is less than 5m, it is considered to have reached the local target point. At this time, the next node is selected as the new local target point and step 3 is re-executed; if the local target point has not been reached, return to step 2 to continue iteration.

[0042] Step 6: Check whether the vehicle has reached the global target point. If the distance between the vehicle and the global target point is less than 2m, it is considered to have reached the global target point. If it has not reached the global target point, return to step 5 until it reaches the global target point.

[0043] Step 7: Use B-spline curves to smooth the planned path and eliminate the more tortuous paths at the nodes to make the path more continuous and smooth.

[0044] Step 5. Execute the planned path of the flying car: Generate flight control instructions based on the planned path, and guide the operation of the flying car through real-time feedback from sensors, including take-off execution, flight process, and landing execution, to ensure that the flying car can take off, fly, and land safely according to the planned path.

[0045] Step 1: Takeoff execution phase The system performs self-checks and environmental monitoring to ensure that all systems are normal and there are no obstacles in the takeoff area. After confirming safety, the PID controller generates detailed takeoff instructions according to the planned path, gradually increases the altitude, and maintains a stable flight state by controlling thrust and attitude to ensure smooth takeoff and enter cruising altitude.

[0046] Step 2: The system uses the PID controller to generate detailed flight control instructions based on the optimal path, covering the position, speed and other parameters of each node to ensure that the flying car performs smoothly throughout the flight. The data processor is responsible for receiving and analyzing the control instruction parameters, converting them into recognizable signals, and accurately conveying them to the flying car control system. Finally, the instructions processed by the data processor are passed to the control system to guide the flying car to adjust its attitude and speed, so as to accurately perform flight operations and safely reach the destination along the planned path.

[0047] Step 3: During the landing phase, the system performs self-tests and environmental tests, generates detailed landing instructions using the PID controller, gradually lowers the altitude and controls the speed to approach the ground smoothly. When approaching the ground, the final speed adjustment is made to ensure a safe landing and a stable stop.

[0048] Step 6: Monitor the path in real time and make adjustments: Use sensor data to monitor the surrounding environment in real time, identify and predict the location and movement of obstacles, and automatically adjust the flight trajectory.

[0049] Step 1: When a dynamic obstacle is detected, the system combines the data from the lidar and camera to identify the obstacle and predict the movement trajectory. If the obstacle is closer than the set safety distance from the flying car, the system will issue an alarm in time and automatically adjust the flight trajectory to ensure flight safety.

[0050] Step 2: When encountering special weather conditions, the temperature and humidity sensor will monitor the temperature and humidity of the environment in real time, and the aerodynamic sensor will measure the wind speed, air pressure and air density of the environment in real time. If high humidity is detected, the system will automatically adjust the flight trajectory to cope with the changing weather conditions.

[0051] The following is a detailed description of the above method with a specific example: Design and build the flying car's structural frame: Figure 1 As shown in FIG. 1 , the flying car is equipped with a fixing device 1, a laser radar 2, a camera 3, a GPS 4, a temperature and humidity sensor 5, a data processor 6, a thrust device 7, and a height control mechanism 8. To obtain the spatial information of the flight environment and the distribution of obstacles and transmit the data in time, the layout is as follows: Figure 2 shown.

[0052] The laser radar 2 is installed at the bottom of the flying car to identify obstacles, buildings, etc. and provide high-precision map data. Cameras 3 are installed around the flying car to obtain information about the surrounding environment. GPS 4 is installed in the flying car fixture 1 to provide real-time location information and determine the current location of the flying car or dynamic obstacles.

[0053] The temperature and humidity sensor 5 is installed at the rear of the flying car to measure the temperature and humidity data in the environment and predict weather changes. The data processor 6 is installed at the center of the flying car to process and manage the data collected by the sensor, calculate, store and analyze the data for subsequent application and decision-making.

[0054] The thrust device 7 is installed at the tail of the flying car to provide the thrust required by the flying car and control the speed and direction of the flying car. The height control mechanism 8 is installed at the center of the flying car wing to control the flying height of the flying car to ensure its stable flight in the air and avoid obstacles.

[0055] Step 1: Initial system parameter setting and initial configuration: Use the environmental information obtained by the sensor and MATLAB to build a three-dimensional simulation space to establish the starting position and target position of the flying car.

[0056] Step 1: Generate high-precision 3D point clouds through LiDAR and visual information provided by the camera, and combine MATLAB to process and fuse sensor data to create a detailed 3D space with a size of , the unit is km.

[0057] Step 2: The flying car reaches a certain altitude, such as ; Use GPS coordinate system to determine the grid map size during the flight phase , and the starting coordinates of the flying car and the target point coordinates , the unit is km.

[0058] Step 2: Adjust the basic parameter settings of the ant colony algorithm: through the detected environmental information, set parameters such as multi-factor heuristic function, adaptive volatility coefficient, dynamic weight coefficient and multi-factor pheromone update mechanism, update the path in real time to ensure flight efficiency.

[0059] Step 1, multi-factor heuristic function: In the early stage of iteration, the larger values ​​of the distance correction function and the turning point smoothing factor function reduce the number of required iterations and the number of path turns. In the later stage, the values ​​of these two functions are appropriately reduced to avoid the algorithm falling into the local optimal solution. Therefore, the corrected distance function from the next node to the end point is added to the heuristic function. and turning point smoothing factor , set the multi-factor heuristic function , the calculation formula is as follows: (1) (2) (3) (4) In the formula, The maximum distance from the next node adjacent to the current node to the end point; The minimum distance from the next node adjacent to the current node to the end point; is the distance from the next node to the end point; is the current iteration number; is a positive number, set to 0.01; It is the function expression adjustment coefficient, usually set to a positive number, set to 10; is the correction factor, set to 2; is the set length of the next selectable node; The previous node of the current node With the current node The angle between the transfer direction and the x-axis; Current Node With the next node The angle between the transfer direction and the x-axis; is the path heuristic constant, set to 10; The importance coefficient of ants moving straight is usually set to a positive constant less than 1, which is set to 0.5; For the current node With the next node distance; For the current node The coordinates of For the next node The coordinates of .

[0060] Step 2, adaptive volatility coefficient: In the early stage of iteration, the adaptive volatility coefficient value is relatively large, which can improve the path search ability of ants and increase the probability of finding the optimal path. In the later stage of iteration, the adaptive volatility coefficient value is reduced, and the pheromone volatility is reduced, which can improve the convergence speed of the optimal path. Therefore, the adaptive volatility coefficient is set to change with the number of iterations. , the calculation formula is as follows: (5) In the formula, the adaptive volatility coefficient .

[0061] Step 3, dynamic weight coefficient: the pheromone weight coefficient and heuristic function weight coefficient change with the number of iterations. In the early stage of algorithm iteration, the pheromone weight coefficient and heuristic function weight coefficient are assigned the minimum value to improve the global search ability and the probability of finding the global optimal solution. When the set iteration limit is reached, the pheromone weight coefficient and heuristic function weight coefficient gradually increase to improve the convergence ability of the algorithm. Therefore, the pheromone weight coefficient is set and the heuristic function weight coefficient , the calculation formula is as follows: (6) (7) In the formula, is the pheromone weight coefficient The maximum value is set to 4; is the pheromone weight coefficient Minimum value, set to 1; is the weight coefficient of the heuristic function The maximum value is set to 8; is the weight coefficient of the heuristic function Minimum value, set to 3; is the limit value of the number of iterations, usually a constant, set to 30; is the total number of iterations, set to 100.

[0062] Step 4: Multi-factor pheromone update mechanism: In the process of pheromone update, the path length and number of turns are considered to improve the quality of the planned path. Therefore, a multi-factor pheromone update mechanism is set up. , the calculation formula is as follows: (8) (9) In the formula, is the ant path distance; is the number of turns along the ant path; It is the comprehensive index of ant path.

[0063] Step 3: Design evaluation function: Design the safety distance of the artificial potential field method and the trajectory evaluation function of the dynamic window method, and make a weighted combination of these factors to determine the score of each path in order to select the best and safest path.

[0064] Step 1: Design the safety distance function of the artificial potential field method: Figure 3 As shown in the figure, O is the location of the flying car, which is set as the origin; R is the radius of the circular obstacle; L is the safety distance, which is set to 5m; M is the radius of the obstacle range; S is the vertical distance between the obstacle and the line connecting the flying car to the target point (black five-pointed star). When R<S<L, the flying car is affected by the repulsive force of the obstacle, and when L<S<M, the flying car is not affected by the repulsive force of the obstacle. Therefore, the flying car can avoid excessive obstacle avoidance and cause the planned path to be too long.

[0065] Step 2: Trajectory evaluation function: setting the initial average speed , and calculate the trajectory evaluation function Each generated predicted trajectory needs to be quantitatively evaluated by the trajectory evaluation function to determine the best trajectory. The flying car will It is used as a speed control instruction to drive and update the trajectory evaluation function used by the dynamic window method. , as the evaluation criterion for the next path planning, the calculation formula is as follows: (10) (11) In the formula, is the distance between the vehicle and the target point at the end of the trajectory; is the coordinate of the target point; is the coordinate of the end point of the vehicle trajectory; It is the vehicle target distance evaluation sub-function.

[0066] Step 4: Fusion of multiple algorithms for flying car path planning: Combine the global path planning of the improved ant colony algorithm with the local obstacle avoidance of the dynamic window method to plan the optimal path of the flying car and perform smoothing processing.

[0067] Step 1: Set the number of ants m=50 and place the ants at the starting point , by choosing the probability Move to the next node , subsequent paths are planned in the same way, such as using the selection probability Make the ant leave the current node at time t Move to the next node When the ant completes the path, it updates the pheromone of each path it passes through. Until you reach the destination ,like Figure 7 As shown, the probability of selection The calculation formula is as follows: (12) (13) in, is the set of next feasible nodes; is the distance heuristic function.

[0068] Step 2: By judging the current node With interval nodes and the position relationship of obstacles to infer whether the adjacent nodes can be discarded If the position relationship is horizontal, vertical, or the distance between the line and the obstacle is less than the range of the obstacle, the adjacent node is discarded. If the node cannot be discarded, select the next node as the current key node , the current key node Set as local target point , and record key nodes, and continuously iterate and update.

[0069] Step 3: At the local target point Generate M candidate trajectories nearby. Each trajectory can be generated by the vehicle's motion model. The optimal speed is selected through the safety distance of the artificial potential field method and the trajectory evaluation function of the dynamic window method. Control as output.

[0070] Step 4: Use sensors to monitor the surrounding environment in real time, update environmental information, and check the current local target point Is it blocked by a new obstacle? , it means it is blocked by obstacles. If the local target point is unreachable, select the node closest to the vehicle's current position , and Set it as a new starting point and return to step 1 to replan the global path.

[0071] Step 5: Check whether the current position of the flying car has reached the local target point. If the distance between the current position of the flying car and the local target point is less than 5m, it is considered to have reached the local target point. At this time, the next node is selected as the new local target point and step 3 is re-executed; if the local target point has not been reached, return to step 2 to continue iteration.

[0072] Step 6: Check whether the vehicle has reached the global target point. If the distance between the vehicle and the global target point is less than 2m, it is considered to have reached the global target point. If it has not reached the global target point, return to step 5 until it reaches the global target point.

[0073] Step 7: Use B-spline curve to smooth the planned path and eliminate the more tortuous path at the node to make the path more continuous and smooth. Figure 8 shown.

[0074] Step 5. Execute the planned path of the flying car: Generate flight control instructions based on the planned path, and guide the operation of the flying car through real-time feedback from sensors, including take-off execution, flight process, and landing execution, to ensure that the flying car can take off, fly, and land safely according to the planned path.

[0075] Step 1: Takeoff execution phase The system performs self-checks and environmental monitoring to ensure that all systems are normal and there are no obstacles in the takeoff area. After confirming safety, the PID controller generates detailed takeoff instructions according to the planned path, gradually increases the altitude, and maintains a stable flight state by controlling thrust and attitude to ensure smooth takeoff and enter cruising altitude.

[0076] Step 2: The system uses the PID controller to generate detailed flight control instructions based on the optimal path, covering the position, speed and other parameters of each node to ensure that the flying car performs smoothly throughout the flight. The data processor is responsible for receiving and analyzing the control instruction parameters, converting them into recognizable signals, and accurately conveying them to the flying car control system. Finally, the instructions processed by the data processor are passed to the control system to guide the flying car to adjust its attitude and speed, so as to accurately perform flight operations and safely reach the destination along the planned path.

[0077] Step 3: During the landing phase, the system performs self-tests and environmental tests, generates detailed landing instructions using the PID controller, gradually lowers the altitude and controls the speed to approach the ground smoothly. When approaching the ground, the final speed adjustment is made to ensure a safe landing and a stable stop.

[0078] Step 6: Monitor the path in real time and make adjustments: Use sensor data to monitor the surrounding environment in real time, identify and predict the location and movement of obstacles, and automatically adjust the flight trajectory.

[0079] Step 1: When a dynamic obstacle is detected, the system combines the data from the lidar and camera to identify the obstacle and predict the movement trajectory. If the obstacle is closer than the set safety distance from the flying car, the system will issue an alarm in time and automatically adjust the flight trajectory to ensure flight safety.

[0080] Step 2: When encountering special weather conditions, the temperature and humidity sensor will monitor the temperature and humidity of the environment in real time, and the aerodynamic sensor will measure the wind speed, air pressure and air density of the environment in real time. If high humidity is detected, the system will automatically adjust the flight trajectory to cope with the changing weather conditions.

[0081] The following are device embodiments of the present invention, which can be used to perform method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0082] In another embodiment of the present invention, a flying car multi-sensor fusion path planning system is provided. The flying car multi-sensor fusion path planning system can be used to implement the above-mentioned flying car multi-sensor fusion path planning method. Specifically, the flying car multi-sensor fusion path planning system includes a coordinate establishment module, an ant colony algorithm improvement module, a function establishment module and a path planning module.

[0083] Among them, the coordinate establishment module is used to obtain the surrounding environment information of the flying car, construct a three-dimensional simulation space, and establish the starting point coordinates and target point coordinates of the flying car.

[0084] The ant colony algorithm improvement module is used to improve the ant colony algorithm, including setting multi-factor heuristic functions, adaptive volatility coefficients, dynamic weight coefficients and multi-factor pheromone update mechanisms.

[0085] The function building module is used to set the safety distance function of the artificial potential field method and the trajectory evaluation function of the dynamic window method.

[0086] The path planning module is used to adjust the ant colony algorithm to perform global path planning based on the surrounding environment information and the starting point coordinates and target point coordinates of the flying car. The local path is evaluated and judged by the safety distance function and the trajectory evaluation function to obtain the optimal path for the flying car.

[0087] In another embodiment of the present invention, a terminal device is provided, the terminal device includes a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGAs), or other processors. GateArray, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding functions; the processor described in the embodiment of the present invention can be used for the operation of the multi-sensor fusion path planning method for a flying car, including: S1, obtaining the surrounding environment information of the flying car, constructing a three-dimensional simulation space, and establishing the starting point coordinates and target point coordinates of the flying car; S2, improving the ant colony algorithm, including setting a multi-factor heuristic function, an adaptive volatility coefficient, a dynamic weight coefficient and a multi-factor pheromone update mechanism; S3, setting a safe distance function of the artificial potential field method and a trajectory evaluation function of the dynamic window method; S4, the adjusted ant colony algorithm performs global path planning based on the surrounding environment information and the starting point coordinates and target point coordinates of the flying car, wherein the local path is evaluated and judged by the safe distance function and the trajectory evaluation function to obtain the optimal path of the flying car.

[0088] In another embodiment, the present invention further provides a computer-readable storage medium (Memory), which is a memory device in a terminal device for storing programs and data. It is understandable that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory.

[0089] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the multi-sensor fusion path planning method for a flying car in the above-mentioned embodiment; one or more instructions in the computer-readable storage medium are loaded by the processor and the following steps are executed: S1, obtaining the surrounding environment information of the flying car, constructing a three-dimensional simulation space, and establishing the starting point coordinates and target point coordinates of the flying car; S2, improving the ant colony algorithm, including setting a multi-factor heuristic function, an adaptive volatility coefficient, a dynamic weight coefficient and a multi-factor pheromone update mechanism; S3, setting a safe distance function of the artificial potential field method and a trajectory evaluation function of the dynamic window method; S4, the adjusted ant colony algorithm performs global path planning based on the surrounding environment information and the starting point coordinates and target point coordinates of the flying car, wherein the local path is evaluated and judged by the safe distance function and the trajectory evaluation function to obtain the optimal path of the flying car.

[0090] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0091] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0092] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0093] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0094] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0095] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0097] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0098] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

[0099] It should be understood that the above description is for illustration and not for limitation. Many embodiments and many applications beyond the examples provided will be apparent to those skilled in the art upon reading the above description. Therefore, the scope of this patent should not be determined with reference to the above description, but rather with reference to the preceding claims and the full scope of equivalents to which such claims are entitled. For the purpose of comprehensiveness, all articles and references, including disclosures of patent applications and publications, are incorporated herein by reference. The omission of any aspect of the subject matter disclosed herein in the preceding claims is not intended to be a waiver of such subject matter, nor should it be considered that the applicant has not considered such subject matter to be part of the disclosed inventive subject matter.

Claims

1. A flying car multi-sensor fusion path planning method, characterized in that: The process includes: S1, obtain the surrounding environment information of the flying car, construct a three-dimensional simulation space, and establish the starting point coordinates and target point coordinates of the flying car; S2, improved ant colony algorithm, including setting multi-factor heuristic function, adaptive volatility coefficient, dynamic weight coefficient and multi-factor pheromone update mechanism; S3, setting the safety distance function of the artificial potential field method and the trajectory evaluation function of the dynamic window method; S4, the adjusted ant colony algorithm performs global path planning based on the surrounding environment information and the starting point coordinates and target point coordinates of the flying car, where the local path is evaluated and judged by the safety distance function and the trajectory evaluation function to obtain the optimal path of the flying car.

2. The flying car multi-sensor fusion path planning method according to claim 1, characterized in that: The specific process of S1 is: obtain the three-dimensional point cloud of the lidar and the visual information of the camera; process and fuse them with MATLAB to create a three-dimensional space; when the flying car reaches a certain flight altitude, determine the grid map size of the flight stage, as well as the starting point coordinates and target point coordinates of the flying car.

3. The flying car multi-sensor fusion path planning method according to claim 1, characterized in that: In S2, in the ant colony algorithm, the modified distance function from the next node to the end point and the turning point smoothing factor are added to the heuristic function, and a multi-factor heuristic function is set; an adaptive volatility coefficient that changes with the number of iterations is set; the pheromone weight coefficient and the heuristic function weight coefficient are set in the dynamic weight coefficient; and a multi-factor pheromone update mechanism is set.

4. The flying car multi-sensor fusion path planning method according to claim 1, characterized in that: In S3, in the safety distance function, the position of the flying car is set as the origin, and the obstacle radius R, the safety distance L from the obstacle, the obstacle range radius M and the vertical distance S between the obstacle and the line connecting the flying car to the target point are obtained; when R<S<L, the flying car is affected by the repulsive force of the obstacle, and when L<S<M, the flying car is not affected by the repulsive force of the obstacle.

5. The flying car multi-sensor fusion path planning method according to claim 1, characterized in that: In S3, in the trajectory evaluation function, the initial average speed is set, and the trajectory evaluation function is calculated. Each generated predicted trajectory is quantitatively evaluated by the trajectory evaluation function to determine the best trajectory, and the trajectory evaluation function used by the dynamic window method is updated as the evaluation criterion for the next path planning.

6. The flying car multi-sensor fusion path planning method according to claim 1, characterized in that: The specific process of S4 is as follows: S41, set the number of ants, and place the ants at the starting point, and move to the next node by selecting the probability. The subsequent paths are planned in the same way, such as using the selection probability to make the ants move from the current node to the next node at time. When the ants complete the path, the pheromone of each path they pass through is updated until they reach the target point; S42, by judging the positional relationship between the current node and the interval node and the obstacle, infer whether the adjacent node can be discarded. If the positional relationship is horizontal, vertical, or the distance between the line and the obstacle is less than the range of the obstacle, the adjacent node is discarded. If the node cannot be discarded, the next node is selected as the current key node, the current key node is set as the local target point, and the key node is recorded and continuously updated iteratively; S43, generating multiple candidate trajectories near the local target point; each trajectory can be generated by a vehicle motion model, and the optimal speed control is selected as output through the safety distance of the artificial potential field method and the trajectory evaluation function of the dynamic window method; S44, using sensors to monitor the surrounding environment in real time, updating environmental information, checking whether the current local target point is blocked by a new obstacle, if the local target point is unreachable, selecting the node closest to the current position of the vehicle, and setting the closest node as the new starting point, and returning to S41 to re-plan the global path; S45, detecting whether the current position of the flying car has reached the local target point. If the distance between the current position of the flying car and the local target point is less than the set distance, it is considered to have reached the local target point. At this time, the next node is selected as the new local target point, and S43 is re-executed; if the local target point has not been reached, return to S42 to continue iteration; S46, detecting whether the vehicle has reached the global target point. If the distance between the vehicle and the global target point is less than a set distance, it is considered that the vehicle has reached the global target point. If the global target point has not been reached, return to S45 until the global target point is reached.

7. The flying car multi-sensor fusion path planning method according to claim 1, characterized in that: After obtaining the optimal path of the flying car, the B-spline curve is used to smooth the planned path and eliminate the more tortuous paths at the nodes.

8. A flying car multi-sensor fusion path planning method, characterized in that: include: A coordinate establishment module is used to obtain the surrounding environment information of the flying car, construct a three-dimensional simulation space, and establish the starting point coordinates and target point coordinates of the flying car; Ant colony algorithm improvement module, used to improve the ant colony algorithm, including setting multi-factor heuristic function, adaptive volatility coefficient, dynamic weight coefficient and multi-factor pheromone update mechanism; Function building module, used to set the safety distance function of the artificial potential field method and the trajectory evaluation function of the dynamic window method; The path planning module is used to adjust the ant colony algorithm to perform global path planning based on the surrounding environment information and the starting point coordinates and target point coordinates of the flying car. The local path is evaluated and judged by the safety distance function and the trajectory evaluation function to obtain the optimal path of the flying car.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the flying car multi-sensor fusion path planning method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the flying car multi-sensor fusion path planning method as described in any one of claims 1 to 7 are implemented.

Images