MULTIMODAL EDGE AI ORCHESTRATION SYSTEM AND METHOD FOR AIR-LAND INTEGRATED HYBRID TRAFFIC MANAGEMENT

Abstract

The invention specifically relates to managing movement conflicts, traffic congestion, route intersections, and safety risks that arise when unmanned aerial vehicles, vertical take-off and landing aircraft, autonomous ground vehicles, driverless logistics vehicles, and connected vehicles operate simultaneously in the same geographical area.

Description

1 TARIFF MULTI-MODAL EDGE AI FOR AIR-GROUND INTEGRATED HYBRID TRAFFIC MANAGEMENT. ORCHESTRATION SYSTEM AND METHOD Technical Area 5 The invention enables autonomous movement in air and land environments within smart city infrastructures. multimodal systems that ensure the safe, coordinated and uninterrupted operation of vehicles. It is related to autonomous mobility systems. The invention is particularly relevant to unmanned aerial vehicles, vertical take-off and landing aircraft, autonomous land vehicles, driverless logistics vehicles and connected vehicles in the same geographic 10 movement overlaps that occur when operating simultaneously in the same area, It involves managing traffic congestion, route intersections, and safety risks. State of the Art Today, the management of autonomous ground vehicles and unmanned aerial vehicles differs significantly. It is carried out within two completely independent technological ecosystems. Autonomous 15 Land vehicles, V2X-based communication protocols, intelligent intersection management systems, sensors fusion mechanisms and rule-based or Model Predictive Control (MPC) While drones are guided by algorithms, unmanned aerial vehicles are guided by UTM (Unmanned Traffic) Management) infrastructures, altitude layering methods, flight authorization systems and It is controlled by sense-and-avoid algorithms. Both fields have 20 within them. Although advanced, current methods focus only on the relevant vehicle type, air complex urban environments where vehicles and land vehicles interact simultaneously It does not offer any integrated solution for this. Autonomous land vehicles use V2V, V2I, and V2N communications to approach intersections. Algorithms that evaluate speeds, lane change decisions, and collision risk 25 These systems operate taking into account other vehicles on the land surface. They are designed and their infrastructure includes decision-making mechanisms solely for road traffic. This allows for the estimation of the location, altitude, or route of drones or other aerial vehicles. It is not processed in algorithms in any way. Similarly, for unmanned aerial vehicles. The UTM systems used focus on drone-to-drone interactions; flight corridors of 30 regulation, altitude layering and the relationship between unmanned aerial vehicles Rules and automation systems have been developed to prevent collisions. However, this 2 systems that interact with autonomous ground traffic, drones in areas near ground level modeling the risk of collisions with road vehicles or generating a common optimization No modules are available. Drone collision avoidance algorithms are similarly designed only for the air environment. It is designed as follows: potential field methods, dynamic window approaches or 5 Reactive maneuver patterns describe the drone's interaction with other drones or obstacles in the air. It evaluates. However, the ground surface traffic under the drone, intersections, vehicle movement, Vehicle speed profiles or estimated vehicle locations are not taken into account. Urban Air Mobility (UAM) planning mostly involves vertiport layout, landing / takeoff scenarios, and It is based on airspace regulations. These systems are integrated with land transportation. 10 It does not have any management algorithm in place. The fundamental common deficiency of these current techniques is the lack of complete autonomous air and ground systems. In essence, they operate independently of each other. Drone corridors, intersections, bridges land-based areas such as hilltops, tunnel exits, roof landing areas or storage-distribution centers At points where their infrastructures overlap, the existing algorithmic structure allows for 15 different modes of transportation. They are unable to perceive each other and reach a common decision. Drones and ground vehicles vehicles' velocity vectors, acceleration capabilities, positional uncertainties, operational constraints And because there are significant differences in response times, these different dynamics cannot be understood in the same way. There is also no multimodal algorithm available that can optimize in the problem space. Another shortcoming of current aero-land systems is that they rely solely on central structures. 20 They are. Because UTM and C-ITS systems are mostly central server-based, Delays increase during peak traffic hours, and the capacity for real-time decision-making is diminished. The risk of collision between drones and autonomous ground vehicles in the same area is decreasing. The scenarios it presents require decision-making at the sub-second level. Local An Edge-AI based multi-agent orchestration model that utilizes computational power 25 Since they are not, the movements of drones and ground vehicles are brought under a common umbrella. It is unmanageable. These shortcomings result in various technical problems. The first problem is the air. and the risk of collision in common areas where road and land vehicles intersect. Drone descent or the landing route and the autonomous ground vehicle's movement route are on the same vertical axis 30 In the event of an intersection, neither system has any information about the other. The probability of collision increases. Another technical problem is co-optimization. The inability to achieve this. Drone and ground vehicle speed profiles, acceleration limits, stopping power... 3 Because distances, sensor delays, and position estimates cannot be processed together, a collision occurs. Detection and avoidance decisions are both delayed and suffer from a loss of credibility. Furthermore, the incompatibility of UTM protocols and V2X protocols means that the two systems This prevents them from sharing a common communication layer, and this situation affects air-land communication. This makes integrated traffic management impossible. In centralized systems, 5 Potential delays also exacerbate the technical problem, as the maneuvering of the drones Even millisecond delays in decision-making can pose serious security risks. In summary, current techniques exist at the architectural level, the algorithmic level, and the communication level. an integrated solution that will encompass air-ground interaction at the protocol level It does not offer this. These shortcomings pose serious safety risks, particularly in mixed mobility areas. 10 This leads to delay and coordination problems for autonomous air and land vehicles. This prevents its safe operation in the same environment. As a result of the research conducted on this subject, the case numbered US20230005378A1, “Conflict Detection and Avoidance for a Robot with Right-of-Way Rule Compliant Maneuver An application titled "Selection" was found. The system involves a robot and 15 nearby moving objects. By comparing the object's future trajectories, the collision is detected and the transit is identified. The right determines a proper evasive maneuver in accordance with the rules. However, the issue in question The application analyzes the movements of drones and ground vehicles within a common 3D–2D space-time framework. It does not integrate collision volume into the model; furthermore, multi-modal sensor fusion, risk scoring, multi-agent Edge-AI coordination, and coordinated 20 for air-to-ground vehicles. It does not offer maneuver optimization. In conclusion, due to the negative aspects described above and the current solutions being the subject of discussion... Due to its shortcomings, an improvement is needed in the relevant technical field. It has been made. Purpose of the Invention 25 The invention was created by drawing inspiration from existing situations and overcoming the aforementioned drawbacks. It aims to solve the problem. The main purpose of the invention is to enable autonomous ground vehicles and unmanned aerial vehicles to operate in shared areas. eliminating the risk of collision that arises from their simultaneous movement a multimodal, multi-agent and Edge-AI based combined traffic 30 that enables its removal The goal is to develop a management architecture. 4 Another purpose of the invention is to analyze motion, sensors, and other components related to drones, ground vehicles, and infrastructure. position estimation, velocity and acceleration data are processed within the same mathematical framework. The goal is to provide real-time collision detection and resolution. Another aim of the invention is to determine the future positions of air and land vehicles using a common method. Predicting on the space-time model and the overlap between the instruments in question 5 It is to calculate the probability probabilistically. Another aim of the invention is to combine data from drones and ground vehicles using sensor fusion. by combining them, the future positions and movement statuses of both vehicles are recorded continuously. It is about making predictions based on a common state model. Another aim of the invention is to combine the three-dimensional movement of the drone with the two-dimensional movement of the ground vehicle. by evaluating the movement within a shared collision volume, the potential collision risk The goal is to determine the locations and time of the collision. Another objective of the invention is to address velocity difference, positional uncertainty, height difference, and acceleration capacity. calculating a risk score that indicates the risk of collision using the parameters and... If the risk score exceeds the defined threshold, the avoidance algorithm will be activated. 15 It is to activate it automatically. Another purpose of the invention is to utilize drones, ground vehicles, and infrastructure units as agents. through a multi-agent artificial intelligence structure that defines air and land vehicles The goal is to produce coordinated evasive maneuvers. Another purpose of the invention is to reduce the risk of collision, maintain energy efficiency, 20 to avoid disrupting vehicle speed profiles and to minimize overall traffic flow Optimize feasible maneuvers by considering the impact on the targets together. to do. Another purpose of the invention is to enable the drone to change altitude, reduce speed, and shift course. Micro speed changes and short stops for land vehicles with options 25 The goal is to determine the appropriate solution space by evaluating the options. Another purpose of the invention is to utilize Edge-AI architecture to process data from drones and ground vehicles. By processing data on-site, we can generate collective decisions with millisecond delays and centralized... The goal is to create immediate solutions without being dependent on the system. Another objective of the invention is rapid reaction in low altitude-ground interaction regions. By doing so, we aim to reduce delays and eliminate the risk of collisions. Another purpose of the invention is to create urban drone corridors, autonomous vehicle lanes, and delivery robots. routes and bridges, intersections, tunnel entrances, underpasses, roof landing areas and depot-distribution Simultaneously managing air and land traffic at critical transportation infrastructures such as the central hub. It is to manage. Another aim of the invention is to predict the movements of autonomous vehicles on the ground and in the air. collision probabilities, height and distance relationships, traffic flow conditions, and safety By analyzing maneuvering options in real time, vehicles can avoid each other. The goal is to create dynamic solutions that prevent it from having an impact. 10 Another purpose of the invention is for logistics, emergency management, disaster areas, and air-ground integration. delivery operations, urban air mobility, connected and autonomous vehicles ecosystems, smart road infrastructures, airspace management, and V2X communication-based mobility The goal is to ensure the coordination of air and land vehicles in their networks. Another purpose of the invention is to create smart city mobility centers, industrial campuses, 15 Air and land autonomous systems in port operations and autonomous fleet management platforms. The goal is to ensure the common, real-time, and secure management of vehicles. Another objective of the invention is to create a system that can be integrated into existing UTM, C-ITS, and V2X infrastructures. an integrated autonomous mobility system that can be operated by public institutions or the private sector The goal is to provide a management system. 20 Another aim of the invention is to enable autonomous air and ground systems in urban and semi-urban environments. orchestration in a common and safe manner in industrial and institutional settings while ensuring energy efficiency, operational optimization and traffic safety. to increase. To achieve the purposes described above, the invention combines drones and ground vehicles. multimodal autonomous systems that enable safe and coordinated movement in various fields. It is a mobility system. Accordingly, the system;  drones, ground vehicles and infrastructure agents under a common decision-making mechanism At least one multi-agent Edge-AI that coordinates and generates real-time collaborative decisions. orchestration module, 30 6  A unified space-time model of the future positions of drones and ground vehicles At least one dynamic weather system determines the collision zone by making predictions based on the weather conditions. Land collision detection module,  Collision risk detected by the aforementioned dynamic air-ground collision detection module If detected, evasive maneuver 5 for drone and ground vehicle. at least one maneuver and solution optimization unit that calculates,  Speed, acceleration, altitude difference, sensor uncertainty related to drones and ground vehicles, and at least one risk assessment tool that calculates a risk score using collision time parameters. scoring and probability modeling engine,  Sensor data from drones and ground vehicles can be integrated into a single model (10). at least one multimodal sensor fusion layer that combines,  Position and velocity variables of drone and ground vehicle in a common state vector at least one common location that calculates their future positions by integrating them. prediction engine,  The three-dimensional movement volume of the drone and the two-dimensional path area of ​​the ground vehicle are combined in a common 15 at least one 3D–2D integrated collision volume that transforms into a collision zone model,  the aforementioned maneuver and solution are calculated by the optimization unit By transmitting evasive maneuvers to the drone and ground vehicle in real time. at least one tool and drone command / output interface that enables its implementation, 20  Real-time data from drones and ground vehicles in the field at least one infrastructure edge node that enables processing and collaborative decision-making. It includes. The invention also enables drones and ground vehicles to operate safely and coordinately in shared areas. also the multimodal autonomous mobility method that enables it to act as It includes the following method:  Multimodal sensor fusion of sensor data from drones and ground vehicles collected and combined by the layer and from the aforementioned sensor data Joint position-prediction 30 for drone and ground vehicle position and speed variables. by integrating a common state vector by the engine in the future Calculation of locations,  using the aforementioned future locations, the drone's three-dimensional flight volume 3D–2D integrated collision volume model of the two-dimensional road space of the land vehicle 7 matching within a common spacetime model and potential Identifying the collision zone,  by using the aforementioned potential collision region and future locations The collision time is determined by the dynamic air-ground collision detection module. Identifying and determining situations with a risk of conflict using Multi-Agent Edge-AI 5 transferring it to the orchestration module,  Speed, acceleration, altitude difference, sensor uncertainty related to drones and ground vehicles, and Collision time parameter risk scoring and probability modeling engine The risk score is calculated by processing the data.  Multi-agent Edge-AI orchestration module 10 for drones, ground vehicles and infrastructure agents coordinated by and capable of being carried out by drones and ground vehicles Identifying actions,  If the aforementioned risk score exceeds the critical threshold value, a collision will occur. avoidance maneuver that will eliminate the risk, maneuver and solution Energy consumption, latency, traffic integrity and 15 by the optimization unit Calculations taking into account the risk of collision.  The aforementioned avoidance maneuver vehicle and drone command / output interface the transmission of information to drones and ground vehicles and the implementation of the aforementioned processes infrastructure edge node in real time on site execution 20 It includes the steps involved in the process. The structural and characteristic features and all the advantages of the invention are given in the figures below. This becomes clearer thanks to the detailed explanation written with references to these figures. This will be understood as such, and therefore the evaluation will also be based on these forms and detailed explanations. 25 This should be done taking that into consideration. Figures that will help understand the invention. Figure 1 shows a schematic representation of the system that is the subject of the invention. Explanation of Part References 1. Multi-agent Edge-AI orchestration module 30 2. Dynamic air-ground collision detection module 8 3. Maneuver and solution optimization unit 4. Risk scoring and probability modeling engine 5. Multimodal sensor fusion layer 6. Common location-estimation engine 7. 3D–2D integrated collision volume model 5 8. Vehicle and drone command / output interface 9. Infrastructure Edge Node Detailed Description of the Invention This detailed explanation describes the preferred system and method for the invention. Their structures are explained solely for the purpose of better understanding the subject. 10 The invention enables drones and ground vehicles to move safely and coordinately in shared spaces. It is a multimodal autonomous mobility system that enables this. The invention is shown in Figure 1. A schematic representation of the system is provided. Accordingly, the system consists of a drone, a ground vehicle, and infrastructure. real-time collaboration by coordinating its agents under a common decision-making mechanism. decision-making at least one multi-agent Edge-AI orchestration module (1), drone and ground 15 Predicting the future positions of vehicles on a unified spacetime model by identifying at least one dynamic air-ground collision that determines the collision zone. module (2), by the aforementioned dynamic air-ground collision detection module (2) If a collision risk is identified, an avoidance maneuver will be performed for both the drone and the ground vehicle. at least one maneuver and solution optimization unit (3) calculating, drone and ground vehicle 20 related parameters such as speed, acceleration, height difference, sensor uncertainty, and collision time. at least one risk scoring and probability modeling engine that calculates a risk score using (4), combining sensor data from drone and ground vehicle into a single model at least one multimodal sensor fusion layer (5), position and speed of drone and ground vehicle By integrating the variables into a common state vector, their future positions can be determined. at least one common position-estimation engine (6) calculates the drone's three-dimensional volume of motion. at least the two-dimensional road space of the land vehicle that transforms the road surface into a common collision zone. a 3D–2D integrated collision volume model (7), Maneuvering and Resolution mentioned Avoidance maneuvers calculated by the Optimization Unit (3) for drone and ground at least one vehicle and drone 30 that transmits the information to the vehicle, enabling its real-time implementation. 9 Command / output interface (8), data from drones and ground vehicles in the field at least one infrastructure that enables real-time processing and collaborative decision-making. It contains edge nodes (9). The invention enables autonomous vehicles to move in both air and land environments within the same operational area. multimodal, multi-agent and safe and coordinated management within it The invention relates to an Edge-AI-based integrated traffic management system. The invention includes a drone. Motion, sensor, position, velocity, and acceleration obtained from land vehicles and infrastructure elements. The data is processed within a common mathematical structure; potential future occurrences Conflicts are being identified and coordinated efforts are being made to prevent the identified conflicts. Evasive maneuvers are being developed. The invention works by combining sensor data and determining future locations. calculation, identification of potential collision zone, collision time identification, calculation of risk score, coordination of agents and actions 15 determination, calculation of the evasive maneuver and the determined evasive maneuver The process consists of several steps involved in communicating and implementing the maneuver to the relevant vehicles. In the first step, sensor data from drones and ground vehicles is processed using multi-modal sensors. The sensor data is collected and combined by the fusion layer (5). The sensor data in question is 20 GNSS, lidar, camera and IMU data from the drone, and data from ground vehicles and infrastructure. The received data includes GNSS, lidar, camera, accelerometer, and V2X messages. Multimodal sensor. fusion layer (5), common data of different qualities obtained from air and land vehicles It integrates within a data structure. In a preferred application of the invention, the multimodal sensor fusion layer (5) Extended Kalman filter (EKF) for sensor fusion performed by, particle filter (PF) and deep learning-based time series prediction models LSTM or GRU is used. In the preferred application, the sensor data... Uncertainties are evaluated via a covariance matrix and the sensor noise is 30 However, the stability of location predictions is ensured. Sensor data combined by the multimodal sensor fusion layer (5) are common The position-estimation engine (6) is processed by the common position-estimation engine (6), drone and the position and speed variables of the land vehicle within a common state vector It integrates and calculates the future positions of both vehicles. 35 In a preferred application of the invention, a combination of drone and ground vehicle is created. The state vector is as follows: S(t) = [x_d(t), y_d(t), z_d(t), v_dx(t), v_dy(t), v_dz(t), x_g(t), y_g(t), v_gx(t), v_gy(t)]ᵀ Here, x_d(t), y_d(t) and z_d(t) represent the three-dimensional position of the drone; v_dx(t), v_dy(t) and 5 v_dz(t) represents the drone's velocity components; x_g(t) and y_g(t) represent the two-dimensional position of the ground vehicle; v_gx(t) and v_gy(t) represent the velocity components of the land vehicle. The state vector in question is the covariance matrix P(t) representing the sensor uncertainties. It is evaluated together with the drone and the land vehicle. The joint dynamic system model of the invention. In its preferred application, it is expressed by the following nonlinear differential function: 10 is done: Ṡ(t) = F(S(t), u_d(t), u_g(t)) + w(t) Here, u_d(t) are the thrust, yaw, pitch, and roll control inputs for the drone; u_g(t), the acceleration and steering angle control inputs of the land vehicle; w(t) is the Gaussian distribution system. It refers to the noise level. With this model, the dynamic constraints of the drone and the ground vehicle are the same. 15 It is being monitored within the context of the situation structure. In a preferred application of the invention, the drone's movement pattern is described as follows: is done: ṗ_d = v_d v̇_d = R(ψ, θ, φ) · T − g 20 The movement of the land vehicle is, in the preferred application, a kinematic bicycle model. It is modeled using the following method: ẋ_g = v_g · cos(θ_g) ẏ_g = v_g · sin(θ_g) v̇_g = a_g 25 θ̇_g = (v_g / L) · tan(δ) 11 The common position-estimation engine (6) is a hybrid EKF and LSTM based engine in the preferred application. It uses a prediction structure. In this structure, position estimation is performed as follows: Ŝ(t+1) = f(Ŝ(t)) + K(t) · [Z(t) − HŜ(t)] Here, Z(t) is the composite sensor derived from drone, ground vehicle and infrastructure sensors. data; K(t) represents the EKF gain. The LSTM model represents short-term momentum and 5 By learning the velocity patterns, the input estimation used by EKF is as follows: corrects: u_pred(t) = LSTM[Z(t−k:t)] In this way, the Common position-estimation engine (6) reduces sensor noise and position uncertainty. It calculates the future position and speed values ​​of the drone and ground vehicle below. 10 In the second processing step, the position obtained by the Common position-estimation engine (6) and Velocity estimates are evaluated by the 3D–2D integrated collision volume model (7). 3D– 2D integrated collision volume model (7) shows the three-dimensional flight volume of the drone and the ground vehicle. by mapping the two-dimensional path space within a common spacetime model, potential It determines the collision zone. 15 In a preferred application of the invention, the potential collision volume is as follows: It is defined as: C = {(x, y, z, t) | ‖p_d(t) − p_g(t)‖ ≤ r_safe} Here, p_d(t) is the drone's position vector; p_g(t) is the ground vehicle's position vector; r_safe This refers to the safe distance. The safe distance between the drone and the ground vehicle is 20. spacetime points where the distance is equal to or less than the safe distance, potential It is considered a collision zone. Another preferred application of the invention is the collision volume at a given moment. It is expressed as follows: C(t) = {(x, y, z) | ‖p_d(t) − p_g(t)‖₂ ≤ r_safe} 25 The 3D–2D integrated collision volume model (7) also shows the vertical difference between the drone and the ground vehicle. It includes risk assessment. Risk associated with vertical difference in the preferred practice. The density function is calculated as follows: 12 C_z(t) = e^(−|z_d(t) − z_min|) The effective collision volume depends on the drone's landing or take-off angle as follows: expanded: r_eff = r_safe + k · |tan(θ_d)| Thus, the drone's three-dimensional position, vertical movement, and landing or take-off angle are determined by the ground. the vehicle's two-dimensional position on the road within the same collision volume It is evaluated. In the next processing step, the 3D–2D integrated collision volume model (7) is used by the joint location-prediction engine (6) with the determined potential collision zone Calculated future position and velocity values ​​for dynamic air-ground collision detection 10 The dynamic air-ground collision detection module (2) is processed by the drone and By evaluating the future motion vectors of the ground vehicle, the collision time and It identifies situations where there is a risk of conflict. Dynamic air-ground collision detection module (2) in a preferred application of the invention. Predictive Collision Cone (PCC), Time-to-Collision (TTC), Velocity Obstacle (VO) and 15 It uses 3D-Relative Motion Analysis methods together. The relative speed between the drone and the ground vehicle is calculated as follows: v_r = v_d − v_g In the Predictive Collision Cone method, the collision cone is defined as follows: CC = {v_r | θ(v_r, p_r) < θ_safe} 20 Here, p_r represents the relative position between the drone and the ground vehicle; θ_safe is the safe angle. It expresses its value. The Time-to-Collision value is calculated as follows: TTC = ‖p_r‖ / ‖v_r‖ The Velocity Obstacle set is defined as follows: VO = {v_d | (v_d − v_g) ∈ CC} 13 Dynamic air-ground collision detection module (2), also between drone and ground vehicle It calculates the collision time when the distance is shortest as follows: t_c = arg min_t ‖p_d(t) − p_g(t)‖ By applying these methods together, the drone's three-dimensional movement and the ground vehicle's two-dimensional movement can be achieved. The time and location of collisions resulting from dimensional movement are determined. Collision Situations with risk are marked by Multi-agent Edge-AI orchestration module (1) It is transferred onto it. Data on overlap were collected by the risk scoring and probability modeling engine (4) 10 The risk scoring and probability modeling engine (4) is processed. The speed of the drone and land vehicle. difference, acceleration capacity, position estimation uncertainty, height difference, and collision time It creates a risk score by evaluating these parameters together. The risk score for a preferred application of the invention is determined using the following function: Calculated: 15 R = α · ‖v_d − v_g‖ + β · σ_p + γ · |Δh|⁻¹ + δ · t_c⁻¹ Here, ‖v_d − v_g‖ represents the speed difference between the drone and the ground vehicle; σ_p represents the position estimate. Δh represents the height difference, t_c represents the calculated collision time, and α, β, γ, and δ are the values ​​for the distance between the two points. It represents the weights applied to the relevant parameters. Another preferred application of the invention is risk scoring and probability modeling. 20 engine (4) calculates the total risk energy through the multivariate threat area integral It calculates as follows: R = ∫[t₀→t_c] {α / ‖p_r(t)‖ + β · σ_p(t) + γ / |Δh(t)| + δ · ‖v_r(t)‖} dt This integral represents the difference between the initial time t₀ and the collision time t_c between the drone and the ground. It represents the total risk value generated throughout the vehicle's dynamics. Risk scoring and 25 The risk score calculated by the probability modeling engine (4) of the vehicle agents The behavioral models are passed to the Infrastructure Edge node (9) to feed them. 14 The system includes a drone agent, a ground vehicle agent, and an infrastructure agent. Multi-agent Edge-AI orchestration. The module (1) is coordinated under a common decision-making mechanism. The agents operate as independent but coordinated artificial intelligence units. Multi-agent Edge-AI orchestration module (1), instantaneous status of the system and risk scoring and probability According to the risk score calculated by the modeling engine (4), each vehicle has 5 It determines the actions that can be taken. A preferred application of the invention is the multi-agent Edge-AI orchestration module (1), MADDPG, QMIX, or Multi-Agent PPO-based Multi-Agent Reinforcement Learning It utilizes algorithms. The reward function of the agents is to reduce the risk of collision, preserving energy efficiency, not disrupting vehicle speed profiles, and total traffic 10 It includes objectives to minimize the impact on the flow. The policy applied to each agent is expressed as follows: π_i(a_i | s_i) The common value function is calculated as follows: Q_tot(s, a) = Σ_i Q_i(s_i, a_i) 15 The QMIX mixing network combines the agent values ​​in the following way: Q_tot = f_mix(Q₁, Q₂, …, Q_n) Agent training loss is calculated as follows: L = [y_target − Q_tot]² Through this structure, the Multi-Agent Edge-AI orchestration module (1) can be used with drones, ground vehicles and 20 It determines the local and joint maneuver choices of infrastructure agencies. The risk score calculated by the risk scoring and probability modeling engine (4) If the value exceeds the critical threshold, the maneuvering and solution optimization unit (3) comes into play. Maneuver and solution optimization unit (3) eliminates the risk of collision. avoidance behavior will reduce energy consumption, delay, traffic integrity and collision 25 It calculates the risk by considering the parameters together. For drones, altitude change is Δz, speed decrease is Δv_d, and course shift is Δθ_d. The options include micro-speed change Δv_g and short-term pause for land vehicles. The options are evaluated. Maneuver and solution optimization unit in a preferred application of the invention (3) The optimization performed by [company name] is expressed as follows: min_(u_d,u_g) J = λ₁R + λ₂E_cost + λ₃ΔT + λ₄‖Δv‖ Here R is the risk calculated by the Risk scoring and probability modeling engine (4). E_cost represents the score, ΔT represents the energy cost, and Δv represents the delay in traffic flow. λ₁, λ₂, λ₃, and λ₄ represent the change in velocity and the weights applied to these parameters. It expresses. During optimization, the drone's altitude limit is applied as follows: z_min ≤ z_d ≤ z_max The road geometry constraint of the land vehicle is expressed as follows: 10 (x_g, y_g) ∈ lane(t) In the preferred application of the invention, the optimization problem in question is gradient descent, Sequential Quadratic Programming (SQP) or multi-agent learning-based action It is resolved by using policy. To determine the energy cost of the evasive maneuver related to the drone, the drone's energy 15 Consumption is calculated as follows: E = ∫[P_hover + P_move + P_asc / desc] dt This energy consumption allows the drone to remain stationary, move, and ascend in the air. Evasive maneuvers involving descent or landing are rated according to energy cost. The avoidance maneuvers determined by the maneuver and solution optimization unit (3) 20 The vehicle and drone are transmitted to the relevant vehicles via the command / output interface (8). Command / output interface (8), changing altitude, reducing speed or shifting course of the drone The command sends a micro-speed adjustment or short-term pause command to the land vehicle. In a preferred application of the invention, the Vehicle and drone command / output interface (8) to the drone It transmits commands to the ground vehicle via the V2D protocol, and to the drone via V2X. Vehicle and drone 25 Command / output interface (8), also MQTT or low latency Edge protocols They can communicate via this channel. Thus, the determined avoidance maneuvers can be performed by drone and ground vehicle. It is implemented in real time by the device. 16 Collecting, combining and processing sensor data, operating agents, collaborative The processes of generating the decision and transmitting the avoidance commands are carried out by the Infrastructure Edge node (9) It is carried out on the field. The Infrastructure Edge node (9) is both a road unit It also has a hybrid structure that functions as an air corridor unit and has low latency. The processor includes a local multi-agent learning module and a sensor collector. 5 In a preferred application of the invention, the Infrastructure Edge node (9) uses a processor, digital signal It consists of a processor, an artificial intelligence accelerator, and V2X communication components. It has a modular Edge structure: Node = {CPU, DSP, AI-Accelerator, V2X} Sensor fusion in the system enables prediction, detection, risk scoring, decision-making, and action. Its functions are run as independent microservices: Service = {Fuse, Predict, Detect, Score, Decide, Act} Total latency on the Infrastructure Edge node (9); receiving sensor data, the sum of the time taken for calculation and execution of the action It is expressed as follows: 15 T_latency = T_sense + T_compute + T_action In the preferred implementation of the invention, the total is used by using the Infrastructure Edge node (9). The delay time is ensured to be kept below the following value: T_latency < 5 ms Thus, data from drones and ground vehicles can be processed independently of a central system. processed in the field and necessary avoidance for low altitude-ground interaction zones. Decisions are made with low latency. In centralized systems, this value is 60–120 ms. That's possible. This is where the novelty of the invention becomes apparent. Within the scope of this invention, the combined traffic density of air and land vehicles can also be modeled. In a preferred application of the invention, the drone density is ρ_d and the ground vehicle density is 25. Hybrid traffic density is calculated using ρ_g as follows: ρ_hybrid = w₁ρ_d + w₂ρ_g 17 The propagation of a traffic wave is expressed by the following equation: ∂ρ_hybrid / ∂t + ∂(v · ρ_hybrid) / ∂x = 0 This model combines drone density and ground vehicle density to create a common traffic density. It is evaluated within this context. The system also includes a 5 for situations where the risk of collision cannot be kept within safe limits. It implements a transition to a safe state mechanism. Between the risk score and the critical risk threshold. A safe working condition is defined as follows: R < R_crit The safe transition mechanism in case the aforementioned condition is not met. It is triggered. During the transition to a safe state, the drone maintains a stationary position in the air, while the ground vehicle takes 10 minutes. then a stop command is generated: u_d = hover, u_g = stop These commands are given by the Vehicle and drone command / output interface (8) to the drone and ground. It is transmitted to the vehicle. As a result of these elements working together, the Multimodal Sensor Fusion Layer (5) and 15 Future drone and ground vehicles by the joint position-estimation engine (6) movements are predicted; 3D–2D integrated collision volume model (7) and Dynamic potential collision zones by air-ground collision detection module (2) The time of collision is determined; Risk scoring and probability modeling engine (4) Risk level is calculated by; Multi-agent Edge-AI orchestration module (1) and 20 Coordinated avoidance maneuvers by the maneuver and solution optimization unit (3) The vehicle and drone command / output interface (8) is being created and the maneuvers in question are being carried out. These processes are transmitted to the relevant vehicles via the Infrastructure Edge node (9). Air and land autonomous vehicles will conduct a joint operation in the field. It is managed in real-time and in a coordinated manner within the field. 25

Claims

18 REQUESTS 1. Safe and coordinated movement of drones and ground vehicles in shared areas. It is a multimodal autonomous mobility system that enables this, and its feature is;  drones, ground vehicles and infrastructure agents under a common decision-making mechanism Edge-AI 5, a multi-agent system that coordinates and generates real-time collaborative decisions. orchestration module (1),  A unified space-time model of the future positions of drones and ground vehicles At least one dynamic weather system determines the collision zone by making predictions based on the weather conditions. Land collision detection module (2),  Collision 10 by the aforementioned dynamic air-ground collision detection module (2) If a risk is identified, an evasive maneuver will be performed for the drone and ground vehicle. at least one maneuver and solution optimization unit that calculates (3),  Speed, acceleration, altitude difference, sensor uncertainty related to drones and ground vehicles, and at least one risk assessment tool that calculates a risk score using collision time parameters. scoring and probability modeling engine (4), 15  Sensor data from drones and ground vehicles within a single model combining at least one multimodal sensor fusion layer (5),  Position and velocity variables of drone and ground vehicle in a common state vector at least one common location that calculates their future positions by integrating them. prediction engine (6), 20  The three-dimensional movement volume of the drone and the two-dimensional path of the ground vehicle are combined at least one 3D–2D integrated collision volume that transforms into a collision zone model (7),  Calculated by the Maneuver and Solution Optimization Unit (3) mentioned above By transmitting evasive maneuvers to the drone and ground vehicle in real time, 25 at least one tool and drone command / output interface (8) that enables its implementation,  Real-time data from drones and ground vehicles in the field at least one infrastructure edge node that enables processing and collaborative decision-making. (9) It includes. 30 2. Safe and coordinated movement of drones and ground vehicles in shared areas. It is a multimodal autonomous mobility method that enables this, and its characteristic feature is; 19  Multimodal sensor fusion of sensor data from drones and ground vehicles collected and combined by layer (5) and the mentioned sensor Common data on position and speed variables of drones and ground vehicles position-estimation engine (6) in a common state vector Calculation of future positions by integration (1001), 5  using the aforementioned future locations, the drone's three-dimensional flight volume 3D–2D integrated collision volume model of the two-dimensional road space of the land vehicle (7) matching within a common spacetime model and potential Determination of the collision zone (1002),  Using the aforementioned potential collision region and future locations, 10 The collision time is determined by the dynamic air-ground collision detection module (2). Identifying and determining potential conflicts using Multi-Agent Edge-AI Transfer to orchestration module (1) (1003),  Speed, acceleration, altitude difference, sensor uncertainty related to drones and ground vehicles, and Collision time parameters risk scoring and probability modeling engine 15 Calculation of risk score by processing by (4) (1004),  Multi-agent Edge-AI orchestration module for drones, ground vehicles and infrastructure agents (1) coordinated by and can be carried out by drone and land vehicle determination of actions (1005),  If the aforementioned risk score exceeds the critical threshold, collision 20 avoidance maneuver that will eliminate the risk, maneuver and solution energy consumption, delay, traffic integrity and optimization unit (3) Calculation taking into account the risk of collision (1006),  The aforementioned evasion maneuver Vehicle and drone command / output interface (8) the transmission of information to drones and ground vehicles and the implementation of the aforementioned processes 25 real-time on-site by the infrastructure edge node (9) execution (1007) It includes the steps of the process.

Images