Unmanned aerial vehicle conflict resolution obstacle avoidance method

Abstract

The invention relates to an unmanned aerial vehicle conflict resolution obstacle avoidance method and device, electronic equipment and a storage medium. The method comprises the following steps: based on preset unmanned aerial vehicle parameters, establishing an unmanned aerial vehicle conflict resolution obstacle avoidance mathematical model; according to the unmanned aerial vehicle conflict resolution obstacle avoidance mathematical model, a preset NURBS curve is adopted to smooth the waypoints, and a multi-target optimization model of unmanned aerial vehicle conflict resolution obstacle avoidance is established; solving the multi-target optimization model for unmanned aerial vehicle conflict resolution obstacle avoidance by adopting a preset differential evolution algorithm to generate a multi-target optimization result; and optimizing a multi-target optimization result by adopting a preset deep Q learning algorithm to realize unmanned aerial vehicle conflict resolution and obstacle avoidance. According to the invention, improvement is carried out for unmanned aerial vehicle conflict resolution and obstacle avoidance, firstly, under the background of path planning, an improved NURBS curve is used to smooth a planned path, and curvature constraint is ensured; and meanwhile, a mode of combining deep Q learning and a natural heuristic algorithm is adopted, a planning part is independently considered, and the direction of the unmanned aerial vehicle is not regarded as a constraint.

Description

Background Art

[0002] A drone is an unmanned aerial vehicle (UAV) that is operated using a ground station radio remote control and a self - contained program control device. With the rapid development of technology and society, many scholars from all over the world have carried out a large number of studies in the fields of UAV flight attitude control technology, information processing technology, and radio communication technology. As a result, UAV flight control technology has developed rapidly, and interactive communication technology has become increasingly mature. Compared with traditional manned aircraft, UAVs have the characteristics of diverse payloads, light weight, small size, high mobility, strong adaptability, and low cost, and have been widely used in various fields of civil, industrial, and military applications. To improve the working efficiency of a single UAV, scholars from various countries have started to study the formation control algorithms for multiple UAVs. To ensure the stability of the formation flight of multiple UAVs, it is crucial to ensure the safe flight of each UAV during the formation process. Therefore, the conflict resolution and obstacle avoidance problems of UAVs are another issue to be solved. The formation obstacle avoidance of UAVs mainly includes two aspects. First is the obstacle avoidance problem between the UAV formation and external static or dynamic obstacles; second is the obstacle avoidance problem within the UAV formation itself. When multiple UAVs perform flight tasks, the UAVs need to accurately avoid obstacles while maintaining a certain safe distance between each other to prevent in - flight collisions. The research on the conflict resolution and obstacle avoidance technology for multiple UAVs is a prerequisite for applying multiple UAVs to the real environment and is also the core and difficulty in the research field of UAV formation control.

[0003] In recent years, many experts and scholars have proposed many different solutions and strategies for conflict resolution and obstacle avoidance to allow UAVs to fly safely and reliably in the environment, mainly including artificial potential field methods, geometric methods, graph sampling search methods, nature - inspired algorithms, and deep learning methods. Artificial potential field methods are usually used to solve real - time obstacle avoidance problems, where the potential field represents a virtual force field that guides the movement of UAVs. Geometric methods are usually based on Dubins paths and directly incorporate these kinematic constraints, but usually assume an obstacle - free environment. When extending these Dubins curves to 3D space, the problem of determining the shortest path lacks an analytical solution. If an obstacle - filled environment is considered, the algorithm complexity increases. In this scenario, graph - sampling - based search methods and nature - inspired algorithms are usually used. Graph sampling methods combine graph - based structures and sampling strategies to search for the optimal path based on a cost function. These algorithms usually do not guarantee a smooth path. Although nature - inspired algorithms perform well, they face the problem of premature convergence when particles converge to local optima. Deep learning algorithms have a long training time and high computational complexity.

[0004] Therefore, one or more methods are needed to solve the above problems.

[0005] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0006] An object of the present disclosure is to provide a method, device, electronic device, and computer-readable storage medium for UAV conflict resolution and obstacle avoidance, so as to overcome, at least to some extent, one or more problems caused by the limitations and defects of the related art.

[0007] According to one aspect of the present disclosure, there is provided a method for UAV conflict resolution and obstacle avoidance, including:

[0008] A mathematical model establishment step of establishing a mathematical model for UAV conflict resolution and obstacle avoidance based on preset UAV parameters;

[0009] A multi-objective optimization model establishment step of establishing a multi-objective optimization model for UAV conflict resolution and obstacle avoidance by using a preset NURBS curve to smooth waypoints according to the mathematical model for UAV conflict resolution and obstacle avoidance;

[0010] A differential evolution algorithm solution step of solving the multi-objective optimization model for UAV conflict resolution and obstacle avoidance by using a preset differential evolution algorithm to generate a multi-objective optimization result;

[0011] A deep Q-learning algorithm optimization step of optimizing the multi-objective optimization result by using a preset deep Q-learning algorithm to achieve UAV conflict resolution and obstacle avoidance.

[0012] In an exemplary embodiment of the present disclosure, the preset UAV parameters include:

[0013] The UAV center position coordinates, the Euler angles of roll, pitch, and yaw, the UAV position and yaw angle of the preset waypoints, the climb and dive function of the feasible path, and the obstacle avoidance area.

[0014] In an exemplary embodiment of the present disclosure, the method further includes:

[0015] The preset NURBS curve is an improved non-uniform rational B-spline curve Non-Uniform Rational B-Spline, and the formula is:

[0016]

[0017] Where P i =[x,y,z] T represents the point of the NURBS curve at s, w i is the weight, and B i,k(s) is a k-th order function of a normalized B-spline curve, which can be expressed by the formula v i is a non-decreasing knot vector consisting of M = (k + 1 + N) elements, with a value range of [0, 1].

[0018] In an exemplary embodiment of the present disclosure, the method further includes:

[0019] To generate the initial yaw angle (γ init , γ goal ) and the desired yaw angle (θ init , θ goal ), and define preset control points at the starting and ending points of zero curvature of the NURBS curve;

[0020] The formula of the preset control points is expressed as:

[0021]

[0022] where, P0 and P N-1 are the initial position and the final position respectively, and the constants α1 and α2 balance the distance between these points to obtain two pairs of collinear control points (P0, P1, P2) and (P N-3 , P N-2 , P N-1 ).

[0023] In an exemplary embodiment of the present disclosure, the method further includes:

[0024] Based on the preset NURBS curve and the preset control points, establish a multi-objective optimization model for UAV conflict resolution and obstacle avoidance as

[0025]

[0026] where, is the best free path parameter represented by the NURBS curve, f T is the cost function, defined as the weighted sum of polynomial functions, including the objective function f and the derived constraint penalty g, and the specific formula is where, f i is the objective function, g i is the penalty function, β = [β1,..., β 10 is the weight of these functions, indicating the importance of specific desired goals.

[0027] In an exemplary embodiment of the present disclosure, the preset differential evolution algorithm of the method includes:

[0028] Improve the parameters of the differential evolution algorithm, and add the scaling factor F and the crossover probability factor CR to achieve adaptive control;

[0029] Solve based on the improved differential evolution algorithm to generate multi-objective optimization results.

[0030] In an exemplary embodiment of the present disclosure, the preset deep Q-learning algorithm of the method includes:

[0031] Improve the state-value function and the action-value function of the deep Q-learning algorithm Deep Q-Learning;

[0032] The improvement of the state-value function can be expressed by the formula where E m is the expectation, m is the strategy selected by the agent, R t+1 is the reward expected to be obtained in the next time step, δ is the discount factor, ε is a specific state, and the state-value function is an estimate of the sum of rewards related to subsequent states. It is the total reward expected when reaching a certain state ε;

[0033] The formula for the improvement of the action-value function can be expressed as where μ · is the next action, Q * (ε, μ) can be approximated as Q(ε, μ) at time t, and the expected state at the next moment is denoted as The agent executes an action from the current state to transition to a subsequent state and obtain the corresponding reward R t+1 , and the improvement goal of the state-value function and the action-value function is to achieve the optimal value function.

[0034] In one aspect of the present disclosure, a UAV conflict resolution and obstacle avoidance device is provided, including:

[0035] A mathematical model establishment module for establishing a UAV conflict resolution and obstacle avoidance mathematical model based on preset UAV parameters;

[0036] A multi-objective optimization model establishment module for establishing a multi-objective optimization model for UAV conflict resolution and obstacle avoidance by using preset NURBS curves to smooth waypoints according to the UAV conflict resolution and obstacle avoidance mathematical model;

[0037] A differential evolution algorithm solving module for solving the multi-objective optimization model for UAV conflict resolution and obstacle avoidance by using a preset differential evolution algorithm to generate multi-objective optimization results;

[0038] A deep Q - learning algorithm optimization module is used to optimize the multi - objective optimization result by using a preset deep Q - learning algorithm to achieve UAV conflict resolution and obstacle avoidance.

[0039] In one aspect of the present disclosure, an electronic device is provided, including:

[0040] A processor; and

[0041] A memory, on which computer - readable instructions are stored, and when the computer - readable instructions are executed by the processor, the method according to any one of the above is implemented.

[0042] In one aspect of the present disclosure, a computer - readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the method according to any one of the above is implemented.

[0043] A method for UAV conflict resolution and obstacle avoidance in an exemplary embodiment of the present disclosure, wherein the method includes: based on preset UAV parameters, establishing a mathematical model for UAV conflict resolution and obstacle avoidance; according to the mathematical model for UAV conflict resolution and obstacle avoidance, using a preset NURBS curve to smooth waypoints and establishing a multi - objective optimization model for UAV conflict resolution and obstacle avoidance; solving the multi - objective optimization model for UAV conflict resolution and obstacle avoidance by using a preset differential evolution algorithm to generate a multi - objective optimization result; and optimizing the multi - objective optimization result by using a preset deep Q - learning algorithm to achieve UAV conflict resolution and obstacle avoidance. The present disclosure improves the UAV conflict resolution and obstacle avoidance. First, in the context of path planning, an improved NURBS curve is used to smooth the planned path and ensure curvature constraints; at the same time, a combination of deep Q - learning and nature - inspired algorithms is adopted, independently considering the planning part without regarding the UAV's direction as a constraint.

[0044] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] By referring to the drawings to describe its exemplary embodiments in detail, the above and other features and advantages of the present disclosure will become more obvious.

[0046] Figure 1 Shows a flowchart of a method for UAV conflict resolution and obstacle avoidance according to an exemplary embodiment of the present disclosure;

[0047] Figure 2 Shows a structural block diagram of a device for UAV conflict resolution and obstacle avoidance according to an exemplary embodiment of the present disclosure;

[0048] Figure 3A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is schematically shown;

[0049] Figure 4 A schematic diagram of a computer-readable storage medium according to an exemplary embodiment of the present disclosure is schematically shown. Detailed implementation manners

[0050] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. Like reference numerals in the figures denote like or similar parts, and thus their repetitive description will be omitted.

[0051] In addition, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, materials, devices, steps, etc. can be adopted. In other cases, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid obscuring aspects of the present disclosure.

[0052] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or these functional entities or a part of these functional entities can be implemented in one or more software-hardened modules, or these functional entities can be implemented in different networks and / or processor devices and / or microcontroller devices.

[0053] In the present example embodiment, first, a method for conflict resolution and obstacle avoidance of unmanned aerial vehicles is provided; as shown in Figure 1 the method for conflict resolution and obstacle avoidance of unmanned aerial vehicles may include the following steps:

[0054] Mathematical model establishment step S110, based on preset unmanned aerial vehicle parameters, establish a mathematical model for conflict resolution and obstacle avoidance of unmanned aerial vehicles;

[0055] Multi-objective optimization model establishment step S120, according to the mathematical model for conflict resolution and obstacle avoidance of unmanned aerial vehicles, use a preset NURBS curve to smooth waypoints, and establish a multi-objective optimization model for conflict resolution and obstacle avoidance of unmanned aerial vehicles;

[0056] Differential evolution algorithm solution step S130, use a preset differential evolution algorithm to solve the multi-objective optimization model for conflict resolution and obstacle avoidance of unmanned aerial vehicles, and generate a multi-objective optimization result;

[0057] In the optimization step S140 of the deep Q - learning algorithm, the multi - objective optimization result is optimized using a preset deep Q - learning algorithm to achieve UAV conflict resolution and obstacle avoidance.

[0058] A UAV conflict resolution and obstacle avoidance method in an exemplary embodiment of the present disclosure, wherein the method includes: establishing a UAV conflict resolution and obstacle avoidance mathematical model based on preset UAV parameters; according to the UAV conflict resolution and obstacle avoidance mathematical model, using a preset NURBS curve to smooth waypoints and establishing a multi - objective optimization model for UAV conflict resolution and obstacle avoidance; solving the multi - objective optimization model for UAV conflict resolution and obstacle avoidance using a preset differential evolution algorithm to generate a multi - objective optimization result; and optimizing the multi - objective optimization result using a preset deep Q - learning algorithm to achieve UAV conflict resolution and obstacle avoidance. The present disclosure improves UAV conflict resolution and obstacle avoidance. First, in the context of path planning, an improved NURBS curve is used to smooth the planned path and ensure curvature constraints; at the same time, a combination of deep Q - learning and nature - inspired algorithms is adopted, considering the planning part independently without regarding the UAV's orientation as a constraint.

[0059] Next, a UAV conflict resolution and obstacle avoidance method in the present exemplary embodiment will be further described.

[0060] In the mathematical model establishment step S110, a UAV conflict resolution and obstacle avoidance mathematical model can be established based on preset UAV parameters.

[0061] In the embodiment of the present example, the preset UAV parameters include:

[0062] The UAV center position coordinates, Euler angles of roll, pitch, and yaw, the UAV positions and yaw angles of preset waypoints, the climb - dive function of the feasible path, and the obstacle avoidance area.

[0063] In the embodiment of the present example, the UAV is regarded as a rigid spherical body with a radius r, six degrees of freedom, and kinematic constraints. Its center position is represented as (x, y, z), and the Euler angles of roll, pitch, and yaw are represented as The UAV positions and yaw angles of N waypoints can be expressed by the formula W = {[x1, y1, z1, γ1, θ1],..., [x N , y N , z N , γ N , θ N}, and the feasible path C(s) = [c x (s), c y (s), c z (s)] T, s = [1, 2, ..., N], the climb-dive function of the UAV along the feasible path can be expressed by the formula During the movement of the UAV, the area that needs to avoid obstacles is Therefore, by considering the radius of the UAV and extending it to the obstacle area accordingly, the center of the UAV must be located within and within the limited environment , which can be expressed by the formula During the mission execution, the center of the UAV must be located within so that multiple UAVs can avoid obstacles during flight. The specific formula can be expressed as where h safe is the safe altitude of the UAV flight, and z top represents the highest altitude during the mission execution.

[0064] In the multi-objective optimization model establishment step S120, according to the UAV conflict resolution and obstacle avoidance mathematical model, a preset NURBS curve can be used to smooth the waypoints to establish a multi-objective optimization model for UAV conflict resolution and obstacle avoidance.

[0065] In the embodiment of this example, the method further includes:

[0066] The preset NURBS curve is an improved non-uniform rational B-spline curve Non-Uniform Rational B-Spline, and the formula is:

[0067]

[0068] where P i = [x, y, z] T represents the point of the NURBS curve at s, w i is the weight, and B i,k (s) is the k-th function of the normalized B-spline curve, which can be expressed by the formula v i is a non-decreasing knot vector consisting of M = (k + 1 + N) elements, and the value range is [0, 1].

[0069] In the embodiment of this example, the method further includes:

[0070] To generate the initial yaw angles (γ init , γ goal ) and the desired yaw angles (θ init , θ goal ) required for UAV flight, and define preset control points at the starting and ending points with zero curvature of the NURBS curve;

[0071] The formula of the preset control point is expressed as:

[0072]

[0073] where P0 and P N-1 are the initial position and the final position respectively, and the constants α1 and α2 balance the distance between these points to obtain two pairs of collinear control points (P0, P1, P2) and (P N-3 , P N-2 , P N-1 ).

[0074] In the embodiment of this example, the method further includes:

[0075] Based on the preset NURBS curve and preset control points, a multi-objective optimization model for UAV conflict resolution and obstacle avoidance is established as

[0076]

[0077] where, is the optimal free path parameter represented by the NURBS curve, and f T is the cost function, which is defined as the weighted sum of polynomial functions, including the objective function f and the derived constraint penalty g. The specific formula is where f i is the objective function, g i is the penalty function, and β = [β1,..., β 10 are the weights of these functions, indicating the importance of specific desired objectives.

[0078] In the embodiment of this example, the present disclosure uses an improved NURBS curve to smooth N waypoints, and the specific formula can be expressed as where P i = [x, y, z] T represents the point of the NURBS curve at s, w i is the weight, and B i,k (s) is the k-th function of the normalized B-spline curve, which can be expressed by the formula as v i is a non-decreasing knot vector composed of M = (k + 1 + N) elements, and its value range is [0, 1]. In order to generate the initial yaw angle (γ init , γ goal ) and the desired yaw angle (θ init , θ goal ) of the UAV flight, and at the starting point and the ending point with zero curvature of the NURBS curve, the following control points are defined, and the specific formula can be expressed as Among them, P0 and P N-1 are the initial position and the final position respectively. The constants α1 and α2 balance the distance between these points to obtain two pairs of collinear control points (P0, P1, P2) and (P N-3 , P N-2 , P N-1 ). Combining the above steps of establishing the mathematical model and the steps of establishing the multi-objective optimization model, a multi-objective optimization model is established. The specific formula can be expressed as Among them, is the optimal free path parameter represented by the NURBS curve, and f T is the cost function, which is defined as the weighted sum of polynomial functions, including the objective function f and the derived constraint penalty g. The specific formula is Among them, f i is the objective function, g i is the penalty function, and β = [β1,..., β 10 is the weight of these functions, indicating the importance of specific expected goals.

[0079] The beneficial effects of the steps of establishing the multi-objective optimization model are as follows: By introducing the control point weight, the improved NURBS curve enhances the ability of the B-spline curve, supports the representation of non-rational curves (uniform weights) and ratio curves (variable weights). Due to the localized influence of the knot vector, the NURBS curve can also provide feasible solutions in the path planning optimization of complex environments. In addition, it provides a convenient method to merge multiple NURBS curves into one curve, which can ensure that the generated curve maintains curvature continuity.

[0080] In the differential evolution algorithm solving step S130, the multi-objective optimization model for the conflict resolution and obstacle avoidance of the UAV can be solved by using a preset differential evolution algorithm to generate a multi-objective optimization result.

[0081] In the embodiment of this example, the preset differential evolution algorithm of the method includes:

[0082] Improve the parameters of the differential evolution algorithm and add the scale factor F and the crossover probability factor CR to achieve adaptive control;

[0083] Solve based on the improved differential evolution algorithm to generate a multi-objective optimization result.

[0084] In the embodiments of this example, the present disclosure uses an improved differential evolution algorithm to solve the multi-objective optimization model, improves the parameters of the differential evolution algorithm, adds a scaling factor F and a crossover probability factor CR to achieve adaptive control. The scaling factor F scales the differences between selected individuals and determines the generated step size, and the crossover probability factor CR determines the probability of selecting each component of the trial solution from the mutant vector. The scaling factor F and the crossover probability factor CR are updated according to the historical memory of the success parameters. These historical memories are stored in the vectors M F =[M F,1 ,M F,2 ,...,M F,H and M CR =[M CR,1 ,M CR,2 ,...,M CR,H , where H is the size of the historical memory. For this purpose, whenever the mutant individual is better than the initial individual, its success parameters F i and CR i are respectively appended to the auxiliary vectors S F and S CR . In addition, the present invention uses linear population size reduction to dynamically adjust the population size to improve the performance of the differential evolution algorithm. The specific formula can be expressed as where |P c | G+1 is the population size of the G-th generation, G max is the maximum population size, and the operator int rounds the calculated result to the nearest integer. Therefore, |P c | G+1 is the population size of the next generation, |P c |0 is the size of the initial population, |P c | min is the possible minimum population size of the algorithm. The present invention sets |P c | min to 4.

[0085] When specifically solving the multi-objective optimization model given by the multi-objective optimization model establishment steps, the entire population P c of each generation can be divided into a feasible population when g = 0 and an infeasible population when g > 0. Then, the present invention uses a new mutation operator. The specific formula can be expressed as where and are individuals randomly selected from the union of the entire population P c and the historical memory storage population, They are the best 5% - 95% individuals in the population and are selected with uniform probability from the entire population in generation G when there are no feasible solutions in the population.

[0086] The beneficial effects of the solution steps of the differential evolution algorithm are as follows: The present invention uses an improved differential evolution algorithm to solve the multi-objective optimization model, which can improve the path quality, enhance the processing efficiency and the performance of the differential evolution algorithm, and reduce the computational complexity.

[0087] In the optimization step S140 of the deep Q-learning algorithm, the multi-objective optimization result can be optimized by using a preset deep Q-learning algorithm to achieve UAV conflict resolution and obstacle avoidance.

[0088] In the embodiment of this example, the preset deep Q-learning algorithm of the method includes:

[0089] Improve the state-value function and the action-value function of the deep Q-learning algorithm Deep Q-Learning;

[0090] The improvement of the state-value function can be expressed by the formula where E m is the expectation, m is the strategy selected by the agent, R t+1 is the reward expected to be obtained in the next time step, δ is the discount factor, ε is a specific state, and the state-value function is an estimate of the total reward related to subsequent states. It is the total reward expected when reaching a certain state ε;

[0091] The formula for the improvement of the action-value function can be expressed as where μ · is the next action, Q * (ε, μ) can be approximated as Q(ε, μ) at time t, and the expected state at the next moment is denoted as The agent executes an action from the current state to transition to the subsequent state and obtain the corresponding reward R t+1 , and the improvement goal of the state-value function and the action-value function is to achieve the optimal value function.

[0092] In the embodiment of this example, the present disclosure uses an improved deep Q-learning algorithm to optimize the result, mainly including the improvement of the state-value function and the action-value function. The improvement of the state-value function can be expressed by the formula where E m is the expectation, m is the strategy selected by the agent, R t+1The reward expected to be obtained for the next time step, δ is the discount factor, ε is a specific state. The state-value function is an estimate of the sum of rewards related to subsequent states. It is the total reward expected when reaching a certain state ε. The action-value function characterizes the expected gain obtained by the agent by taking a specific action μ in state ε. It evaluates the rationality of choosing a certain behavior in state s while adhering to policy p. The specific formula for improving the action-value function can be expressed as where μ · is the next action, and Q * (ε, μ) can be approximated as Q(ε, μ) at time t, and the expected state at the next moment is denoted as The agent executes an action from the current state and transitions to the subsequent state and obtains the corresponding reward R t+1 . The improvement goal of the state-value function and the action-value function is to reach the optimal value function, thereby optimizing the results of the differential evolution algorithm solving steps and realizing conflict resolution and obstacle avoidance.

[0093] In the embodiment of this example, in order to ensure path smoothness while meeting the curvature constraint and facilitate considering the UAV direction during conflict resolution and obstacle avoidance, the UAV conflict resolution and obstacle avoidance method of the present disclosure fully considers simple scenarios and complex scenarios in a multi-obstacle environment on the premise of meeting kinematic constraints, which can improve the path quality, increase the processing efficiency, and reduce the computational complexity.

[0094] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that these steps must be executed in that specific order, or that all the shown steps must be executed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.

[0095] The conflict resolution and obstacle avoidance device of the UAV. Referring to Figure 2 As shown, the UAV conflict resolution and obstacle avoidance device 200 may include: a mathematical model establishment module 210, a multi-objective optimization model establishment module 220, a differential evolution algorithm solving module 230, and a deep Q-learning algorithm optimization module 240. Among them:

[0096] The mathematical model establishment module 210 is used to establish a UAV conflict resolution and obstacle avoidance mathematical model based on preset UAV parameters;

[0097] The multi-objective optimization model establishment module 220 is configured to establish a multi-objective optimization model for UAV conflict resolution and obstacle avoidance by smoothing waypoints using a preset NURBS curve according to the UAV conflict resolution and obstacle avoidance mathematical model;

[0098] The differential evolution algorithm solving module 230 is configured to solve the multi-objective optimization model for UAV conflict resolution and obstacle avoidance using a preset differential evolution algorithm to generate a multi-objective optimization result;

[0099] The deep Q-learning algorithm optimization module 240 is configured to optimize the multi-objective optimization result using a preset deep Q-learning algorithm to achieve UAV conflict resolution and obstacle avoidance.

[0100] The specific details of each of the above UAV conflict resolution and obstacle avoidance device modules have been described in detail in the corresponding UAV conflict resolution and obstacle avoidance method, so they will not be elaborated here.

[0101] It should be noted that although several modules or units of the UAV conflict resolution and obstacle avoidance device 200 are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0102] In addition, in the exemplary embodiments of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0103] Those skilled in the art can understand that various aspects of the present invention can be implemented as a system, a method, or a program product. Therefore, various aspects of the present invention can be specifically implemented in the following forms, namely: a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module", or "system" here.

[0104] Next, refer to Figure 3 to describe the electronic device 300 according to this embodiment of the present invention. Figure 3 The electronic device 300 shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0105] As Figure 3 shown, the electronic device 300 is presented in the form of a general-purpose computing device. The components of the electronic device 300 may include, but are not limited to: at least one of the above processing units 310, at least one of the above storage units 320, a bus 330 connecting different system components (including the storage unit 320 and the processing unit 310), and a display unit 340.

[0106] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 310, so that the processing unit 310 executes the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of this specification. For example, the processing unit 310 can execute steps S110 to S140 as shown in Figure 1 .

[0107] The storage unit 320 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 3201 and / or a cache storage unit 3202, and may further include a read-only storage unit (ROM) 3203.

[0108] The storage unit 320 may also include a program / utility 3204 having a set (at least one) of program modules 3205. Such program modules 3205 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0109] The bus 330 may represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any bus structure in a variety of bus structures.

[0110] The electronic device 300 can also communicate with one or more external devices 370 (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 300, and / or communicate with any device that enables the electronic device 300 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 350. And, the electronic device 300 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 360. As shown in the figure, the network adapter 360 communicates with other modules of the electronic device 300 through the bus 330. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0111] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0112] In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having a program product stored thereon that can implement the above method of this specification. In some possible embodiments, various aspects of the present invention can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of this specification.

[0113] Reference Figure 4 As shown, a program product 400 for implementing the above method according to an embodiment of the present invention is described. It can be a portable compact disc read-only memory (CD-ROM) and includes program code, and can run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0114] The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0115] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than a readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0116] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0117] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0118] In addition, the above-mentioned drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present invention, and are not for limiting purposes. It is easy to understand that the processes shown in the above-mentioned drawings do not indicate or limit the chronological order of these processes. Additionally, it is also easy to understand that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0119] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

[0120] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A method for conflict resolution and obstacle avoidance of an unmanned aerial vehicle, characterized in that The method includes: A step of establishing a mathematical model, which is to establish a mathematical model for UAV conflict resolution and obstacle avoidance based on preset UAV parameters; A step of establishing a multi-objective optimization model, which is to smooth waypoints by using a preset NURBS curve according to the mathematical model for UAV conflict resolution and obstacle avoidance, and establish a multi-objective optimization model for UAV conflict resolution and obstacle avoidance; A step of solving by differential evolution algorithm, which is to solve the multi-objective optimization model for UAV conflict resolution and obstacle avoidance by using a preset differential evolution algorithm to generate a multi-objective optimization result; A step of optimizing by deep Q-learning algorithm, which is to optimize the multi-objective optimization result by using a preset deep Q-learning algorithm to achieve UAV conflict resolution and obstacle avoidance.

2. The method according to claim 1, characterized in that, In the method, the preset UAV parameters include: The central position coordinates of the UAV, the Euler angles of roll, pitch and yaw, the UAV positions and yaw angles of preset waypoints, the climb and dive functions of the feasible path, and the obstacle avoidance area.

3. The method according to claim 1, characterized in that, The method further includes: The preset NURBS curve is an improved Non-Uniform Rational B-Spline curve, and the formula is: where P i = [x, y, z] T represents the point of the NURBS curve at s, w i is the weight, and B i,k (s) is a k-th order function of the normalized B-spline curve, which can be expressed by the formula as v i is a non-decreasing knot vector consisting of M = (k + 1 + N) elements, with a value range of [0, 1].

4. The method according to claim 3, wherein The method further includes: To generate the initial yaw angle (γ init , γ goal ) and the desired yaw angle (θ init , θ goal ), and define preset control points at the start and end points of zero curvature of the NURBS curve; The formula representation of the preset control points is: where P0 and P N-1 are the initial and final positions respectively, and the constants α1 and α2 balance the distances between these points to obtain two pairs of collinear control points (P0, P1, P2) and (P N-3 , P N-2 , P N-1 ).

5. The method according to claim 4, wherein The method further includes: Based on the preset NURBS curve and preset control points, the established multi-objective optimization model for UAV conflict resolution and obstacle avoidance is Among them, is the best free path parameter represented by the NURBS curve, and f T is the cost function, defined as the weighted sum of polynomial functions, including the objective function f and the derived constraint penalty g. The specific formula is where f i is the objective function, g i is the penalty function, and β = [β1,..., β 10 is the weight of these functions, representing the importance of specific desired objectives.

6. The method according to claim 1, wherein The preset differential evolution algorithm of the method includes: Improving the parameters of the differential evolution algorithm, and adding a scaling factor F and a crossover probability factor CR to achieve adaptive control; Solving based on the improved differential evolution algorithm to generate a multi-objective optimization result.

7. The method according to claim 1, characterized in that, The preset deep Q-learning algorithm of the method includes: Improving the state-value function and action-value function of the deep Q-learning algorithm Deep Q-Learning; The improvement of the state-value function can be expressed by the formula v m = E m [R t+1 + where E m is the expectation, m is the policy selected by the agent, R t+1 is the reward expected to be obtained in the long term at the next time step, δ is the discount factor, ε is a specific state, and the state-value function is an estimate of the sum of rewards related to subsequent states. It is the total reward expected when choosing to reach a certain state ε; The improved formula for the action-value function can be expressed as where μ · is the next action, and Q * (ε, μ) can be approximated as Q(ε, μ) at time t, and the expected state at the next time step is denoted as The agent executes an action from the current state and transitions to the subsequent state and obtains the corresponding reward R t+1 , and the improvement objective of the state-value function and the action-value function is to achieve the optimal value function.

8. An unmanned aerial vehicle conflict resolution and obstacle avoidance device, characterized in that, The device includes: A mathematical model establishment module, which is used to establish a mathematical model for UAV conflict resolution and obstacle avoidance based on preset UAV parameters; A multi-objective optimization model establishment module, which is used to smooth waypoints by using a preset NURBS curve according to the mathematical model for UAV conflict resolution and obstacle avoidance, and establish a multi-objective optimization model for UAV conflict resolution and obstacle avoidance; A differential evolution algorithm solving module, which is used to solve the multi-objective optimization model for UAV conflict resolution and obstacle avoidance by using a preset differential evolution algorithm to generate a multi-objective optimization result; A deep Q-learning algorithm optimization module, which is used to optimize the multi-objective optimization result by using a preset deep Q-learning algorithm to achieve UAV conflict resolution and obstacle avoidance.

9. An electronic device, characterized in that, Including A processor; and A memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that, On which a computer program is stored, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images