Flight control method and device of flight equipment, storage medium and electronic equipment

Abstract

The embodiment of the invention provides a flight control method and device of flight equipment, a storage medium and electronic equipment, and the method comprises the steps: obtaining target sensor data collected by a sensor on target flight equipment under the condition that the target flight equipment in a plurality of pieces of flight equipment is located at a first task point in a plurality of task points; under the condition that the target flight equipment is about to fly to a second task point in the plurality of task points, performing state prediction on the target obstacle according to the observation data of the target obstacle to obtain a state prediction result; based on the state prediction result, planning an equipment flight path from the first task point to the second task point for the target flight equipment; and controlling the target flight equipment to fly from the first task point to the second task point according to the equipment flight path. Through the flight control method and device, the problem of low task execution efficiency of a flight control method of the flight equipment in the related technology is solved, and the effect of improving the task execution efficiency of the flight equipment is achieved.

Description

Technical Field

[0001] This application relates to the field of flight equipment technology, and more specifically, to a flight control method and apparatus, storage medium and electronic device for flight equipment. Background Technology

[0002] Performing missions with a single flying device has obvious drawbacks, including limited mission capabilities, limited endurance, and low efficiency. To improve mission efficiency, multiple flying devices can work together. However, in related technologies, allocating tasks to multiple flying devices is difficult to balance various indicators, and there may be interference from static or dynamic obstacles during mission execution, leading to low mission efficiency.

[0003] This shows that the flight control methods of flight equipment in related technologies suffer from low mission execution efficiency. Summary of the Invention

[0004] This application provides a flight control method and apparatus for flight equipment, a storage medium and an electronic device, to at least solve the technical problem of low task execution efficiency in flight control methods for flight equipment in related technologies.

[0005] According to one aspect of the embodiments of this application, a flight control method for a flight device is provided, comprising: when a target flight device among a plurality of flight devices is at a first task point among a plurality of task points, acquiring target sensor data collected by sensors on the target flight device, wherein the plurality of flight devices are flight devices that collaboratively execute multiple designated tasks, one of the designated tasks corresponds to one of the task points among the plurality of task points, and the target sensor data includes observation data of a target obstacle observed by sensors on the target flight device; when the target flight device is about to fly to a second task point among the plurality of task points, performing state prediction on the target obstacle based on the observation data of the target obstacle to obtain a state prediction result, wherein the state prediction result is used to indicate the predicted position of the target obstacle at each future time in a set of future time moments; based on the state prediction result, planning a device flight path for the target flight device from the first task point to the second task point, wherein the flight position corresponding to each future time moment on the device flight path does not intersect with the predicted position of the target obstacle at the same future time moment; and controlling the target flight device to fly from the first task point to the second task point according to the device flight path.

[0006] According to another aspect of the embodiments of this application, a flight control device for a flight device is also provided, comprising: an acquisition unit, configured to acquire target sensor data collected by sensors on the target flight device when the target flight device is at a first task point among multiple task points, wherein the multiple flight devices are flight devices that collaboratively execute multiple designated tasks, one of the multiple designated tasks corresponds to one of the multiple task points, and the target sensor data includes observation data of target obstacles observed by sensors on the target flight device; and a prediction unit, configured to predict when the target flight device is about to fly to a second task among the multiple task points. In the case of a point, based on the observation data of the target obstacle, a state prediction is performed on the target obstacle to obtain a state prediction result, wherein the state prediction result is used to indicate the predicted position of the target obstacle at each future time in a set of future time moments; a planning unit is used to plan a flight path for the target flight device from the first task point to the second task point based on the state prediction result, wherein the flight position corresponding to each future time moment on the flight path does not intersect with the predicted position of the target obstacle at the same future time moment; a first control unit is used to control the target flight device to fly from the first task point to the second task point according to the flight path.

[0007] In an exemplary embodiment, the state of the target obstacle includes the position and velocity of the target obstacle, and the observation data of the target obstacle includes observation values ​​corresponding to each historical moment in a set of historical moments; the prediction unit includes: a determination module, configured to determine the state evaluation value of the target obstacle, which is evaluated based on the observation value corresponding to the earliest historical moment in the set of historical moments, as the initial state evaluation value of the target obstacle; and a first execution module, configured to sequentially use the historical moments other than the earliest historical moment in the set of historical moments as specified historical moments to perform the following prediction operation to obtain the current state evaluation value of the target obstacle: based on a specified process model and the state evaluation value of the target obstacle at the previous historical moment at the specified historical moment, perform the following prediction operation on the target obstacle at the specified historical moment. A state prediction module is used to obtain a state prediction value of the target obstacle at a specified historical time. The state prediction value of the target obstacle at the specified historical time is then filtered and updated using the observation value corresponding to that historical time to obtain a state evaluation value of the target obstacle at that historical time. The current state evaluation value of the target obstacle is the state evaluation value of the target obstacle at the latest historical time in a set of historical times. The specified process model is used to describe the change of the state of the target obstacle over time. A prediction module is used to predict the state of the target obstacle at each future time based on the specified process model and the current state evaluation value of the target obstacle, obtaining a state prediction result. The state prediction result includes the state prediction value of the target obstacle at each future time.

[0008] In an exemplary embodiment, both the process noise corresponding to the specified process model and the measurement noise corresponding to the measurement model of the sensor on the target flight device are bounded noise. The prediction module includes: a prediction submodule, used to predict the state of the target obstacle at the specified historical time and the error covariance corresponding to the specified historical time based on the specified process model and the state evaluation value of the target obstacle at the previous historical time, to obtain the predicted state value of the target obstacle at the specified historical time and the prediction error covariance corresponding to the specified historical time. The execution module includes: a calculation submodule, used to calculate a filter gain matrix based on the predicted state value of the target obstacle at the specified historical time, the prediction error covariance corresponding to the specified historical time, and the observed value corresponding to the specified historical time; and an adjustment submodule, used to adjust the predicted state value of the target obstacle at the specified historical time based on the filter gain matrix and the measurement residual, to obtain the state evaluation value of the target obstacle at the specified historical time, wherein the measurement residual is the difference between the following two: the observed value corresponding to the specified historical time, and the product of the measurement model, the measurement matrix corresponding to the specified historical time, and the predicted state value of the target obstacle at the specified historical time.

[0009] In one exemplary embodiment, the apparatus further includes: an allocation unit, configured to allocate the plurality of designated tasks to the plurality of flight devices based on a task execution objective and task execution constraints, wherein the task execution objective is used to evaluate the task allocation quality of the plurality of designated tasks, the task execution constraints are used to limit the constraint conditions satisfied by the allocation of the plurality of designated tasks, and the task execution constraints include: the execution routes of different designated tasks among the plurality of designated tasks do not intersect spatially, and one designated task among the plurality of designated tasks is allocated to one flight device among the plurality of flight devices; and a second control unit, configured to control each flight device among the plurality of flight devices to sequentially fly to the task point corresponding to the designated task allocated to each flight device according to the execution order of the designated tasks allocated to each flight device, and execute the designated task allocated to each flight device.

[0010] In an exemplary embodiment, the task execution objective includes multiple execution objectives, each of which corresponds one-to-one with a target function in a plurality of objective functions, and each target function is used to quantify the corresponding execution objective; the allocation unit includes: a second execution module, configured to construct a set of sub-problems and generate an initial task allocation scheme corresponding to each sub-problem in the set of sub-problems, wherein a sub-problem in the set of sub-problems corresponds to a weight vector of the plurality of objective functions, the weight vectors corresponding to different sub-problems in the set of sub-problems, and the initial task allocation scheme corresponding to each sub-problem are all task allocation schemes for the plurality of execution tasks; and a third execution module, configured to iteratively use each sub-problem as the current sub-problem. The following update operations are performed until the update termination condition is met to obtain the candidate task allocation scheme corresponding to each sub-problem: based on the weight vector corresponding to the current sub-problem, multiple reference sub-problems are selected from the set of sub-problems; based on the initial task allocation scheme corresponding to each of the multiple reference sub-problems, the initial task allocation scheme corresponding to the current sub-problem is updated, wherein the candidate task allocation scheme corresponding to each sub-problem is the initial task allocation scheme corresponding to each sub-problem after the update termination condition is met; the fourth execution module is used to select the target task allocation scheme from the candidate task allocation schemes corresponding to each sub-problem, and allocate the multiple specified tasks to the multiple flight devices according to the target task allocation scheme.

[0011] In one exemplary embodiment, the third execution module includes: a selection submodule, configured to select two subproblems from the current subproblem and the plurality of reference subproblems; an execution submodule, configured to perform a crossover and mutation operation on the initial task allocation schemes corresponding to the two subproblems as parent task allocation schemes to obtain a child task allocation scheme, wherein the child task allocation scheme satisfies the task execution constraints; and an update submodule, configured to update the initial task allocation scheme corresponding to the target subproblem to the child task allocation scheme if the fitness of the child task allocation scheme with the target subproblem in the current neighborhood is higher than the fitness of the initial task allocation scheme corresponding to the target subproblem with the target subproblem.

[0012] In an exemplary embodiment, the planning unit includes: a fifth execution module, configured to, based on the state prediction result, use the first task point as the path start point, the second task point as the path end point, and a specified time as the starting time, perform the following path point search operation until the latest searched path point is reachable from the path end point. The latest path point being reachable from the path end point means that the flight path from the latest path point to the path end point does not intersect with the predicted movement path of the target obstacle in space and time. The predicted movement path of the target obstacle is the movement path of the target obstacle determined based on the predicted position of the target obstacle at each future time: randomly selecting a spatial point within a specified search space to obtain a random point, wherein the specified search space includes the path start point and the path end point; determining neighboring path points from the path map, and matching the neighboring path points with the random point. On the connecting line, spatial points at a specified distance from the neighboring path points are identified as candidate path points. The path map records the reachability relationships between at least one searched path point and the predicted arrival time of each of the at least one path points. The at least one path point includes the path origin. The neighboring path point is the path point closest to the random point among the at least one path points. Based on the predicted arrival time of the neighboring path points, a predicted flight period from the neighboring path point to the candidate path point and the predicted flight path corresponding to the predicted flight period are planned. The predicted flight path describes the correspondence between flight time and path points within the predicted flight period. If the predicted flight path and the predicted movement path of the target obstacle do not intersect in space and time, the candidate path point is added as a new path point to the path map.

[0013] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed by a processor.

[0014] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the steps in any of the method embodiments described above.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to perform the steps of any of the above method embodiments through the computer program.

[0016] This application describes a method for acquiring target sensor data collected by sensors on a target flight device when the target flight device is at a first task point among multiple task points. The multiple flight devices are flight devices that collaboratively execute multiple designated tasks, and one of the designated tasks corresponds to one of the task points. The target sensor data includes observation data of target obstacles observed by sensors on the target flight device. When the target flight device is about to fly to a second task point among the multiple task points, the state of the target obstacle is predicted based on the observation data, resulting in a state prediction result. This state prediction result indicates the predicted position of the target obstacle at each future moment in a set of future moments. Based on the state prediction result, a flight path is planned for the target flight device from the first task point to the second task point, where the flight position at each future moment on the flight path does not intersect with the predicted position of the target obstacle at the same future moment. The target flight device is then controlled to fly from the first task point to the second task point according to the flight path. Since the obstacle avoidance path is planned based on the prediction results before the flight equipment flies to the next mission point, there is no need to frequently replan the path or detour during the flight. This can solve the problem of low mission execution efficiency in the flight control methods of related technologies and improve the mission execution efficiency of the flight equipment. Attached Figure Description

[0017] Figure 1 This is a schematic diagram illustrating an application scenario of a flight control method for a flight device according to an embodiment of this application;

[0018] Figure 2 This is a schematic flowchart of an optional flight control method for a flight device according to an embodiment of this application;

[0019] Figure 3 This is a schematic diagram of an optional flight control method for a flight device according to an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of another optional flight control method for a flight device according to an embodiment of this application;

[0021] Figure 5 This is a schematic flowchart of another optional flight control method for a flight device according to an embodiment of this application;

[0022] Figure 6This is a structural block diagram of a flight control device for an optional flight equipment according to an embodiment of this application;

[0023] Figure 7 This is a computer system architecture block diagram of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] According to one aspect of the embodiments of this application, a flight control method for a flight device is provided. Optionally, in this embodiment, the flight control method for the flight device may be applied, but is not limited to, to applications such as... Figure 1 The diagram shows a hardware environment including multiple flight devices 102 and a server 104. The server 104 can connect to the multiple flight devices 102 via a wireless network and can be used to provide services (e.g., positioning services, control services, etc.) to the multiple flight devices 102. A database can be set up on or independently of the server 104. The multiple flight devices 102 can also connect to each other via a wireless network and communicate with each other via a wireless network.

[0027] The aforementioned wireless network may include, but is not limited to, at least one of the following: Wireless Fidelity (WIFI) and Bluetooth. Multiple flight devices 102 may be, but are not limited to, drones, swarm aircraft, micro-aircraft, etc. Server 104 may be, but is not limited to, a cloud server, a server cluster, or other server types.

[0028] The flight control method of the flight device in this embodiment can be executed by server 104, by flight device 102, or by both server 104 and flight device 102. Taking the execution of the flight control method of the flight device in this embodiment by flight device 102 as an example... Figure 2 This is a schematic flowchart of an optional flight control method for a flight device according to an embodiment of this application, as shown below. Figure 2 As shown, the process of this method may include the following steps:

[0029] Step S202: When the target flight device among multiple flight devices is at the first task point among multiple task points, acquire the target sensor data collected by the sensors on the target flight device. The multiple flight devices are flight devices that cooperate to execute multiple specified tasks. One of the specified tasks corresponds to one of the task points among multiple task points. The target sensor data includes the observation data of the target obstacle observed by the sensors on the target flight device.

[0030] Step S204: When the target flight equipment is about to fly to the second mission point among multiple mission points, the state of the target obstacle is predicted based on the observation data of the target obstacle, and the state prediction result is obtained. The state prediction result is used to indicate the predicted position of the target obstacle at each future moment in a set of future moments.

[0031] Step S206: Based on the state prediction results, plan the flight path of the target flight equipment from the first task point to the second task point, wherein the flight position corresponding to each future time on the flight path does not intersect with the predicted position of the target obstacle at the same future time.

[0032] Step S208: Control the target flight equipment to fly from the first mission point to the second mission point according to the equipment flight path.

[0033] The flight control method for the flight equipment in this embodiment can be applied to the field of flight equipment technology and to scenarios where multiple flight devices are controlled to perform tasks collaboratively.

[0034] With the rapid development of flight equipment, autonomous navigation, and sensor technology, flight equipment can be applied to perform tasks autonomously in various fields, such as environmental monitoring, rescue and reconnaissance, and agricultural and forestry irrigation. However, performing tasks with a single flight device has obvious drawbacks, such as limited mission capabilities, limited endurance, and low efficiency. In order to improve mission execution efficiency, multiple flight devices can work together to perform tasks.

[0035] However, the collaborative execution of missions by multiple flight devices involves complex task allocation and mission execution path planning. It is difficult to balance various indicators when allocating tasks among multiple flight devices, and there may be interference from static or dynamic obstacles during mission execution, resulting in low mission execution efficiency.

[0036] In related technologies, Dijkstra's algorithm and A0 algorithm are commonly used. Algorithms such as Q-learning algorithms are used to plan obstacle avoidance paths for flight equipment. However, these algorithms require frequent replanning for dynamic obstacles, which consumes a lot of resources in scenarios where multiple flight devices are working together. This is difficult to support on hardware, resulting in poor obstacle avoidance path planning and consequently affecting the efficiency of the mission.

[0037] To at least partially solve the above-mentioned technical problems, in this embodiment, the flight device collects sensor data and predicts obstacle status when it reaches a task point, then plans an obstacle avoidance path based on the predicted obstacle status, and flies to the next task point along the planned obstacle avoidance path to perform the task. Since the obstacle avoidance path is planned before flight, it does not need to frequently replan the path during flight, which can reduce the consumption of computing resources and improve the execution efficiency of the task in multi-flight device scenarios.

[0038] In this embodiment, when the target flight device among multiple flight devices is located at the first task point among multiple task points, target sensor data collected by sensors on the target flight device is acquired. Here, the multiple flight devices are flight devices collaboratively executing multiple designated tasks, and one of the designated tasks corresponds to one of the task points. The target sensor data includes observation data of target obstacles observed by sensors on the target flight device. Here, the multiple flight devices are flight devices collaboratively executing multiple designated tasks, and each flight device is assigned one or more of the designated tasks. The task assigned to each flight device includes a task point, which is the execution location for the corresponding designated task.

[0039] It should be noted that the task types and execution methods for multiple designated tasks can differ depending on the flight equipment or application scenario. Taking drones as an example, application scenarios for multi-drone collaborative work can include, but are not limited to, multiple drone models performing different tasks, and single-scenario multi-task operations. Multiple drone models performing different tasks can include multiple scenarios such as land irrigation, pesticide spraying, environmental monitoring, and agricultural and forestry inspection. Single-scenario multi-task operations can include photographing and inspecting farmland growth, irrigating farmland, spraying pesticides, and other application scenarios. In some examples of this embodiment, multi-drone multi-task inspection is used for explanation. Multi-drone multi-task inspection refers to multiple drones inspecting multiple task points. Each drone will inspect multiple task points, ensuring that no task points are missed or inspected repeatedly.

[0040] Optionally, when the target flying device is located at the first mission point among multiple mission points, target sensor data can be collected by sensors such as millimeter-wave radar, binocular vision, and ultra-wideband (UWB) positioning module. Among them, millimeter-wave radar can obtain the distance and azimuth of obstacles, binocular vision can identify the outline and movement trend of obstacles, and UWB positioning module can perform high-precision spatiotemporal calibration.

[0041] When the target flight equipment is about to fly to the second mission point out of multiple mission points, the state of the target obstacle is predicted based on the observation data of the target obstacle, and the state prediction result is used to indicate the predicted position of the target obstacle at each future time in a set of future time points. Here, the state prediction of the target obstacle detected by the target flight equipment is based on the observation data of the target obstacle. Optionally, the target obstacle can be an obstacle within a specified range around the shortest path from the first mission point to the second mission point, avoiding the consumption of additional resources for predicting irrelevant obstacles. The set of future time points is all future time points within a specified future time period, that is, all future time points before the specified future time point. Optionally, the expected arrival time of the target flight equipment to the second mission point can be predicted first, and this can be used as the specified future time point. Then, the predicted position of the target obstacle at each future time point before the specified future time point can be predicted to obtain the state prediction result, avoiding the additional resource consumption caused by redundant predictions. Optionally, the state prediction can use Kalman filtering algorithm, particle filtering algorithm, deep learning algorithm, or set-valued filtering algorithm, etc.

[0042] Based on the state prediction results, a flight path is planned for the target flight equipment from the first task point to the second task point. The flight position at each future time step on the equipment's flight path does not intersect with the predicted position of the target obstacle at the same future time step. After obtaining the state prediction results of the target obstacle, a flight path can be planned for the flight equipment from the first task point to the second task point. Optionally, a path planning algorithm (such as genetic algorithm, dynamic programming, RRT algorithm, A) can be used. The algorithm, combining the predicted positions of obstacles, the performance parameters of the flight equipment (turning speed, flight speed, etc.), and the positions of the starting and ending points, generates a trajectory that avoids collisions with the predicted positions of obstacles at any given time during flight, and without unnecessary detours. Here, the flight position at each future time on the equipment's flight path does not intersect with the predicted position of the target obstacle at the same future time. That is, when the flight equipment flies along the equipment's flight path, its position at each future time will not coincide with the predicted position of the target obstacle at the same time. The flight equipment can continuously avoid the target obstacle and proceed to the second task point through the equipment's flight path. One way to ensure that the flight position at each future time on the equipment's flight path does not intersect with the predicted position of the target obstacle at the same future time can be: when planning the equipment's flight path, avoid the predicted position of the target obstacle at each future time, or avoid the obstacle avoidance zone centered on the predicted position of the target obstacle at that future time.

[0043] After path planning is completed, the target flight device is controlled to fly from the first task point to the second task point according to the device's flight path. Optionally, the aforementioned path planning can be performed by a remote server, that is, the flight device collects data through sensors and uploads the data to the server, which then performs obstacle state prediction and path planning, and generates corresponding control commands to control the flight device to the second task point; alternatively, the flight device itself can perform the planning, that is, after collecting data through sensors, the flight device performs obstacle state prediction and path planning locally, and controls itself to move to the second task point. This embodiment does not limit this approach. Optionally, during the flight of the target flight device from the first task point to the second task point according to the device's flight path, the sensors can continuously collect data, and when a collision with an approaching obstacle is detected, the device's flight path can be temporarily fine-tuned to avoid the obstacle, further ensuring safety during flight.

[0044] According to the embodiments provided in this application, when a target flight device is located at a first task point among multiple task points, target sensor data collected by sensors on the target flight device is acquired. The multiple flight devices are flight devices that collaboratively execute multiple designated tasks, and one of the designated tasks corresponds to one of the task points. The target sensor data includes observation data of target obstacles observed by sensors on the target flight device. When the target flight device is about to fly to a second task point among the multiple task points, the state of the target obstacle is predicted based on the observation data of the target obstacle, resulting in a state prediction result. The state prediction result indicates the predicted position of the target obstacle at each future moment in a set of future moments. Based on the state prediction result, a flight path is planned for the target flight device from the first task point to the second task point, wherein the flight position corresponding to each future moment on the flight path does not intersect with the predicted position of the target obstacle at the same future moment. The target flight device is controlled to fly from the first task point to the second task point according to the flight path. Since the obstacle avoidance path is planned based on the prediction results before the flight equipment flies to the next mission point, there is no need to frequently replan the path or detour during the flight. This solves the problem of low mission execution efficiency in the flight control methods of related technologies and improves the mission execution efficiency of the flight equipment.

[0045] In one exemplary embodiment, the state of the target obstacle includes the position and velocity of the target obstacle, and the observation data of the target obstacle includes observations corresponding to each historical moment in a set of historical moments.

[0046] To accurately predict the future position of a target obstacle, its state includes not only its position but also its velocity, allowing for analysis of its motion. The obstacle's observation data includes observations corresponding to each historical moment in a set of historical data. This historical sensor data can be used to calibrate parameters in state prediction; for example, using real data from one historical moment to predict the state at the next historical moment, and then combining this real data with the next historical moment's data to calibrate the parameters in state prediction, thereby improving the accuracy of state prediction.

[0047] Alternatively, the state of the target obstacle can be as shown in formula (1):

[0048] (1)

[0049] Where, x k Let p be the 6-dimensional state of the target obstacle at time k. x p y p zThese represent the positions of the target obstacle in the x, y, and z directions, respectively, and v x v y v z Let x, y, and z be the velocities of the target obstacle in the x, y, and z directions, respectively, and T be the transpose of the vector, meaning that the state vector of the target obstacle is a column vector.

[0050] The observed values ​​(observation equations) can be represented as shown in formula (2):

[0051] (2)

[0052] Among them, y k Let C be the sensor data corresponding to time k. k For the observation model at time k, v k Let be the observation noise at time k.

[0053] Correspondingly, based on the observation data of the target obstacle, the state of the target obstacle is predicted to obtain the state prediction result, including: determining the initial state evaluation value of the target obstacle, which is evaluated based on the observation value corresponding to the earliest historical moment in a set of historical moments; performing the following prediction operation on the historical moments other than the earliest historical moment in a set of historical moments as specified historical moments to obtain the current state evaluation value of the target obstacle: predicting the state of the target obstacle at the specified historical moment based on the specified process model and the state evaluation value of the target obstacle at the previous historical moment in the specified historical moment to obtain the current state evaluation value of the target obstacle in the specified historical moment. The state prediction value at a specified historical moment is obtained by filtering and updating the state prediction value of the target obstacle at the specified historical moment using the observation value corresponding to the specified historical moment. The current state evaluation value of the target obstacle is the state evaluation value of the target obstacle at the latest historical moment in a set of historical moments. A specified process model is used to describe the change of the state of the target obstacle over time. Based on the specified process model and the current state evaluation value of the target obstacle, the state of the target obstacle is predicted sequentially at each future moment to obtain the state prediction result. The state prediction result includes the state prediction value of the target obstacle at each future moment.

[0054] To improve the accuracy of predicting the future state of a target obstacle, the model's prediction accuracy can be gradually improved by iterating based on historical data until the current data is used. The iterated model can then be used to predict the future state of the target obstacle, thus obtaining an accurate prediction result.

[0055] In this embodiment, the initial state evaluation value of the target obstacle, determined based on the observation value corresponding to the earliest historical moment in a set of historical moments, is defined as the initial state evaluation value of the target obstacle. Here, the state evaluation value of the target obstacle is the evaluation state of the obstacle at the next moment, and the initial evaluation state is the evaluation state at the moment following the earliest historical moment. Optionally, the earliest moment in a set of historical moments can be the earliest moment with sensor data stored locally on the flight equipment, or the number of iterations required can be determined based on the requirements for prediction accuracy and performance conditions (the higher the accuracy requirement and the better the performance conditions, the more iterations are needed) to determine the earliest historical moment (the more iterations, the earlier the earliest historical moment). This embodiment does not limit this.

[0056] The following prediction operation is performed sequentially on each historical moment in a set of historical moments, excluding the earliest historical moment, to obtain the current state evaluation value of the target obstacle. Here, the prediction operation involves predicting the state evaluation value of the next historical moment based on the actual data of one historical moment, and then adjusting the parameters used in the prediction based on the deviation between the state evaluation value of the next historical moment and the actual data of the next historical moment. The parameters are iteratively updated from the earliest historical moment to the current moment.

[0057] The prediction operation includes: taking each historical moment in a set of historical moments except the earliest historical moment as a specified historical moment and performing the following prediction operation to obtain the current state evaluation value of the target obstacle: based on the specified process model and the state evaluation value of the target obstacle at the previous historical moment at the specified historical moment, predicting the state of the target obstacle at the specified historical moment to obtain the state prediction value of the target obstacle at the specified historical moment; using the observation value corresponding to the specified historical moment to filter and update the state prediction value of the target obstacle at the specified historical moment to obtain the state evaluation value of the target obstacle at the specified historical moment, wherein the current state evaluation value of the target obstacle is the state evaluation value of the target obstacle at the latest historical moment in a set of historical moments, and the specified process model is used to describe the change of the state of the target obstacle over time.

[0058] Based on the specified process model and the state evaluation value of the target obstacle at the previous historical moment at the specified historical moment, the state of the target obstacle at the specified historical moment is predicted, and the predicted state value of the target obstacle at the specified historical moment is obtained. The specified process model is used to describe the change of the state of the target obstacle over time. Here, the specified process model may include a constant velocity model or a constant acceleration model, or a combination of the two models. The specified process model can be gradually updated to a process model that better matches the actual motion state of the target obstacle in the iteration of the prediction operation. The predicted state value of the target obstacle at the specified historical moment is the position and velocity of the target obstacle at the specified historical moment. Here, the state equation for predicting the state of the target obstacle at the specified historical moment can be as shown in formula (3):

[0059] (3)

[0060] Among them, A k For a specified process model at time k, w k+1 The process noise is at time k+1 (i.e., the time after time k).

[0061] Furthermore, the formula for predicting the state of the target obstacle at a specified historical moment can be shown in formula (4):

[0062] (4)

[0063] in, This represents the predicted state of the target obstacle at a specified historical time (here, time k+1). The state evaluation value of the previous historical moment for the specified historical moment.

[0064] The predicted state value of the target obstacle at the specified historical time is filtered and updated using the observation value corresponding to the specified historical time to obtain the state evaluation value of the target obstacle at the specified historical time. The current state evaluation value of the target obstacle is the state evaluation value of the target obstacle at the latest historical time in a set of historical times.

[0065] Through the above prediction operations, the model can be iterated from the earliest historical moment in a set of historical moments to the current moment. After that, based on the specified process model and the current state of the target obstacle (i.e., formula (3)), the state of the target obstacle at each future moment is predicted in turn to obtain the state prediction result. The state prediction result is used to indicate the predicted state of the target obstacle at each future moment. Here, after predicting the state of the target obstacle at the next future moment (e.g., moment k+1), the state of the target obstacle at the next moment after that moment (i.e., moment k+2) can be predicted based on the predicted state of the target obstacle. The state of the target obstacle at each future moment is then predicted in a recursive manner to obtain the state prediction result.

[0066] In this embodiment, by iteratively updating the model using historical data and then predicting the future state of the target obstacle based on the updated model, the accuracy of predicting the state of the target obstacle can be improved.

[0067] In an exemplary embodiment, the filtering method used to update the predicted state value of the target obstacle at a specified historical moment can be selected as needed, such as Kalman filtering. Considering that the obstacles encountered by the flight equipment during flight may be static or dynamic, and the probability of encountering dynamic obstacles is higher, requiring dynamic obstacle avoidance, set-valued filtering can be used to improve the rationality of filtering. Here, set-valued filtering refers to an algorithm for estimating and predicting the position and velocity of dynamic obstacles. Set-valued filtering has a more relaxed requirement for the unknown but bounded measurement noise compared to the normal distribution requirement of Kalman filtering, which is more in line with practical needs. Dynamic obstacle avoidance refers to obstacle avoidance when the type of obstacle is clearly defined, including both static and dynamic obstacles.

[0068] When using set-valued filtering to update the predicted state of a target obstacle at a specified historical moment, both the process noise corresponding to the specified process model and the measurement noise corresponding to the measurement model of the sensors on the target flight equipment are bounded noise. Set-valued filtering can efficiently process unknown but bounded noise to provide the optimal estimate for the dynamic system. It can process the continuous state of the dynamic system in real time with limited memory and computational resources, enabling real-time dynamic obstacle avoidance. The flight control method for the flight equipment provided in this embodiment can integrate path planning with accurate and real-time dynamic obstacle prediction methods, making it applicable to obstacle avoidance scenarios with multiple dynamic obstacles in complex airspace.

[0069] In this embodiment, based on a specified process model and the state evaluation value of the target obstacle at the previous historical time, the state of the target obstacle at the specified historical time is predicted to obtain the predicted state value of the target obstacle at the specified historical time. This includes: based on the specified process model and the state evaluation value of the target obstacle at the previous historical time, the state of the target obstacle at the specified historical time and the error covariance corresponding to the specified historical time are predicted to obtain the predicted state value of the target obstacle at the specified historical time and the prediction error covariance corresponding to the specified historical time.

[0070] The calculation of the filter gain matrix is ​​a step in the set-valued filtering algorithm. The filter gain matrix determines the contribution of measurement information to state updates, ensuring the accuracy of state estimation. To improve the accuracy of filter gain matrix determination, it can be dynamically determined at different times. The filter gain matrix is ​​related to the prediction error covariance matrix. Therefore, besides predicting the state of the target obstacle at a specified historical time based on a given process model and the target obstacle's state evaluation value at the previous historical time, we can also predict the error covariance corresponding to a specified historical time based on the given process model and the target obstacle's state evaluation value at the previous historical time. Here, the prediction error covariance matrix refers to the uncertainty measure of the state estimation at the next time step after applying the state transition equation (i.e., the process model). It comprehensively considers the uncertainty of the current state estimation (i.e., the error covariance matrix of the previous time step), the uncertainty and noise influence of the process model itself, and the dynamic characteristics of the state transition equation. The prediction error covariance matrix reflects the expected error of the next state estimation in the absence of new observation information.

[0071] Correspondingly, the predicted state value of the target obstacle at the specified historical time is filtered and updated using the observation value corresponding to the specified historical time to obtain the state evaluation value of the target obstacle at the specified historical time. This includes: calculating the filtering gain matrix based on the predicted state value of the target obstacle at the specified historical time, the prediction error covariance corresponding to the specified historical time, and the observation value corresponding to the specified historical time; and adjusting the predicted state value of the target obstacle at the specified historical time based on the filtering gain matrix and the measurement residual to obtain the state evaluation value of the target obstacle at the specified historical time.

[0072] The filter gain matrix can be calculated based on the predicted state of the target obstacle at a specified historical time, the prediction error covariance at that specified historical time, and the observed value at that specified historical time. The calculated filter gain matrix is ​​the filter gain matrix at the specified historical time. The process of calculating the filter gain matrix can be as follows: calculate the innovation covariance (the innovation covariance matrix, i.e., the covariance matrix of the measurement residuals, where innovation refers to the measurement residuals), and use the prediction error covariance and the innovation covariance to calculate the filter gain matrix. The innovation covariance can be calculated by combining the prediction error covariance and the measurement noise covariance.

[0073] Optionally, the measurement residual can be calculated first, which can be the sensor measurement value (y) at time k+1 (e.g., a specified historical time). k+1 The difference between the predicted measurement value and the predicted measurement value based on the predicted state is determined as the measurement residual at time k+1. The covariance matrix of the measurement residual is then updated to obtain the covariance matrix of the measurement residual at time k+1. The predicted measurement value based on the predicted state is the measurement value obtained by substituting the predicted state at time k+1 into the measurement model. Then, the innovation covariance matrix at time k+1 is calculated. The innovation covariance matrix at time k+1 can be the product of the measurement Jacobian matrix at time k+1, the prediction error covariance matrix at time k+1, and the transpose of the measurement Jacobian matrix at time k+1, plus the covariance matrix of the measurement residual at time k+1. Finally, the product of the prediction error covariance matrix at time k+1, the transpose of the measurement Jacobian matrix at time k+1, and the inverse of the innovation covariance matrix at time k+1 is determined as the filter gain matrix at time k+1.

[0074] Based on the filter gain matrix and measurement residuals, the predicted state value of the target obstacle at a specified historical time can be adjusted to obtain the state evaluation value of the target obstacle at the specified historical time. Here, the measurement residual is the difference between the following two: the observation value corresponding to the specified historical time, and the product of the measurement model, the measurement matrix corresponding to the specified historical time, and the predicted state value of the target obstacle at the specified historical time.

[0075] Here, based on the filter gain matrix and the measurement residual, the way to adjust the predicted state value of the target obstacle at a specified historical moment can be as shown in formula (5):

[0076] (5)

[0077] Among them, y k+1 The observation value corresponding to time k+1 (i.e., a specified historical time) can be directly obtained from the target sensor data. k+1The filter gain matrix at time k+1 can be obtained by minimizing the state estimation confidence interval, C k+1 This is the measurement matrix corresponding to the measurement model and a specified historical moment.

[0078] In this embodiment, based on unknown but bounded process noise and observation noise, set-valued filtering is used to filter and update the predicted state value of the target obstacle at a specified historical moment, thereby improving the reliability of dynamic obstacle avoidance.

[0079] In an exemplary embodiment, the method further includes: assigning multiple specified tasks to multiple flight devices based on task execution objectives and task execution constraints, wherein the task execution objectives are used to evaluate the task assignment quality of the multiple specified tasks, and the task execution constraints are used to limit the constraint conditions satisfied by the assignment of multiple specified tasks, the task execution constraints including: the execution routes of different specified tasks among the multiple specified tasks do not intersect spatially, and one specified task among the multiple specified tasks is assigned to one flight device among the multiple flight devices; controlling each flight device among the multiple flight devices to fly sequentially to the task point corresponding to the specified task assigned to each flight device according to the execution order of the specified tasks assigned to each flight device, and to execute the specified task assigned to each flight device.

[0080] To improve mission execution efficiency, multiple specified tasks can be rationally allocated to multiple flight devices based on mission execution objectives and constraints. The quality of task allocation is evaluated based on the mission execution objectives, and the feasibility of the allocation scheme is ensured based on the constraints. Here, mission execution constraints include: the execution paths of different specified tasks are spatially non-intersecting; one specified task is assigned to one flight device, meaning the execution paths of different flight devices will not intersect, preventing interference between flight devices during mission execution; and each task is assigned to one flight device, preventing missed or duplicate assignments. Optionally, mission execution objectives may include: the total flight distance of multiple flight devices, the total time to complete all specified tasks, and the resource consumption of flight devices during flight and task execution; mission execution constraints may also include specific types of flight devices performing specific tasks.

[0081] The aforementioned allocation of multiple designated tasks includes not only the tasks performed by each flight device, but also the order in which each flight device performs its tasks. After the allocation of multiple designated tasks is completed, the system controls each of the multiple flight devices to fly sequentially to the task point corresponding to the designated task assigned to each flight device, and to execute the designated task assigned to each flight device, in the order in which the designated tasks are executed.

[0082] This embodiment assigns tasks to multiple flight devices based on task execution objectives and constraints, ensuring that tasks are allocated and executed reasonably, thus improving task execution efficiency.

[0083] In an exemplary embodiment, in order to facilitate the evaluation of the quality of task allocation based on task execution objectives and achieve high-quality task allocation, the task execution objectives include multiple execution objectives. The execution objectives among the multiple execution objectives correspond one-to-one with the objective functions among the multiple objective functions, and each objective function among the multiple objective functions is used to quantify the corresponding execution objective.

[0084] Optionally, multiple objective functions can be automatically generated based on multiple execution objectives, or corresponding objective functions can be set while specifying multiple execution objectives. Each objective function can correspond to an ideal point, which indicates the theoretical maximum value of the objective function and the direction of optimization. For example, when multiple execution objectives include the total flight distance of multiple flight devices and the adaptation degree between the task and the flight devices, the ideal point for the total flight distance of multiple flight devices can be 0, indicating that the smaller the objective function value of the execution objective, the better; the ideal point for the adaptation degree between the task and the flight devices can be 1, indicating that the larger the objective function value of the execution objective, the better.

[0085] Correspondingly, based on the task execution objectives and constraints, multiple specified tasks are assigned to multiple flight devices, including: constructing a set of sub-problems and generating an initial task allocation scheme corresponding to each sub-problem in the set, wherein a sub-problem in the set corresponds to a weight vector of multiple objective functions, and the weight vectors corresponding to different sub-problems in the set, and the initial task allocation scheme corresponding to each sub-problem is a task allocation scheme for multiple execution tasks; iteratively performing the following update operations on each sub-problem as the current sub-problem until the update termination condition is met, to obtain a candidate task allocation scheme corresponding to each sub-problem: selecting multiple reference sub-problems from the set based on the weight vector corresponding to the current sub-problem; updating the initial task allocation scheme corresponding to the current sub-problem based on the initial task allocation scheme corresponding to each of the multiple reference sub-problems, wherein the candidate task allocation scheme corresponding to each sub-problem is the initial task allocation scheme corresponding to each sub-problem after the update termination condition is met; selecting a target task allocation scheme from the candidate task allocation schemes corresponding to each sub-problem, and assigning multiple specified tasks to multiple flight devices according to the target task allocation scheme.

[0086] In this embodiment, a set of sub-problems is constructed, and an initial task allocation scheme corresponding to each sub-problem in the set is generated. Each sub-problem in the set corresponds to a weight vector of multiple objective functions, and the weight vectors corresponding to different sub-problems in the set, along with the initial task allocation scheme for each sub-problem, constitute a task allocation scheme for multiple execution tasks. Here, a sub-problem in the set corresponds to a weight vector of multiple objective functions, and the weight vectors corresponding to different sub-problems in the set represent different tendencies towards different execution objectives, indicating varying degrees of importance placed on different execution objectives and thus different optimization directions.

[0087] Optionally, generating an initial task allocation scheme corresponding to each subproblem in the set of subproblems can be achieved by randomly generating an initial task allocation scheme for each subproblem. The initial task allocation scheme needs to satisfy task execution constraints and ensure that the initial task allocation scheme is feasible.

[0088] The process iteratively updates each subproblem as the current subproblem until the update termination condition is met, resulting in candidate task allocation schemes for each subproblem. Here, the update operation can be used to iteratively update the initial task allocation scheme for each subproblem. Optionally, after multiple rounds of update operations until the update termination condition is met, the updated initial task allocation scheme for each subproblem is used as a candidate task allocation scheme for that subproblem. The update termination condition can be reaching the maximum number of iterations or population convergence, etc.

[0089] Here, the update operation includes: selecting multiple reference sub-problems from the sub-problem set based on the weight vector corresponding to the current sub-problem; updating the initial task allocation scheme corresponding to the current sub-problem based on the initial task allocation scheme corresponding to each of the multiple reference sub-problems. Optionally, selecting multiple reference sub-problems from the sub-problem set based on the weight vector corresponding to the current sub-problem can involve selecting multiple reference sub-problems with similar weight vectors. The similarity between the weight vectors of the reference sub-problems and the current sub-problem indicates that the optimization directions of the sub-problems are similar. The number of reference sub-problems can be set based on the total number of sub-problems, which is not limited in this embodiment. A new task allocation scheme can be generated based on the reference sub-problems, and when it is determined that the new task allocation scheme is better, the new task allocation scheme is used to replace the initial task allocation scheme corresponding to the current sub-problem, updating the initial task allocation scheme corresponding to the current sub-problem to obtain the updated initial task allocation scheme corresponding to the current sub-problem. Optionally, after completing one round of updates for all sub-problems, the initial task allocation scheme for each sub-problem can continue to be iteratively updated until the update termination condition is met, thus completing the update operation and confirming the candidate task allocation scheme.

[0090] After the update operation is completed, a target task allocation scheme can be selected from the candidate task allocation schemes corresponding to each sub-problem, and multiple specified tasks can be assigned to multiple flight devices according to the target task allocation scheme. Optionally, to improve the quality of the target allocation scheme, dominant solutions can be removed from the candidate task allocation schemes. A dominant solution is a solution that is superior to other solutions on all objectives. Here, the target task allocation scheme can be selected based on the user's actual needs, or the score of each candidate task allocation scheme can be calculated based on a specific algorithm, and the task allocation scheme with the highest score can be automatically selected. This embodiment does not limit this.

[0091] In this embodiment, the task allocation scheme can be determined using a decomposition-based multi-objective evolutionary algorithm. This algorithm can decompose a multi-objective optimization problem into multiple single-objective sub-problems and solve them collaboratively. Its core idea is to transform a multi-objective problem into multiple single-objective sub-problems through aggregation functions and to achieve collaborative optimization among sub-problems by utilizing neighborhood structures.

[0092] This embodiment uses multiple execution objectives, multiple objective functions, and multiple sub-problems with different optimization directions to obtain a target task allocation scheme through iterative updates. This can achieve reasonable task allocation and improve task execution efficiency in complex scenarios of multi-tasking flight equipment.

[0093] In an exemplary embodiment, updating the initial task allocation scheme corresponding to the current subproblem based on the initial task allocation scheme corresponding to each candidate subproblem among multiple reference subproblems includes: selecting two subproblems from the current subproblem and multiple reference subproblems; performing a crossover and mutation operation on the initial task allocation schemes corresponding to the two subproblems as parent task allocation schemes to obtain a child task allocation scheme, wherein the child task allocation scheme satisfies task execution constraints; and updating the initial task allocation scheme corresponding to the target subproblem to the child task allocation scheme if the fitness of the child task allocation scheme with the target subproblem in the current neighborhood is higher than the fitness of the initial task allocation scheme corresponding to the target subproblem with the target subproblem.

[0094] In order to obtain a higher quality candidate task allocation scheme, in this embodiment, the initial task allocation scheme corresponding to each sub-problem can be updated through multiple rounds of iterative updates to obtain the optimal task allocation scheme under each sub-problem, that is, the candidate task allocation scheme corresponding to each sub-problem.

[0095] In this embodiment, two subproblems are selected from the current subproblem and a plurality of reference subproblems. Optionally, the two selected subproblems can be randomly selected, or they can be selected based on the similarity between the weight vectors of the reference subproblems and the current subproblem. The similarity between the weight vectors of the reference subproblems and the current subproblems can be measured by Euclidean distance.

[0096] After identifying two subproblems, the initial task allocation schemes corresponding to the two subproblems are used as parent task allocation schemes to perform crossover and mutation operations to obtain child task allocation schemes, where the child task allocation schemes satisfy task execution constraints. Here, crossover can exchange some task allocation information from the parent schemes; for example, for solutions represented by integer vectors, single-point crossover can be used, selecting a task point and exchanging all task allocation schemes after that task point in the two parent schemes. Mutation can randomly adjust the allocation of some tasks; for example, randomly selecting a task point and assigning it to another flight device that satisfies the task execution constraints. Optionally, the crossover and mutation operations performed on the task allocation schemes can be performed in multiple rounds, and the resulting child task allocation schemes must satisfy the task execution constraints.

[0097] After obtaining the child task allocation scheme, the fitness of the child allocation scheme with all subproblems in the current neighborhood, as well as the fitness of the initial task allocation schemes corresponding to all subproblems in the current neighborhood with the subproblems themselves, can be calculated. Subproblems whose fitness is higher than that of their corresponding initial task allocation schemes are identified as target subproblems, and the initial task allocation schemes corresponding to the target subproblems are updated as child task allocation schemes. Here, the current neighborhood includes the current subproblem and multiple reference subproblems. Fitness is a standard for measuring the performance of a solution (i.e., a task allocation scheme) in a specific problem environment (i.e., a subproblem). Optionally, linear weighting, product methods, penalty function methods, piecewise function methods, etc., can be used to calculate the fitness of the child task allocation scheme with each subproblem.

[0098] Optionally, after obtaining the sub-task allocation scheme, the function value of sub-task allocation scheme can be calculated and substituted into each objective function. If the function value is better than the ideal point of the objective function, the function value corresponding to the sub-task allocation scheme is preferred to the ideal point.

[0099] For example, such as Figure 3 As shown, the process begins by generating subproblems, creating an initial task allocation scheme for each subproblem, calculating the ideal point, constructing a neighborhood for each subproblem, traversing the subproblems, selecting a parent from the neighborhood, performing crossover and mutation to generate offspring, updating the ideal point based on the offspring, traversing all subproblems in the neighborhood, calculating the corresponding task allocation scheme and the fitness of the offspring, replacing the task allocation scheme with the offspring if the offspring has higher fitness, and maintaining the original task allocation scheme if the offspring has lower fitness. The process then checks if all subproblems are completed. If not, it returns to traversing the subproblems; if all are completed, it checks if the iteration termination condition is met. If not, it returns to traversing the subproblems; if the iteration termination condition is met, the candidate task allocation schemes for each subproblem are obtained.

[0100] In this embodiment, by using crossover and mutation operations and fitness calculations to update the initial task allocation scheme corresponding to each sub-problem, multi-objective optimization can be achieved, balancing multiple task execution objectives, making the task allocation scheme of the flight equipment more reasonable, and improving the task execution efficiency of the flight equipment.

[0101] In an exemplary embodiment, based on the state prediction results, planning a flight path for the target flight device from a first mission point to a second mission point includes: based on the state prediction results, using the first mission point as the path start point, the second mission point as the path end point, and a specified time as the starting time, performing the following path point search operation until the latest searched path point is reachable from the path end point. Reachable from the latest path point to the path end point means that the flight path from the latest path point to the path end point does not intersect with the predicted movement path of the target obstacle in space and time. The predicted movement path of the target obstacle is the movement path of the target obstacle determined based on the predicted position of the target obstacle at each future time. Randomly selecting a spatial point within a specified search space to obtain a random point, wherein the specified search space includes the path start point and the path end point; determining neighboring path points from the path map. The spatial points on the line connecting the nearest path points and the random point, where the distance to the nearest path point is a specified distance, are identified as candidate path points. The path map records the reachability relationships between at least one searched path point and the predicted arrival time of each of the at least one path point. Each path point includes a path origin, and the nearest path point is the path point closest to the random point among the at least one path points. Based on the predicted arrival times of the nearest path points, the predicted flight time intervals from the nearest path points to the candidate path points and the corresponding predicted flight paths are planned. The predicted flight paths describe the correspondence between flight time and path points within the predicted flight time intervals. If the predicted flight paths and the predicted movement paths of the target obstacles do not intersect in space and time, the candidate path point is added as a new path point to the path map.

[0102] To achieve precise planning of obstacle avoidance paths and prevent obstacles from affecting the mission execution of flight equipment, a path planning method based on path graphs can be used to plan the equipment's flight path from the first mission point to the second mission point.

[0103] In this embodiment, based on the state prediction results, a path point search operation is performed with the first task point as the path start point, the second task point as the path end point, and a specified time as the starting time, until the latest searched path point is reachable from the path end point. Reachable from the latest path point to the path end point means that the flight path from the latest path point to the path end point does not intersect with the predicted movement path of the target obstacle in space and time. The predicted movement path of the target obstacle is determined based on the predicted position of the target obstacle at each future time. Here, the search operation can be performed cyclically, continuously adding the latest path point. Each time a new path point is added, it is determined whether the latest path point is reachable from the path end point. Reachable from the latest path point to the path end point means that the flight path from the latest path point to the path end point does not intersect with the predicted movement path of the target obstacle in space and time; that is, the predicted flight path of the flying equipment from the latest path point to the path end point and the position of the flying equipment on the flight path at each expected flight time will not conflict with the predicted position of the obstacle at that time.

[0104] The path point search operation includes: randomly selecting a spatial point within a specified search space to obtain a random point, wherein the specified search space includes a path start point and a path end point; determining neighboring path points from the path graph, and identifying spatial points on the line connecting the neighboring path points and the random point that are at a specified distance from the neighboring path point as candidate path points, wherein the path graph is used to record the reachability relationships between at least one searched path point and the predicted arrival time of each of the at least one path point, wherein the at least one path point includes a path start point, and the neighboring path point is the path point among the at least one path points that is closest to the random point; planning the predicted flight period from the neighboring path point to the candidate path point and the predicted flight path corresponding to the predicted flight period based on the predicted arrival time of the neighboring path point, wherein the predicted flight path is used to describe the correspondence between the flight time and the path point within the predicted flight period; and adding the candidate path point as a new path point to the path graph when the predicted flight path and the predicted movement path of the target obstacle do not intersect in time and space.

[0105] A random point is obtained by randomly selecting a spatial point within a specified search space, where the specified search space includes the path start point and the path end point. Optionally, the spatial point can be randomly selected using a uniform distribution or a Gaussian distribution. The specified search space includes the path start point and the path end point, as well as the space within a certain range around the shortest path between the path start point and the path end point.

[0106] Neighboring path points are determined from the path graph. Spatial points on the line connecting these neighboring path points and a random point, located at a specified distance from a neighboring path point, are identified as candidate path points. The path graph records the reachability relationships between at least one searched path point and the predicted arrival time of each of the at least one path point. Each at least one path point includes a path origin, and a neighboring path point is the path point among the at least one path points that is closest to the random point. Initially, the path graph only includes the path origin and can be gradually expanded with the addition of newer path points. Optionally, the nearest neighboring path point can be the path point with the closest Euclidean distance.

[0107] To avoid candidate path points being too close to existing path points, which could lead to insufficient updates to the path map, the distance between candidate path points and their neighboring path points needs to be a specified distance. Specifically, on the line connecting neighboring path points and random points, spatial points at a specified distance from neighboring path points are identified as candidate path points. This specified distance can be a preset distance or can be automatically adjusted based on the size of the search space; this embodiment does not impose any limitations on this.

[0108] Based on the predicted arrival times of neighboring path points, the predicted flight periods from neighboring path points to candidate path points and the corresponding predicted flight paths are planned. The predicted flight paths describe the correspondence between flight times and path points within the predicted flight periods. A path origin can reach any path point in the path map via one or more path points. Since the origin can reach any path point in the path map via one or more path points, the predicted arrival time of each path point can be predicted and recorded in the path map. This is used to plan obstacle avoidance paths in conjunction with the predicted movement paths of target obstacles. Here, based on the predicted arrival times of neighboring path points, the predicted flight periods from neighboring path points to candidate path points and the corresponding predicted flight paths are planned; that is, the position of the flight device at each moment during its flight from neighboring path points to candidate path points is predicted.

[0109] If the predicted flight path and the predicted movement path of the target obstacle do not intersect in space and time, a candidate path point is added as a new path point to the path map. Here, "no intersection in space and time" means that the position of the predictive flight device at each moment during its flight from a nearby path point to a candidate path point does not overlap with the predicted position of the target obstacle at that moment; in other words, the path from a nearby path point to a candidate path point is reachable. This method ensures that every path point in the path map is reachable.

[0110] For example, such as Figure 4As shown, first initialize the path map, path start point, and path end point, then randomly sample random points, determine the neighboring path points in the path map based on the random points, and determine the candidate path points, determine whether the neighboring path points can reach the candidate path points, if not, return to randomly sample random points, if they can reach, add the candidate path points to the path map, and determine whether the newly added path points (here, candidate path points) can reach the path end point, if not, return to randomly sample random points, if they can reach, the flight path can be obtained.

[0111] In this embodiment, by planning the flight path based on the path map and the direct reachability relationship of the waypoints, a flight path that can effectively avoid obstacles can be obtained, thereby improving the mission execution efficiency of the flight equipment.

[0112] The flight control method of the flight device in this application embodiment will be explained below with reference to an optional example. In this optional example, the flight device is a drone, and the flight control method of the flight device can be completed through the cooperation of the ground system and the drone system.

[0113] Figure 5 This is a flowchart illustrating the flight control method of the flight equipment in this optional example, such as... Figure 5 As shown, the flight control method of this flight equipment can include the following steps: the ground system establishes a grid map, marks task points and takeoff points, and determines the task assignments for all UAVs. The UAVs use sensors to perceive and calculate, obtain obstacle trajectories based on sensor data, plan obstacle avoidance paths, and proceed to the task points to perform tasks based on the obstacle avoidance paths. If not all task points are completed, the UAVs repeat the above operations. If all task points are completed, the task ends.

[0114] After this, it can be determined whether all drones have completed their respective task checks. If not, the process is repeated for subsequent drones. Figure 5 The steps of the unmanned aerial vehicle (UAV) system are followed until all UAVs have completed their missions.

[0115] Optionally, the ground system and the UAV system can communicate through a lightweight encryption and encapsulation scheme. The data encryption method can be Secure Hash Algorithm 2 (SHA-2) or Elliptic Curve Cryptography (ECC), etc. The communication content includes mission location, mission path, UAV number, etc.

[0116] Optionally, when the UAV collects data through sensors, it can perform sequential processing and compression (such as differential encoding or run-length encoding) on ​​the sensor data, and process missing data using methods such as Newton interpolation or quadratic interpolation. After detecting an obstacle, the sensor can achieve dynamic obstacle avoidance by predicting the obstacle trajectory and planning a path, solving the problem of insufficient real-time performance in traditional dynamic obstacle avoidance and improving the safety and effectiveness of obstacle avoidance. The obstacle trajectory prediction method can use set-valued filtering algorithms. Set-valued filtering can effectively process unknown but bounded noise. Due to the use of set-valued filtering algorithms, the UAV does not need to store sensor data, reducing memory usage.

[0117] Through this optional example, the ground system and the UAV system work together to transmit the calculated task path from the ground system to the UAV. The UAV then autonomously completes the task using its built-in sensors and microcomputing module. Through reasonable task allocation and planning, systematic management of the UAV, cost control of the UAV, and battery management of the UAV can be achieved.

[0118] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0119] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0120] According to another aspect of the embodiments of this application, a flight control device for a flight device is also provided. This flight control device can be used to implement the flight control method for the flight device provided in the above embodiments, and will not be repeated hereafter. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0121] Figure 6 This is a structural block diagram of a flight control device for an optional flight equipment according to an embodiment of this application, such as... Figure 6 As shown, the flight control device of the flight equipment includes:

[0122] The acquisition unit 602 is used to acquire target sensor data collected by the sensors on the target flight device when the target flight device is at the first task point among multiple task points. The multiple flight devices are flight devices that cooperate to execute multiple specified tasks. One of the specified tasks corresponds to one of the task points. The target sensor data includes observation data of target obstacles observed by the sensors on the target flight device.

[0123] The prediction unit 604 is used to predict the state of the target obstacle based on the observation data of the target obstacle when the target flight equipment is about to fly to the second task point among multiple task points, and obtain the state prediction result. The state prediction result is used to indicate the predicted position of the target obstacle at each future time in a set of future time moments.

[0124] Planning unit 606 is used to plan the flight path of the target flight equipment from the first mission point to the second mission point based on the state prediction results, wherein the flight position corresponding to each future time on the flight path does not intersect with the predicted position of the target obstacle at the same future time.

[0125] The first control unit 608 is used to control the target flight equipment to fly from the first mission point to the second mission point according to the equipment flight path.

[0126] It should be noted that the acquisition unit 602 in this embodiment can be used to execute the above step S202, the prediction unit 604 in this embodiment can be used to execute the above step S204, the planning unit 606 in this embodiment can be used to execute the above step S206, and the first control unit 608 in this embodiment can be used to execute the above step S208.

[0127] According to the embodiments provided in this application, when a target flight device is located at a first task point among multiple task points, target sensor data collected by sensors on the target flight device is acquired. The multiple flight devices are flight devices that collaboratively execute multiple designated tasks, and one of the designated tasks corresponds to one of the task points. The target sensor data includes observation data of target obstacles observed by sensors on the target flight device. When the target flight device is about to fly to a second task point among the multiple task points, the state of the target obstacle is predicted based on the observation data of the target obstacle, resulting in a state prediction result. The state prediction result indicates the predicted position of the target obstacle at each future moment in a set of future moments. Based on the state prediction result, a flight path is planned for the target flight device from the first task point to the second task point, wherein the flight position corresponding to each future moment on the flight path does not intersect with the predicted position of the target obstacle at the same future moment. The target flight device is controlled to fly from the first task point to the second task point according to the flight path. Since the obstacle avoidance path is planned based on the prediction results before the flight equipment flies to the next mission point, there is no need to frequently replan the path or detour during the flight. This solves the problem of low mission execution efficiency in the flight control methods of related technologies and improves the mission execution efficiency of the flight equipment.

[0128] In an exemplary embodiment, the state of the target obstacle includes the position and velocity of the target obstacle, and the observation data of the target obstacle includes observation values ​​corresponding to each historical moment in a set of historical moments; the prediction unit includes: a determination module, configured to determine the state evaluation value of the target obstacle, which is evaluated based on the observation value corresponding to the earliest historical moment in the set of historical moments, as the initial state evaluation value of the target obstacle; and a first execution module, configured to sequentially use the historical moments other than the earliest historical moment in the set of historical moments as specified historical moments to perform the following prediction operation to obtain the current state evaluation value of the target obstacle: based on a specified process model and the state evaluation value of the target obstacle at the previous historical moment of the specified historical moment, perform prediction on the target obstacle at the specified historical moment. The system predicts the state of the target obstacle at a specified historical moment, obtaining the predicted state value of the target obstacle at that specified historical moment. It then filters and updates the predicted state value of the target obstacle at that specified historical moment using the observations corresponding to those moments, obtaining the state evaluation value of the target obstacle at that specified historical moment. The current state evaluation value of the target obstacle is the state evaluation value of the target obstacle at the latest historical moment in a set of historical moments. A specified process model is used to describe the change of the target obstacle's state over time. A prediction module is used to predict the state of the target obstacle at each future moment based on the specified process model and the current state evaluation value of the target obstacle, obtaining a state prediction result. The state prediction result includes the predicted state value of the target obstacle at each future moment.

[0129] In an exemplary embodiment, both the process noise corresponding to the specified process model and the measurement noise corresponding to the measurement model of the sensor on the target flight equipment are bounded noise. The prediction module includes: a prediction submodule, used to predict the state of the target obstacle at a specified historical time and the error covariance corresponding to the specified historical time based on the specified process model and the state evaluation value of the target obstacle at the previous historical time, to obtain the predicted state value of the target obstacle at the specified historical time and the prediction error covariance corresponding to the specified historical time; the execution module includes: a calculation submodule, used to calculate the filter gain matrix based on the predicted state value of the target obstacle at the specified historical time, the prediction error covariance corresponding to the specified historical time, and the observed value corresponding to the specified historical time; and an adjustment submodule, used to adjust the predicted state value of the target obstacle at the specified historical time based on the filter gain matrix and the measurement residual, to obtain the state evaluation value of the target obstacle at the specified historical time, wherein the measurement residual is the difference between the following two: the observed value corresponding to the specified historical time, and the product of the measurement model and the measurement matrix corresponding to the specified historical time with the predicted state value of the target obstacle at the specified historical time.

[0130] In one exemplary embodiment, the apparatus further includes: an allocation unit, configured to allocate multiple specified tasks to multiple flight devices based on a task execution objective and task execution constraints, wherein the task execution objective is used to evaluate the task allocation quality of the multiple specified tasks, the task execution constraints are used to limit the constraint conditions satisfied by the allocation of the multiple specified tasks, and the task execution constraints include: the execution routes of different specified tasks among the multiple specified tasks do not intersect spatially, and one specified task among the multiple specified tasks is allocated to one flight device among the multiple flight devices; and a second control unit, configured to control each of the multiple flight devices to sequentially fly to the task point corresponding to the specified task allocated to each flight device according to the execution order of the specified tasks allocated to each flight device, and execute the specified task allocated to each flight device.

[0131] In an exemplary embodiment, the task execution objective includes multiple execution objectives, each of which corresponds one-to-one with a target function among multiple target functions, and each target function is used to quantify the corresponding execution objective. The allocation unit includes: a second execution module, configured to construct a set of sub-problems and generate an initial task allocation scheme corresponding to each sub-problem in the set of sub-problems, wherein a sub-problem in the set of sub-problems corresponds to a weight vector of multiple target functions, and the weight vectors corresponding to different sub-problems in the set of sub-problems, and the initial task allocation scheme corresponding to each sub-problem, are all task allocation schemes for multiple execution tasks; and a third execution module, configured to iteratively use each sub-problem as the current sub-problem. The problem performs the following update operations until the update termination condition is met, obtaining the candidate task allocation scheme for each sub-problem: Based on the weight vector corresponding to the current sub-problem, select multiple reference sub-problems from the sub-problem set; based on the initial task allocation scheme corresponding to each of the multiple reference sub-problems, update the initial task allocation scheme corresponding to the current sub-problem, wherein the candidate task allocation scheme corresponding to each sub-problem is the initial task allocation scheme corresponding to each sub-problem after the update termination condition is met; the fourth execution module is used to select the target task allocation scheme from the candidate task allocation schemes corresponding to each sub-problem, and allocate multiple specified tasks to multiple flight devices according to the target task allocation scheme.

[0132] In one exemplary embodiment, the third execution module includes: a selection submodule, configured to select two subproblems from the current subproblem and a plurality of reference subproblems; an execution submodule, configured to perform a crossover and mutation operation on the initial task allocation schemes corresponding to the two subproblems as parent task allocation schemes to obtain a child task allocation scheme, wherein the child task allocation scheme satisfies task execution constraints; and an update submodule, configured to update the initial task allocation scheme corresponding to the target subproblem to the child task allocation scheme if the fitness of the child task allocation scheme with the target subproblem in the current neighborhood is higher than the fitness of the initial task allocation scheme corresponding to the target subproblem with the target subproblem.

[0133] In an exemplary embodiment, the planning unit includes: a fifth execution module, configured to perform the following path point search operation based on the state prediction result, using a first task point as the path start point, a second task point as the path end point, and a specified time as the start time, until the latest searched path point is accessible to the path end point. Accessibility to the latest path point means that the flight path from the latest path point to the path end point does not intersect with the predicted movement path of the target obstacle in space and time. The predicted movement path of the target obstacle is determined based on the predicted position of the target obstacle at each future time. The module further includes: randomly selecting a spatial point within a specified search space to obtain a random point, where the specified search space includes the path start point and the path end point; determining neighboring path points from the path map and connecting the neighboring path points with the random point. Online, spatial points at a specified distance from neighboring path points are identified as candidate path points. The path map records the reachability relationships between at least one searched path point and the predicted arrival time of each of the at least one path point. The at least one path point includes a path origin, and the neighboring path point is the path point closest to the random point among the at least one path points. Based on the predicted arrival time of the neighboring path points, the predicted flight period from the neighboring path points to the candidate path points and the predicted flight path corresponding to the predicted flight period are planned. The predicted flight path describes the correspondence between flight time and path points within the predicted flight period. If the predicted flight path and the predicted movement path of the target obstacle do not intersect in space and time, the candidate path point is added as a new path point to the path map.

[0134] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0135] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein the program executes the steps in any of the above method embodiments when it is run.

[0136] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0137] According to another aspect of the embodiments of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor is configured to perform the steps of any of the method embodiments described above via the computer program. In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0138] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0139] According to another aspect of the embodiments of this application, a computer program product is also provided, comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit 701, it performs various functions provided in the embodiments of this application. The sequence numbers of the embodiments of this application above are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0140] Figure 7 A schematic block diagram of a computer system architecture for implementing embodiments of the present application is shown. Figure 7 As shown, the computer system 700 includes a Central Processing Unit (CPU) 701, which performs various appropriate actions and processes based on programs stored in ROM 702 or loaded into RAM 703 from storage section 708. Random access memory 703 also stores various programs and data required for system operation. The CPU 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0141] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card, such as a local area network card or modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0142] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit 701, it performs various functions defined in the system of this application.

[0143] It should be noted that, Figure 7 The computer system 700 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0144] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0145] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A flight control method of a flying device, characterized by, The method comprises: obtaining target sensor data collected by a sensor on a target flight device, in a case where the target flight device among a plurality of flight devices is at a first task point among a plurality of task points, wherein the plurality of flight devices are flight devices that cooperatively perform a plurality of specified tasks, one of the plurality of specified tasks corresponds to one of the plurality of task points, and the target sensor data comprises observation data of a target obstacle observed by the sensor on the target flight device; performing state prediction on the target obstacle according to the observation data of the target obstacle, to obtain a state prediction result, in a case where the target flight device is to fly to a second task point among the plurality of task points, wherein the state prediction result is used to indicate a predicted position of the target obstacle at each future time in a set of future times; planning a device flight path from the first task point to the second task point for the target flight device based on the state prediction result, wherein a flight position corresponding to each future time on the device flight path does not intersect with the predicted position of the target obstacle at the same future time; controlling the target flight device to fly from the first task point to the second task point according to the device flight path.

2. The method of claim 1, wherein, The state of the target obstacle comprises a position of the target obstacle and a velocity of the target obstacle, and the observation data of the target obstacle comprises an observation value corresponding to each historical time in a set of historical times; The state prediction on the target obstacle according to the observation data of the target obstacle comprises: determining a state evaluation value of the target obstacle evaluated based on the observation value corresponding to the earliest historical time in the set of historical times as an initial state evaluation value of the target obstacle; performing the following prediction operations on the historical times other than the earliest historical time in the set of historical times as specified historical times in turn to obtain a current state evaluation value of the target obstacle: predicting a state of the target obstacle at the specified historical time based on a specified process model and a state evaluation value of the target obstacle at a previous historical time of the specified historical time to obtain a state prediction value of the target obstacle at the specified historical time; and filtering and updating the state prediction value of the target obstacle at the specified historical time using the observation value corresponding to the specified historical time to obtain the state evaluation value of the target obstacle at the specified historical time, wherein the current state evaluation value of the target obstacle is a state evaluation value of the target obstacle at the latest historical time in the set of historical times, and the specified process model is used to describe the change of the state of the target obstacle over time; predicting the state of the target obstacle at each future time in turn based on the specified process model and the current state evaluation value of the target obstacle to obtain the state prediction result, wherein the state prediction result comprises the state prediction value of the target obstacle at each future time.

3. The method of claim 2, wherein, The process noise corresponding to the specified process model and the measurement noise corresponding to the measurement model of the sensor on the target flight device are bounded noises; The predicting, based on the specified process model and the state estimation value of the target obstacle at the previous historical moment, of the state of the target obstacle at the specified historical moment to obtain the state prediction value of the target obstacle at the specified historical moment comprises: predicting, based on the specified process model and the state estimation value of the target obstacle at the previous historical moment, the state of the target obstacle at the specified historical moment and the error covariance corresponding to the specified historical moment to obtain the state prediction value of the target obstacle at the specified historical moment and the prediction error covariance corresponding to the specified historical moment; The filtering and updating of the state prediction value of the target obstacle at the specified historical moment by using the observation value corresponding to the specified historical moment to obtain the state estimation value of the target obstacle at the specified historical moment comprises: calculating a filtering gain matrix according to the state prediction value of the target obstacle at the specified historical moment, the prediction error covariance corresponding to the specified historical moment and the observation value corresponding to the specified historical moment; and adjusting the state prediction value of the target obstacle at the specified historical moment based on the filtering gain matrix and a measurement residual to obtain the state estimation value of the target obstacle at the specified historical moment, wherein the measurement residual is the difference between the observation value corresponding to the specified historical moment and the product of the measurement model, the measurement matrix corresponding to the specified historical moment and the state prediction value of the target obstacle at the specified historical moment.

4. The method of claim 1, wherein, The method further comprises: allocating the plurality of specified tasks to the plurality of flight devices based on a task execution target and a task execution constraint, wherein the task execution target is used to evaluate the task allocation quality of the plurality of specified tasks, and the task execution constraint is used to define a constraint condition to be satisfied when the plurality of specified tasks are allocated, and the task execution constraint comprises: the execution routes of different specified tasks in the plurality of specified tasks are not intersected in space, and one specified task in the plurality of specified tasks is allocated to one flight device in the plurality of flight devices; controlling each flight device in the plurality of flight devices to fly to a task point corresponding to a specified task allocated to the each flight device in an execution order of the specified task allocated to the each flight device and execute the specified task allocated to the each flight device.

5. The method of claim 4, wherein, The task execution target comprises a plurality of execution targets, and each execution target in the plurality of execution targets corresponds to a target function in a plurality of target functions, and each target function in the plurality of target functions is used to quantify a corresponding execution target; The allocating, based on a task execution target and a task execution constraint, of the plurality of specified tasks to the plurality of flight devices comprises: The task execution target comprises a plurality of execution targets, and each execution target in the plurality of execution targets corresponds to a target function in a plurality of target functions, and each target function in the plurality of target functions is used to quantify a corresponding execution target; construct a sub-problem set, and generate an initial task allocation scheme corresponding to each sub-problem in the sub-problem set, wherein one sub-problem in the sub-problem set corresponds to one weight vector of the plurality of objective functions, different sub-problems in the sub-problem set correspond to different weight vectors, and the initial task allocation scheme corresponding to each sub-problem is a task allocation scheme of the plurality of execution tasks; perform the following update operation on each sub-problem as a current sub-problem until an update termination condition is met, to obtain a candidate task allocation scheme corresponding to each sub-problem: selecting a plurality of reference sub-problems from the sub-problem set based on the weight vector corresponding to the current sub-problem; and updating the initial task allocation scheme corresponding to the current sub-problem based on the initial task allocation scheme corresponding to each reference sub-problem in the plurality of reference sub-problems, wherein the candidate task allocation scheme corresponding to each sub-problem is the initial task allocation scheme corresponding to each sub-problem after the update termination condition is met; selecting a target task allocation scheme from the candidate task allocation scheme corresponding to each sub-problem, and allocating the plurality of specified tasks to the plurality of flying devices according to the target task allocation scheme.

6. The method of claim 5, wherein, The updating of the initial task allocation scheme corresponding to the current sub-problem based on the initial task allocation scheme corresponding to each candidate sub-problem in the plurality of reference sub-problems comprises: selecting two sub-problems from the current sub-problem and the plurality of reference sub-problems; performing a cross-mutation operation on the initial task allocation schemes corresponding to the two sub-problems as parent task allocation schemes to obtain a child task allocation scheme, wherein the child task allocation scheme satisfies the task execution constraint; in a case where the fitness of the target sub-problem and the child task allocation scheme is higher than the fitness of the initial task allocation scheme corresponding to the target sub-problem and the target sub-problem, updating the initial task allocation scheme corresponding to the target sub-problem to the child task allocation scheme.

7. The method according to any one of claims 1 to 6, characterized in that, The planning of a device flight path from the first task point to the second task point for the target flying device based on the state prediction result comprises: based on the state prediction result, performing the following path point search operation with the first task point as a path starting point, the second task point as a path ending point, and a specified time as a starting time, until a latest path point searched is reachable to the path ending point, the latest path point being reachable to the path ending point means that a flight path from the latest path point to the path ending point has no intersection with a predicted moving path of the target obstacle in space-time, and the predicted moving path of the target obstacle is determined based on a predicted position of the target obstacle at each future time: randomly selecting a space point in a specified search space to obtain a random point, wherein the specified search space includes the path starting point and the path ending point; Determine a neighboring path point from a path atlas, and determine a space point on a line connecting the neighboring path point and the random point, and having a distance to the neighboring path point being a specified distance, as a candidate path point, wherein the path atlas is used to record a reachable relationship between at least one path point searched and a predicted arrival time of each path point in the at least one path point, the at least one path point including the path starting point, and the neighboring path point is a path point in the at least one path point closest to the random point; Plan a predicted flight time period of the neighboring path point to the candidate path point and a predicted flight path corresponding to the predicted flight time period based on a predicted arrival time of the neighboring path point, wherein the predicted flight path is used to describe a corresponding relationship between flight time and path points within the predicted flight time period; In a case where the predicted flight path and a predicted movement path of the target obstacle do not have a space-time intersection, add the candidate path point as a new path point to the path atlas.

8. A flight control device of a flying apparatus, characterized by comprising: Comprise: An acquisition unit is configured to acquire target sensor data collected by a sensor on a target flight device in a case where the target flight device is at a first task point in a plurality of task points, wherein the plurality of flight devices are flight devices that cooperatively perform a plurality of specified tasks, one of the plurality of specified tasks corresponds to one of the plurality of task points, and the target sensor data includes observation data of a target obstacle observed by the sensor on the target flight device; A prediction unit is configured to perform state prediction on the target obstacle according to the observation data of the target obstacle to obtain a state prediction result in a case where the target flight device is to fly to a second task point in the plurality of task points, wherein the state prediction result is used to indicate a predicted position of the target obstacle at each future time in a group of future times; A planning unit is configured to plan a device flight path from the first task point to the second task point for the target flight device based on the state prediction result, wherein a flight position corresponding to each future time on the device flight path does not intersect with a predicted position of the target obstacle at the same future time; A first control unit is configured to control the target flight device to fly from the first task point to the second task point according to the device flight path.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, wherein the computer program is executed by a processor to implement the steps of the method in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 7.

Images