A method suitable for multi-target observation and assignment of aircraft clusters

Abstract

The application provides a method suitable for multi-target observation and distribution of aircraft cluster, introduces cluster state estimation based on Monte Carlo positioning and extended Hungarian algorithm as key technologies supporting cooperative interception / collision. It covers the technical fields of multi-view processing, target estimation, Monte Carlo positioning, task distribution, autonomous interception, autonomous collision, etc., aiming to improve the success rate of autonomous aircraft interception / collision tasks. The application verifies the effectiveness of the algorithm under airborne perception and control, and consolidates its feasibility in practical application. These contributions jointly promote the development of multi-target interception / collision technology, and provide important tools and methods for effectively dealing with multi-target invasion threats.

Description

Technical Field

[0001] This invention relates to key technologies for autonomous interception / collision of aircraft, including multi-view information integration, target estimation, Monte Carlo localization, task allocation, autonomous interception, and autonomous collision. In particular, it introduces swarm state estimation based on Monte Carlo localization and an extended Hungarian assignment method to solve the position estimation and task allocation problems in aircraft interception / collision missions. This covers technical fields such as multi-view processing, target estimation, Monte Carlo localization, task allocation, autonomous interception, and autonomous collision, aiming to improve the success rate of autonomous aircraft interception / collision missions. Background Technology

[0002] In recent years, drones, commonly known as aircraft, have rapidly developed in various fields such as surveillance, reconnaissance, delivery, and entertainment. However, this expansion has also highlighted the urgent need for effective countermeasures to address the potential security threats posed by unauthorized or malicious aircraft operations. In response to this growing concern, the concept of aircraft interception / collision has become crucial in the field of aerospace defense strategy.

[0003] The fundamental concept of cooperative aircraft swarm interception / collision is that it improves the overall success rate of interception / collision operations while extending single-platform interception / collision to aircraft swarm scenarios. In cooperative interception / collision, multiple platforms work together, which not only increases the chance of successfully neutralizing target aircraft but also helps manage complex engagements in scenarios where adversarial drone swarms pose significant challenges.

[0004] This invention employs a gyroscope and a monocular camera as the sensing sensors for the aircraft. The interception / collision performance of strapdown monocular camera fusion has been validated in our previous work. Strapdown monocular camera fusion contributes to the compactness and miniaturization of interception / collision aircraft, improving their flexibility. However, monocular camera perception lacks depth information about the target, and commonly used methods can be categorized as geometric methods and deep learning depth estimation methods. These methods require prior knowledge of the target's actual size, or the target must be present in the training data, making them inaccurate when estimating targets not included in the data.

[0005] In swarm adversarial scenarios, a potential challenge is the mismatch between the number of aircraft and the number of targets. Directly using integer programming to solve the optimization problem is computationally expensive and struggles to guarantee real-time response. Meanwhile, consensus-based auction algorithms offer an alternative, requiring multiple communications between aircraft swarms. However, it's important to note that this approach places higher demands on bandwidth and the topology of the aircraft swarm.

[0006] Addressing the multifaceted challenges associated with aircraft interception / collision requires a comprehensive understanding of the intricate dynamics between the interceptor / collision aircraft and the target aircraft. Two key aspects stand out as significant challenges in this endeavor: (1) accurately estimating the target's three-dimensional position from a monocular camera; and (2) real-time allocation of multiple aircraft to interact with multiple targets.

[0007] Considering these challenges, successfully executing aircraft interception / collision missions requires a focus on improving the accuracy of target 3D position estimation and effectively allocating multiple interception / collision aircraft to deal with multiple targets, especially in swarm-based adversarial scenarios. This involves in-depth research in multiple fields, including computer vision, machine learning, optimization algorithms, and cooperative control, to ensure the efficient and successful execution of interception / collision missions. Summary of the Invention

[0008] This invention proposes a cooperative interception / collision scheme to counter multiple intruding aerial targets, extending previous single-target interception / collision methods to the multi-target domain, thereby improving the overall success rate. This invention introduces Monte Carlo localization-based swarm state estimation and an extended Hungarian algorithm as key technologies supporting cooperative interception / collision. This invention verifies the effectiveness of the algorithm under airborne perception and control, consolidating its feasibility in practical applications. These contributions collectively promote the development of multi-target interception / collision technology, providing important tools and methods for effectively addressing multi-target intrusion threats.

[0009] First, this invention introduces the aircraft motion model, target motion model, and camera imaging model relevant to this invention. Using these models, this invention models the intercept / collision aircraft and the target. Furthermore, this invention proposes a swarm state model, a measurement model (including an IMU noise measurement model and an image delay measurement model), and a target allocation model, and studies the problems of swarm state estimation and task allocation.

[0010] Coordinate systems: For the i-th aircraft in the intercept / collision aircraft swarm, this invention uses four coordinate systems, such as... Figure 1 As shown, it includes:

[0011] ·{e}={o e -x e y e z e} is the Earth-fixed coordinate system (EFCS), used to represent the position and velocity of aircraft and targets in a global inertial system;

[0012] ·{b i}={o bi -x bi y bi z bi} is the body coordinate system (BCS) of the i-th aircraft, used to represent the variables of the aircraft's current attitude;

[0013] ·{c i}={o ci -x ci y ci z ci} is the camera coordinate system (CCS) of the i-th aircraft;

[0014] ·{s i} = S 2 is the image coordinate system (ICS) of the i-th aircraft, used to represent the image position of target features in the image plane;

[0015] In SCS, all target features reside on a 2-sphere. The definition of a 2-sphere is as follows:

[0016]

[0017] matrix This represents the rotation matrix from coordinate system F1 to F2. The 3-D eigenorthogonal group is defined as follows:

[0018]

[0019] Aircraft Motion Model: The attitude of the intercepting / collision aircraft is represented by a rotation matrix, and the aircraft motion is represented using a flight control rigid body model, as shown below:

[0020]

[0021] in Indicates the aircraft's position under the EFCS; The vector represents the velocity under EFCS; m is the mass of the aircraft; g is the local gravitational acceleration, g = [0 0 g] T ; It refers to the controllable forces of the aircraft in the EFCS; It is the angular velocity under BCS; matrix [ b ω] × It is an antisymmetric matrix, representing angular velocity. b The cross product mapping of ω.

[0022] Cross product mapping [·] × : The following definition is used, such that [x]×y=x×y, and For any The inverse of the cross product mapping is represented by the VEX mapping vex(·): The set of 3×3 antisymmetric matrices for Cross product mapping and VEX mapping can be summarized as follows:

[0023]

[0024] For multi-rotor interceptor / collision aircraft, the controllable force is generated by the thrust of the propellers, as shown below:

[0025]

[0026] Where 0≤f≤f m ,f m It is the maximum thrust; R f ∈SO(3) is the conversion of the thrust vector into o in BCS. b z b The rotation matrix of the axis;

[0027] For aircraft, R f It is a constant because the propeller provides thrust relative to the fuselage in a fixed direction, whereas for tiltrotor aircraft, it is variable. Vector e3 is the third column of the 3×3 identity matrix I, where I = [e1 e2 e3], e3 = [0 0 1]. T .

[0028] Target motion model: The motion of the target is represented using a point mass model, as shown below:

[0029]

[0030] in and These represent the target's position, velocity, and thrust acceleration under EFCS, respectively; noise. It is a Gaussian random variable with zero mean.

[0031] Camera imaging model: such as Figure 1 As shown, perspective projection performs a mapping from 3D space to an image sphere: Assume the origin of CCS coincides with the origin of BCS. That is, their translation vectors... And rotation matrix It is a constant. The camera used in this invention is integrated with the aircraft body; when the camera makes contact with the target, it is considered a successful interception / collision. Therefore, there is a constant rotation between the optical axis and the direction of the aircraft's head. set up c p is the visible target point in the camera coordinate system, represented as a vector in the camera coordinate system. The image points observed by the camera are represented as s p, by rescaling to the camera's image surface S 2 This is obtained from the above. We consider a camera with a spherical image plane, therefore,

[0032]

[0033] Where r = || c p|| represents the relative depth of the target. Here, using spherical coordinates to represent the motion of target imaging feature points solves the problem that traditional planar imaging models cannot represent feature points when the target is behind the camera.

[0034] The aircraft motion model and camera imaging model describe the motion and perception of a single interceptor / collision aircraft, while the target motion model describes the motion of a single target. In the following, this invention describes the cooperative interception / collision mission using a cluster state model that describes the relationship between the aircraft cluster and the target cluster.

[0035] Intercept / collision clusters and hostile clusters: Let and These represent the set of aircraft in the intercept / collision group and the set of intruders in the hostile group, respectively, where IS i ,i=1,…,N represents the i-th intercept / collision aircraft, HS j Let j = 1, ..., M represent the j-th intruder. The sizes of the interceptor / collision aircraft cluster and the hostile cluster are N and M, respectively. Considering that N ≠ M, that is, when the size of the interceptor / collision aircraft cluster does not match the size of the hostile cluster, it can be called a multi-task allocation problem, which can be divided into the following two cases: When N < M, our interceptor / collision aircraft are equipped with reusable tools, such as net guns. One interceptor / collision aircraft can be assigned to multiple intruders. When N > M, multiple interceptor / collision aircraft are assigned to the same intruder to improve the interception / collision success rate and make full use of resources.

[0036] Cluster State Model: Each interceptor / collision vehicle maintains the state of the vehicle cluster and the target cluster, estimating global information for subsequent task allocation. Vehicle The state is represented as:

[0037]

[0038] Including the rotation matrix of the aircraft Position in EFCS e p and velocity vector e v. Gyroscope bias and accelerometer bias The proposed state representation x i Located within manifold M, According to the aircraft motion model (3), the state equation of the intercepting / collision aircraft i can be expressed as:

[0039]

[0040] Among them, motion commands This is a noisy IMU measurement. The state transition function f(·) maps the previous state and motion command to a state from time t. k-1 By time t k The predicted state, Similarly, the target The state is defined as:

[0041]

[0042] According to the target motion model (6), the state equation of target j can be expressed as:

[0043]

[0044] The cluster state is represented as:

[0045]

[0046] IMU Noise Measurement Model: The IMU (Inertial Measurement Unit) provides measurements of gyroscope angle increments and accelerometer velocity increments under BCS. Combining these measurements with the aircraft motion model (3) yields the state. and e The recursive expression for v. • Rotation matrix The derivative, i.e., angular velocity b ω can be obtained from gyroscope measurements, as shown below:

[0047] b ω=ω gyr -b gyr -n gyr (13)

[0048] Among them, the gyroscope measurement noise n gyr It is a Gaussian random variable with zero mean; bias b gyr Following the Wiener process, as follows:

[0049]

[0050] ·speed e The derivative of v, i.e., acceleration e 'a' can be obtained from accelerometer measurements, as shown below:

[0051]

[0052] Among them, the accelerometer measurement noise n acc It is a Gaussian random variable with zero mean; bias b acc Following the Wiener process, as follows:

[0053]

[0054] Image delay measurement model: Image processing provides the image coordinates of the target point, resulting in image measurement. s p m Since camera imaging, image processing, and network transmission all require time, the obtained measurements are data from a certain period of time ago. Assuming the measurement delay is D cycles based on the IMU update frequency, and the delay time is t... D >0 is known. Therefore, the image measurement at time t can be expressed as:

[0055] s p m,k = s p k-D +n img (17)

[0056] Wherein, image measurement noise n img It is a Gaussian random variable with zero mean. Note the delay time t of this aircraft. D This includes camera imaging and image processing time, which can be obtained from timers in camera parameters and image processing algorithms. Other aircraft latency also includes network transmission time, which can be calculated from message timestamps.

[0057] Target assignment model: This problem involves N intercept / collision aircraft, denoted as... Where A = {0, 1} M Let s represent a finite set of pure policies. i ∈A represents IS i The pure strategy is defined by S = [s1, ..., s...]. M ] T This represents the pure policy configuration for intercepting / colliding aircraft. The goal of the multi-objective allocation problem is to minimize cost. In this objective allocation problem, the cost matrix C = (c ij ) N×M Determined by the distance between the aircraft and the target, c ij =|| e p i - e p tj ||, i=1,…,N, j=1,...,M. Therefore, the optimal objective allocation problem can be expressed as:

[0058]

[0059] This invention proposes a method for solving cooperative interception / collision by using Monte Carlo localization-based cluster state estimation and an extended Hungarian algorithm. The planning flowchart is shown below. Figure 2As shown, based on the previous definition, the specific implementation steps are as follows:

[0060] Step 1: Cluster State Estimation Based on Monte Carlo Localization

[0061] At a given time t k Target location detected on the RGB image s p m,k and time t k-1 By time t k Motion estimation between u k The multi-view estimator (MTE) estimates the intercept / collision vehicle at time t. k state χ k Specifically, MTE uses a particle filter to estimate the posterior probability. Where z 0:k and u 0:k These represent the time intervals from the initial time to the current time t. k A collection of images and motion measurements.

[0062] S11, Cluster Status Representation

[0063] Let k represent time t. k The index of the IMU measurement. According to the cluster state model, the continuous state equation (9) of aircraft i can be discretized within the IMU sampling period Δt as:

[0064]

[0065] The function F is defined as follows:

[0066]

[0067] Similarly, the continuous state equation (11) for target j can be discretized as follows:

[0068] x tj,k =x tj,k-1 +F t (x tj,k-1 )Δt (21)

[0069] Wherein, function F t The definition is as follows:

[0070]

[0071] posterior distribution Modeled as a weighted set of L particles:

[0072]

[0073] in, This is the state of the l-th particle. These are the corresponding weights. Then, the particle filter updates the particle set at each time point (when new images and measurements are received) and applies four steps: prediction, update, resampling, and forward propagation.

[0074] S12, Aircraft Motion Prediction

[0075] Input sampling IMU information Then, the state is updated based on the IMU noise measurement model as follows:

[0076]

[0077]

[0078]

[0079]

[0080] Among them, the function sample(σ 2 A point is randomly sampled from a Gaussian distribution with zero mean and σ variance. In this step...

[0081]

[0082] S13, Aircraft Measurement Update

[0083]

[0084] Among them, the function prob(a,σ) 2 Calculate its parameter a in a region with variance σ. 2 The probability under a zero-center Gaussian distribution, a measurement quantity s p m It is calculated from the camera imaging model and the image delay measurement model. Equation (29) compares the collected target features. s p m Target characteristics predicted by aircraft motion s p k-D And assign lower weights to particles that do not match.

[0085] S14, Monte Carlo positioning

[0086] After the update step, we use a method with replacement from the collection. Resample L particles, where each particle has a probability The sampled set. Let the new set be...

[0087] S141. For each particle, make the following update:

[0088]

[0089]

[0090]

[0091] S142. For each particle, with probability from Select i in the selection, and... Join middle.

[0092] S15, Forward Propagation

[0093] For delayed image measurements, at time t k-D pass

[0094]

[0095] Obtained probability distribution The probability distribution is updated to the current time t by repeatedly applying prediction steps involving interception / collision of the aircraft's motion. k :

[0096]

[0097] Where dσ is M D The differential area, including the coordinate χ k-1 ,χ k-2 ,...,χ k-D .

[0098] Step 2: Task allocation based on the distributed extended Hungarian algorithm

[0099] S21. Construct the task list and worker list.

[0100] HS and IS are redefined as a Task List and a Worker List, respectively. Based on the target allocation model, important targets are subdivided into multiple tasks, which can be assigned to multiple workers. For aircraft capable of interception / collision, we construct them as multiple workers capable of performing multiple tasks. The specific construction method is described in the next subsection. and These represent the task list set and the worker list set, respectively.

[0101] S211, Judgment The size. If Then jump to step S212; if Then jump to step S213; if Then proceed to step S214.

[0102] S212,

[0103] S213,

[0104] S214

[0105] S22, Extended Hungarian Algorithm

[0106] S221. Input the location set of the interceptor / collider cluster and the target cluster.

[0107] S222. Create a set based on the definitions of intercept / collision cluster and hostile cluster.

[0108] S223, Jump to step S21 to construct the task list and worker list, and obtain

[0109] S224. Calculate the cost matrix based on the cluster state estimated in step one. ij =|| e p i - e p tj ||,i=1,…,N,j=1,,M.

[0110] S225. Use the Hungarian algorithm to calculate the best match for the constructed task list, worker list, and cost matrix.

[0111] S226. Output the set of target locations for allocation.

[0112] This invention introduces cluster state estimation based on Monte Carlo localization and an extended Hungarian algorithm as key technologies to support cooperative interception / collision. The invention verifies the effectiveness of the algorithm under airborne perception and control, consolidating its feasibility in practical applications. These contributions collectively promote the development of multi-target interception / collision technology, providing important tools and methods for effectively addressing multi-target intrusion threats. Attached Figure Description

[0113] Figure 1 This is a schematic diagram of the coordinate system and camera imaging model.

[0114] Figure 2 This is a flowchart of cluster multi-target observation and task allocation.

[0115] Figure 3a This is a diagram of the previous state.

[0116] Figure 3b This is a diagram illustrating the prediction phase.

[0117] Figure 3c This is a diagram illustrating the update phase.

[0118] Figure 3d This is a schematic diagram of the resampling stage.

[0119] Figure 3e This is a schematic diagram of the forward recursion stage.

[0120] Figure 4a This is a diagram illustrating task allocation and combination.

[0121] Figure 4b This is a diagram illustrating the task allocation results. Detailed Implementation

[0122] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0123] Example: Given two drones and two targets in an unknown environment, target estimation and task allocation need to be performed. The specific steps are as follows:

[0124] Step 1: Cluster State Estimation Based on Monte Carlo Localization

[0125] S11, Cluster Status Representation

[0126] Input the estimated position from the previous time step, such as... Figure 3a As shown, the posterior distribution Modeled as a weighted set of 4 particles:

[0127]

[0128] Its specific location coordinates are shown in Table 1 below:

[0129]

[0130] Table 1

[0131] S12, Aircraft Motion Prediction

[0132] Input sampling IMU information like Figure 3b As shown, a state update is performed. After updating based on sensor sampling information and the model, the states of each particle are shown in Table 2:

[0133]

[0134] Table 2

[0135] S13, Aircraft Measurement Update

[0136] Particles are weighted based on the degree of agreement between image measurements and particle states, such as... Figure 3cAs shown, the weighted states of each particle are shown in Table 3:

[0137]

[0138] Table 3

[0139] S14, Monte Carlo positioning

[0140] After the update steps, such as Figure 3d As shown, we start from the set Four particles were resampled to form a new set. Each particle has a probability The samples were taken. The particle states after resampling are shown in Table 4:

[0141]

[0142]

[0143] Table 4

[0144] S15, Forward Propagation

[0145] Multiple application of prediction steps involving interception / collision of aircraft motion, such as Figure 3e As shown, the probability distribution is updated to the current time t. k The updated particle states are shown in Table 5:

[0146]

[0147] Table 5

[0148] Step 2: Task allocation based on the distributed extended Hungarian algorithm

[0149] Based on the cluster state estimation results from step one, an unbiased estimate of the aircraft cluster and the target cluster is obtained by weighted averaging of the particles. The set is then obtained. and

[0150] S21. Construct the task list and worker list.

[0151] we will and Redefine them as task lists and worker lists. In this example... so Possible combinations of task assignments from the task list and worker list include: Figure 4a As shown, aircraft 1 can select target 1 and target 2, and aircraft 2 can also select target 1 and target 2.

[0152] S22, Extended Hungarian Algorithm

[0153] Calculate the cost matrix based on the task list and worker list, and apply the extended Hungarian algorithm to obtain the task allocation results as follows: Figure 4b As shown, aircraft 1 and aircraft 2 selected the corresponding intercept / collision targets, and the specific allocation results are: (aircraft 1, target 1), (aircraft 2, target 2).

Claims

1. A method for multi-target observation and allocation in aircraft swarms, characterized in that: The steps include the following: Step 1: Cluster state estimation based on Monte Carlo positioning; At a given time t k Target location detected on the RGB image s p m,k and time t k-1 By time t k Motion estimation between u k The multi-view estimator (MTE) estimates the intercept / collision vehicle at time t. k state χ k Specifically, MTE uses a particle filter to estimate the posterior probability. Where z 0:k and u 0:k These represent the time intervals from the initial time to the current time t. k A collection of images and motion measurements; Step 2: Task allocation based on the distributed extended Hungarian algorithm; This includes: constructing task lists and worker lists, and extending the Hungarian algorithm; among which, HS and IS are redefined as task lists and worker lists; based on the target allocation model, important targets are subdivided into multiple tasks and assigned to multiple workers; for aircraft capable of interception / collision, they are constructed as multiple workers to perform multiple tasks. and n = max(M, N) represents the task list set and the worker list set, respectively.

2. The method for multi-target observation and allocation of aircraft swarms according to claim 1, characterized in that: Step one further includes the cluster state, represented as: Let k represent time t. k The index of the IMU measurement; according to the cluster state model, the continuous state equation of aircraft i is discretized within the IMU sampling period Δt as follows: The function F is defined as follows: The continuous state equation of target j is discretized as follows: x tj,k =x tj,k-1 +F t (x tj,k-1 )Δt (3) Wherein, function F t The definition is as follows: posterior distribution Modeled as a weighted set of L particles: in, This is the state of the l-th particle. These are the corresponding weights; then, the particle filter updates the particle set at each time point and applies four steps: prediction, update, resampling, and forward propagation.

3. The method for multi-target observation and allocation in aircraft swarms according to claim 2, characterized in that: Step one further includes aircraft motion prediction, expressed as: Input sampling IMU information Then, the state is updated based on the IMU noise measurement model as follows: Among them, the function sample(σ 2 In this step, a point is randomly sampled from a Gaussian distribution with zero mean and σ variance; 4. The method for multi-target observation and allocation of aircraft swarms according to claim 2, characterized in that: Step one further includes aircraft measurement updates, represented as: Among them, the function prob(a,σ) 2 Calculate its parameter a in a region with variance σ. 2 The probability under a zero-center Gaussian distribution, a measurement quantity s p m Calculated from camera imaging model and image delay measurement model; compared with collected target features s p m Target characteristics predicted by aircraft motion s p k-D And assign lower weights to particles that do not match.

5. The method for multi-target observation and allocation in aircraft swarms according to claim 2, characterized in that: In step one, the Monte Carlo localization is further included, represented as: After the update, use a replacement method from the collection. Resample L particles, where each particle has a probability Sampled; let the new set be 6. The method for multi-target observation and allocation in aircraft swarms according to claim 5, characterized in that: For each particle, make the following update: For each particle, with probability from Select i in the selection, and... Join middle.

7. The method for multi-target observation and allocation in aircraft swarms according to claim 2, characterized in that: In step one, the forward propagation is as follows: For delayed image measurements, at time t k-D pass Obtained probability distribution The probability distribution is updated to the current time t by repeatedly applying prediction steps involving interception / collision of the aircraft's motion. k : Where dσ is M D The differential area, including the coordinate χ k-1 ,χ k-2 ,...,χ k-D .

8. The method for multi-target observation and allocation of aircraft swarms according to claim 1, characterized in that: Step two also includes the following steps: Step 2.1, Judgment Size; if Then proceed to step 2.2; if Then proceed to step 2.3; if Then proceed to step 2.4; Step 2.2 Step 2.3 Step 2.4 9. The method for multi-target observation and allocation in aircraft swarms according to claim 1, characterized in that: In step two, the Hungarian algorithm is extended, specifically as follows: a. Input the location set of the interceptor / collider cluster and the target cluster; b. Create sets based on the definitions of intercept / collision clusters and hostile clusters. c. Navigate to the build task list and worker list to obtain... d. Calculate the cost matrix based on the estimated cluster state; c. ij =|| e p i - e p tj ||, i = 1, ..., N, j = 1, ..., M; e. Use the Hungarian algorithm to calculate the best match for the constructed task list, worker list, and cost matrix; f. Output the set of target locations to be assigned.

Images