Situation fusion and task allocation method of distributed unmanned aerial vehicle

Abstract

The invention discloses a situation fusion and task allocation method for distributed unmanned aerial vehicles, which is used for enhancing the collaborative operation capability and task execution efficiency of an unmanned aerial vehicle group. The unmanned aerial vehicle collects environmental data through an airborne sensor, performs denoising processing by using a Gaussian filter, and extracts feature information. And forming a global situation map by using a distributed situation fusion algorithm and unmanned aerial vehicle group synchronization and fusion information. And optimal task allocation is carried out through the Bayesian network, so that optimal utilization of resources is realized. The method can effectively improve the adaptability and decision-making ability of the unmanned aerial vehicle group in a complex environment, and is suitable for military and civil fields.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicles, and in particular relates to a situation fusion and task allocation method for distributed unmanned aerial vehicles. Background Art

[0002] With the continuous advancement of science and technology, drone technology has been widely used in many fields such as military reconnaissance, surveillance, cargo transportation, environmental monitoring, agricultural spraying, etc. Especially in the military field, the collaborative combat capability of drone swarms has become an important force in modern warfare. However, existing drone systems usually face problems such as insufficient information sharing, low collaborative efficiency, and limited situational awareness. In a complex battlefield environment, drones need to accurately acquire and process a large amount of data in real time to achieve effective situational awareness and mission execution.

[0003] Most existing drone systems use centralized control, which not only limits the scalability of the system, but also increases the dependence on the central control unit. Once the control unit is damaged, the entire system may be paralyzed. In addition, the perception range and processing capabilities of a single drone are limited, making it difficult to achieve comprehensive perception of a large-scale battlefield environment. In order to overcome these limitations, a distributed drone situation fusion method is needed to achieve information sharing and collaborative operations between drones and improve the overall combat effectiveness of drone swarms.

[0004] In a distributed UAV system, each UAV acts as an independent node, which can autonomously collect environmental information and share this information with other UAVs. Through situation fusion, the UAV swarm can build a global battlefield situation map to support decision-making. However, existing situation fusion methods often rely on complex data processing algorithms and a large amount of computing resources, which are difficult to implement on resource-constrained UAV platforms. In addition, how to achieve dynamic allocation and optimization of tasks while maintaining the coordination of UAV swarms is also a problem that needs to be solved urgently. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the present invention provides a distributed UAV situation fusion and task allocation method for enhancing the collaborative operation capability and task execution efficiency of UAV swarms. UAVs collect environmental data through airborne sensors, use Gaussian filters for denoising, and extract feature information. Using a distributed situation fusion algorithm, the UAV swarm synchronizes and fuses information to form a global situation map. The optimal task allocation is performed through a Bayesian network to achieve optimal utilization of resources. It can effectively improve the adaptability and decision-making ability of UAV swarms in complex environments, and is suitable for military and civilian fields.

[0006] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0007] Step 1: UAV situational awareness;

[0008] Step 1-1: The drone uses the onboard sensors to collect the surrounding environmental data I, and pre-processes the environmental data I through signal processing operations. The pre-processed environmental data information is I f :

[0009] I f =G*I+λI (1)

[0010] Where: I f is the filtered environmental data information, G is the Gaussian kernel, I is the original environmental data collected by the UAV airborne sensor, and λ is the smoothing parameter;

[0011] Step 1-2: Assume that the drone senses the input filtered environmental data information I f (x, y), where x and y are the horizontal and vertical coordinates of the environmental information perceived by the drone, respectively. f The gradient D of (x,y) in the horizontal and vertical directions x With D y :

[0012]

[0013] in: To find the sign of partial derivative;

[0014] Computation I f The product of the gradients in the two directions (x, y), that is, the product in the direction of the X axis The product of

[0015]

[0016] Use Gaussian function and D x D y Perform Gaussian weighting to generate the intermediate calculation matrix element M A 、M B , and M C :

[0017]

[0018] Where: x ,ω y and ω xy are the weights of Gaussian weighting, is the cross product symbol;

[0019] Step 1-3: Calculate the data sensed by the drone I fHarris response value R for each pixel map :

[0020] R map ={det(M A ,M B ,M C )-α(trace(M A ,M B ,M C )) 2 <T r} (6)

[0021] Where: det(M A ,M B ,M C ) is the determinant of the matrix, trace(M A ,M B ,M C ) is the trajectory of the matrix determinant, α represents the corner point response parameter, T r Indicates the judgment threshold;

[0022] Thus, the characteristic information T of the data perceived by the drone is obtained. For any perceived pixel point (x, y)∈T, the Harris response value R that the pixel point always satisfies is obtained. map ,Right now:

[0023]

[0024] Step 2: Cloud situation integration;

[0025] Step 2-1: For feature information T i and T j , respectively construct the characteristic window Win=(w x ,h y ), and then the two-dimensional window in the feature window is converted into a feature vector n according to the arrangement of a one-dimensional matrix, that is, Win x,y =n 1,xy ; Construct feature window vector

[0026] For the feature vector The similarity constraint problem of the same category is to constrain the similarity of the same category to the minimum value, which is the target category with similar characteristics, and perform similarity matching, as shown in the following formula:

[0027]

[0028] Let the matrix S V The singular values ​​of are:

[0029] S V =PΣP T(9)

[0030] Where: P is the transformation matrix, Σ is the N×N singular value matrix;

[0031] Step 2-2: Set Σ K is the matrix S V The diagonal matrix consisting of the first K singular values, P ·K =Σ K The corresponding left singular value vector is A definite solution of is:

[0032]

[0033] The original objective function problem is rewritten as:

[0034]

[0035] Step 2-3: Put U K = {u 1 ,u 2 ,u 3 ,...,u N} as the input layer of the network, H as the output layer of the network, and solve the objective function of the problem as a deep belief network model; the network input layer u of each layer i ∈U K As a visible variable, h j ∈H is regarded as a hidden variable, and the definition of the energy function is obtained:

[0036]

[0037] Where θ = {T, d, c} is the model parameter, D and M represent the number of visible units and hidden units in the network respectively; W ij represents the characteristic window value based on the i and j parameters, d i represents the corresponding weight of the visible variable, c j represents the corresponding weights of hidden variables;

[0038] Step 2-4: Define the joint distribution of {u,h} as:

[0039]

[0040] Where Z(θ)=∑ u ∑ h exp(-E(u,h;θ)) is the distribution function that ensures probability normalization;

[0041] By defining the value range of the model similarity constraint parameter θ, the similarity of the data feature value classification can be changed, that is, the similarity of the classification of different feature information T can be achieved. i and Tj The matching fusion problem.

[0042] Step 3: Optimal task allocation;

[0043] Step 3-1: For the multi-UAV target assignment task, its Bayesian network is represented as:

[0044] B=<G,P> (14)

[0045] Where: G =<U,S,A> is a directed acyclic graph, U={u 1 ,u 2 ,u 3 ,...,u N} is the UAV member participating in the mission, S={S T1 ,S T2 ,...,S Tm} is to identify the true target to be hit, A is the set of arcs of graph G, P is the probability annotation of graph G; for any UAV member u performing the mission k , each element in P represents the conditional probability density of the target node, according to the probability density rule:

[0046]

[0047] Step 3-2: For any target task node S in the Bayesian network Tm , can find a Tm The smallest subset whose conditions are not independent So that:

[0048] P(S Tm |S T1 ,S T2 ,...,S Tm-1 )=P(S Tm |S u ) (16)

[0049] Where: S u For node S Tm In Figure G =<U,S,A> The parent node set in ;

[0050] The task node S can be uniquely determined Tm Assigned to drone u k The probability distribution of is:

[0051]

[0052] For drone members u l have:

[0053]

[0054] Then we can obtain the situation fusion and task allocation method of the UAV, so that the UAV can fly safely in the environment while collecting environmental information in a distributed manner and integrating all the information. On this basis, the perceived task objectives are uniformly allocated to achieve the global optimum.

[0055] Preferably, the preprocessing is denoising using a Gaussian filter.

[0056] Preferably, the Gaussian kernel G is

[0057] A computer program enables a computer to execute the above situation fusion and task allocation method.

[0058] An electronic device comprises: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the above-mentioned situation fusion and task allocation method.

[0059] A computer-readable storage medium stores a computer program, which implements the above-mentioned situation fusion and task allocation method when executed by a processor.

[0060] A chip includes: a processor, which is used to call and run a computer program from a memory, so that a device equipped with the chip executes the above-mentioned situation fusion and task allocation method.

[0061] A computer program product, comprising a computer storage medium storing a computer program, wherein the computer program comprises instructions executable by at least one processor, and the above-mentioned situation fusion and task allocation method is implemented when the instructions are executed by the at least one processor.

[0062] The beneficial effects of the present invention are as follows:

[0063] The present invention not only improves the situational awareness and information sharing capabilities of drone swarms, but also improves the adaptability and decision-making capabilities of drone swarms in complex environments through intelligent task allocation strategies, and has important military and civilian application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a flow chart of the steps of the present invention.

[0065] Figure 2 It is a schematic diagram of the situation fusion method of the present invention.

[0066] Figure 3A schematic diagram of the feature classification of different features according to an embodiment of the present invention, where objects of different shapes are placed on an open space, a drone is used to take a bird's-eye view of the test area, and a test is performed.

[0067] Figure 4 This is a distributed fusion case of an embodiment of the present invention. The left side shows the local information perceived by the distributed UAV, and the right side shows the overall situation information case after information fusion.

[0068] Figure 5 Schematic diagram of the process for optimal allocation of drone tasks. DETAILED DESCRIPTION

[0069] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0070] like Figure 1 and Figure 2 As shown, the present invention provides a distributed UAV situation fusion and task allocation method, which realizes efficient collaborative operation of UAV groups through three steps: UAV situation awareness, cloud situation fusion and optimal task allocation.

[0071] In the UAV situational awareness step, each UAV collects environmental data using its onboard sensors and performs preprocessing to reduce noise and interference. A Gaussian filter is used to denoise the raw data, and feature information is extracted by calculating the gradient and Harris response value to obtain the feature information of the data perceived by the UAV.

[0072] In the cloud situation fusion step, the drone swarm uses a distributed situation fusion algorithm to synchronize and fuse the local information perceived by each drone to form a global situation map. The algorithm converts feature information into feature vectors and uses a deep belief network model for similarity matching and fusion, thereby achieving accurate fusion of information perceived by different drones.

[0073] In the optimal task allocation step, according to the global situation map, the task allocation module of the drone swarm can dynamically allocate tasks to each drone to optimize resource utilization and improve task execution efficiency. The Bayesian network is introduced for task allocation modeling, and prediction and causal analysis are performed through the probabilistically annotated directed acyclic graph to achieve dynamic adjustment of the real-time task allocation strategy.

[0074] The distributed UAV situation fusion and task allocation method of the present invention not only improves the situation awareness and information sharing capabilities of the UAV swarm, but also improves the adaptability and decision-making ability of the UAV swarm in complex environments through intelligent task allocation strategies, and has important military and civilian application value.

[0075] Example:

[0076] The situation fusion and task allocation method of distributed UAVs mainly includes three steps: UAV situation awareness, cloud situation fusion, and optimal task allocation.

[0077] Step 1: Drone situational awareness.

[0078] Each drone is equipped with a situational awareness module to collect the drone's own status information and surrounding environment information. Assume that the drone uses the onboard sensor (usually the onboard visible light payload) to collect the surrounding environmental data I. At this time, these data are pre-processed through signal processing operations to reduce noise and interference. Here, Gaussian filter is used for denoising. The processed environmental data information is I f :

[0079] I f =G*I+λI (19)

[0080] Where: I f is the filtered environmental data information, G is the Gaussian kernel, generally I is the original environmental data collected by the UAV airborne sensor, and λ is the smoothing parameter.

[0081] In getting I f Finally, in order to reduce the amount of information transmitted and transmit more effective information based on the carrying capacity of the effective communication network, it is necessary to analyze the data sensed by the drone. f Perform feature extraction.

[0082] Assume that the UAV perceives the input filtered environmental data information I f (x, y), where x and y are the horizontal and vertical coordinates of the environmental information perceived by the drone, respectively. Then, I can be obtained by calculation f The gradient Dx and Dy of (x,y) in the horizontal and vertical directions:

[0083]

[0084] in: To find the sign of the partial derivative.

[0085] Computation I f The product of the gradients in the two directions (x, y), that is, the product in the direction of the X axis The product of

[0086]

[0087] Use Gaussian function and D x D yGaussian weighting can generate the intermediate calculation matrix element M A 、M B , and M C :

[0088]

[0089] Where: x ,ω y and ω xy are the weights of Gaussian weighting, is the cross product symbol.

[0090] Calculate the data perceived by the drone I f Harris response value R for each pixel map :

[0091] R map ={det(M A ,M B ,M C )-α(trace(M A ,M B ,M C )) 2 <T r} (twenty four)

[0092] Where: det(M A ,M B ,M C ) is the determinant of the matrix, trace(M A ,M B ,M C ) is the trajectory of the matrix determinant, α represents the corner point response parameter, T r Represents the judgment threshold. If it is greater than this threshold, the point is considered a feature point.

[0093] Thus, the characteristic information T of the data perceived by the drone can be obtained. For any perceived pixel point (x, y)∈T, the Harris response value R that the pixel point always satisfies can be obtained. map ,Right now:

[0094]

[0095] Step 2: Cloud situation integration.

[0096] The drone swarm uses a distributed situation fusion algorithm that can synchronize and fuse situation information between drones to form a global situation map. i and T j The matching fusion problem can be transformed into, for the feature information T i and T jAmong them, the feature window Win = (w x , h y ). The two-dimensional window in the feature window is converted into a feature vector n according to the arrangement of a one-dimensional matrix, that is, Win x,y = n 1,xy . Construct the feature window vector

[0097] Regarding the similarity constraint problem of the feature vector , constraining the similarity of the same category to the minimum value is the target category with similar features, and similarity matching can be performed, as shown in Equation (12):

[0098]

[0099] Since Equation (8) is an optimization problem of a matrix, to obtain its optimal solution, it is first necessary to convert it into the form of a singular value matrix. Let the singular values of matrix S V be respectively:

[0100] S V = PΣP T (27)

[0101] Where: P is the transformation matrix, and Σ is an N×N singular value matrix.

[0102] Assume that Σ K is a diagonal matrix composed of the first K singular values of matrix S V , and P ·K is the left singular value vector corresponding to Σ K , then there is

[0103]

[0104]

[0105] K = U K <T is still the solution to the problem. Therefore, the problem of the original objective function can be rewritten as:

[0105]

[0106] Taking U K = {u 1 , u 2 , u 3 ,..., u N} as the input layer of the network and H as the output layer of the network, then the objective function of this problem can be solved as a deep belief network model. Similar to the energy function of the deep belief network, if the network input layer u i ∈ U KAs a visible variable, h i ∈H is regarded as a hidden variable, then the definition of the energy function can be obtained:

[0107]

[0108] Among them, θ = {T, d, c} is the model parameter, D and M represent the number of visible units and hidden units in the network respectively. The joint distribution of {u, h} is defined as:

[0109]

[0110] Where Z(θ)=∑ u ∑ h exp(-E(u,h;θ)) is the distribution function that ensures probability normalization. By defining the value range of the model similarity constraint parameter θ, the similarity of the data feature value classification can be changed. That is, for different feature information T i and T j The matching fusion problem.

[0111] After fusing the individual environmental information perceived by multiple drones distributed around the battlefield, overall information about the battlefield situation can be formed. On this basis, collaborative planning of multiple drones can be carried out to better execute tasks and ensure the flight safety of drones.

[0112] Step 3: Optimal task allocation.

[0113] According to the global situation map, the task allocation module of the drone swarm can dynamically allocate tasks to each drone to optimize resource utilization and improve task execution efficiency. The Bayesian network is introduced into the drone target allocation task modeling to solve the dynamic adjustment of the real-time allocation strategy in task allocation and select the optimal allocation.

[0114] A Bayesian network is a directed acyclic graph with probability annotations that can be used to reveal learning and statistical inference functions for prediction, causal analysis, etc. For the multi-UAV target allocation task, its Bayesian network can be expressed as:

[0115] B=<G,P> (32)

[0116] Where: G =<U,S,A> is a directed acyclic graph, U={u 1 ,u 2 ,u 3 ,...,u N} is the UAV member participating in the mission, S={S T1 ,S T2 ,...,S Tm} is to identify the true target to be hit, A is the set of arcs of graph G, P is the probability annotation of graph G, for any UAV member u performing the mission k , each element in P represents the conditional probability density of the target node, according to the probability density rule:

[0117]

[0118] Obviously, the calculation of probability P(S) requires 2 m-1 The amount of calculation is very huge. Therefore, the variable independence assumption is introduced in the Bayesian network, which greatly reduces the prior probability required to define the Bayesian network. For the constructed probability density rule, for any target task node S in the network structure Tm , you can find a Tm The smallest subset whose conditions are not independent So that:

[0119] P(S Tm |S T1 ,S T2 ,...,S Tm-1 )=P(S Tm |S u ) (34)

[0120] Where: S u For node S Tm In Figure G =<U,S,A> In this way, the task node S can be uniquely determined Tm Assigned to drone u k The probability distribution of is:

[0121]

[0122] The same is true for the other real targets that need to be assigned to attack drones, such as drone member u l have:

[0123]

[0124] Then we can obtain the situation fusion and task allocation method of UAV, so that the UAV can fly safely in the environment while collecting environmental information in a distributed manner and integrating all the information to form an overall understanding of the environment. On this basis, the perceived task objectives are uniformly allocated to achieve the global optimum.

[0125] Figure 3The feature classification for different features is shown. Objects of different shapes (square, triangle and circle) are placed on the open ground, and the test area is photographed from the air using a drone. It can be seen that no matter how the objects of different shapes are placed, the object types can be accurately identified and classified after translation, rotation and scaling.

[0126] Figure 4 The figure shows a case of distributed fusion. On the left is the local information perceived by the distributed UAVs. It can be seen that each UAV perceives part of the enemy air defense identification zone in the environment, providing data support for the subsequent information fusion of multiple UAVs. On the right is an example of overall situation information after information fusion. The airborne sensors of the distributed UAVs can identify three enemy air defense identification zones in the environment (the actual environment sets four enemy air defense identification zones). There is one air defense identification zone that cannot be presented in the overall situation after information fusion because no UAV perceives the information. This is consistent with the results of actual combat, that is, the overall battlefield situation constructed in an unknown environment can only be a reproduction of the information that has been perceived.

[0127] Figure 5 Flowchart showing the optimal allocation of UAV tasks.

Claims

1. A distributed UAV situation fusion and task allocation method, characterized in that: The steps include: Step 1: UAV situational awareness; Step 1-1: The drone uses the onboard sensors to collect the surrounding environmental data I, and pre-processes the environmental data I through signal processing operations. The pre-processed environmental data information is I f : I f =G*I+λI (1) Where: I f is the filtered environmental data information, G is the Gaussian kernel, I is the original environmental data collected by the UAV airborne sensor, and λ is the smoothing parameter; Step 1-2: Assume that the drone senses the input filtered environmental data information I f (x, y), where x and y are the horizontal and vertical coordinates of the environmental information perceived by the drone, respectively. f The gradient D of (x,y) in the horizontal and vertical directions x With D y : in: To find the sign of partial derivative; Computation I f The product of the gradients in the two directions (x, y), that is, the product in the direction of the X axis The product of Use the Gaussian function and D x D y Perform Gaussian weighting to generate the intermediate calculation matrix element M A 、M B , and M C : Where: x ,ω y and ω xy are the weights of Gaussian weighting, is the cross product symbol; Step 1-3: Calculate the data sensed by the drone I f Harris response value R for each pixel map : R map ={det(M A ,M B ,M C )-α(trace(M A ,M B ,M C )) 2 <T r } (6) Where: det(M A ,M B ,M C ) is the determinant of the matrix, trace(M A ,M B ,M C ) is the trajectory of the matrix determinant, α represents the corner point response parameter, T r Indicates the judgment threshold; Thus, the characteristic information T of the data perceived by the drone is obtained. For any perceived pixel point (x, y)∈T, the Harris response value R that the pixel point always satisfies is obtained. map ,Right now: Step 2: Cloud situation integration; Step 2-1: For feature information T i and T j , respectively construct the characteristic window Win=(w x ,h y ), and then the two-dimensional window in the feature window is converted into a feature vector n according to the arrangement of a one-dimensional matrix, that is, Win x,y =n 1,xy ; Construct feature window vector For the feature vector The similarity constraint problem of the same category is to constrain the similarity of the same category to the minimum value, which is the target category with similar characteristics, and perform similarity matching, as shown in the following formula: Let the matrix S V The singular values ​​of are: S V =PΣP T (9) Where: P is the transformation matrix, Σ is the N×N singular value matrix; Step 2-2: Set Σ K is the matrix S V The diagonal matrix consisting of the first K singular values, P ·K =Σ K The corresponding left singular value vector is A definite solution of is: The original objective function problem is rewritten as: Step 2-3: Put U K ={u1,u2,u3,...,u N } as the input layer of the network, H as the output layer of the network, and solve the objective function of the problem as a deep belief network model; the network input layer u of each layer i ∈U K As a visible variable, h j ∈H is regarded as a hidden variable, and the definition of the energy function is obtained: Where θ = {T, d, c} is the model parameter, D and M represent the number of visible units and hidden units in the network respectively; W ij represents the characteristic window value based on the i and j parameters, d i represents the corresponding weight of the visible variable, c j represents the corresponding weights of hidden variables; Step 2-4: Define the joint distribution of {u,h} as: Where Z(θ)=∑ u ∑ h exp(-E(u,h;θ)) is the distribution function that ensures probability normalization; By defining the value range of the model similarity constraint parameter θ, the similarity of the data feature value classification can be changed, that is, the similarity of the classification of different feature information T can be achieved. i and T j The matching fusion problem; Step 3: Optimal task allocation; Step 3-1: For the multi-UAV target assignment task, its Bayesian network is represented as: B=<G,P> (14) Where: G =<U,S,A> is a directed acyclic graph, U = {u1,u2,u3,...,u N } is the UAV member participating in the mission, S={S T1 ,S T2 ,...,S Tm } is to identify the true target to be hit, A is the set of arcs of graph G, P is the probability annotation of graph G; for any UAV member u performing the mission k , each element in P represents the conditional probability density of the target node, according to the probability density rule: Step 3-2: For any target task node S in the Bayesian network Tm , can find a Tm The smallest subset whose conditions are not independent So that: P(S Tm |S T1 ,S T2 ,...,S Tm-1 )=P(S Tm |S u ) (16) Where: S u For node S Tm In Figure G =<U,S,A> The parent node set in ; The task node S can be uniquely determined Tm Assigned to drone u k The probability distribution of is: For drone members u l have: Then we can obtain the situation fusion and task allocation method of the UAV, so that the UAV can fly safely in the environment while collecting environmental information in a distributed manner and integrating all the information. On this basis, the perceived task objectives are uniformly allocated to achieve the global optimum.

2. A distributed UAV situation fusion and task allocation method according to claim 1, characterized in that: The preprocessing is to use a Gaussian filter to perform denoising.

3. The distributed UAV situation fusion and task allocation method according to claim 1 is characterized in that: The Gaussian kernel G is 4. A computer program, characterized in that The computer program enables a computer to execute the method according to any one of claims 1 to 3.

5. An electronic device, characterized in that: include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method as claimed in any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

7. A chip, characterized in that: include: A processor, configured to call and run a computer program from a memory, so that a device equipped with the chip executes a method as claimed in any one of claims 1 to 3.

8. A computer program product, characterized in that The computer program product comprises a computer storage medium storing a computer program, wherein the computer program comprises instructions executable by at least one processor, and when the instructions are executed by the at least one processor, the method according to any one of claims 1 to 3 is implemented.

Images