Multi-aircraft airborne meteorological collaborative awareness method

Abstract

The invention relates to a multi-aircraft airborne meteorological collaborative awareness method, belongs to the technical field of aviation traffic management, and solves the problems of low precision and small range of airborne meteorological situation awareness in the prior art. According to the multi-aircraft airborne weather cooperative sensing method, the multi-aircraft airborne weather cooperative sensing method is oriented to autonomous operation multi-aircraft situation intelligent cooperative sensing, and a dynamic cooperative multi-aircraft sensing network in a region is constructed by combining aircraft radar echo data, weather information and flight intention, so that efficient transmission of situation feature information is realized; the credible sensing range and credibility are improved, the robustness of an air traffic system is improved, and the flight safety is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of aviation traffic management, and specifically to a method for collaboratively sensing airborne meteorological conditions of multiple aircraft. Background Art

[0002] With the continuous development of the civil aviation industry, the number of flights and passenger transport has increased year by year. The rapid growth of air transport volume has increased the complexity of air traffic operations. The current centralized interval control mode is difficult to meet the safety operation requirements in a highly complex airspace environment. Many existing trunk route networks have exceeded their capacity limits, and flight delays are frequent during peak hours, and air traffic management is facing tremendous pressure. The traditional centralized operation mode centered on ground control is gradually unable to meet the operation requirements in a highly complex airspace environment.

[0003] To meet this challenge, the civil aviation industry has proposed the concept of "autonomous air traffic operation" to improve navigation flexibility and safety. Airborne situational awareness is the primary prerequisite for autonomous air traffic operation. Reliable and accurate airspace situation information is the basis for subsequent track decision-making, conflict resolution and other tasks. There are currently two main ways to achieve situational awareness in civil aviation: 1) Traditional airborne situational awareness: relying on single-machine sensors and instruments, with limited sensing range and capabilities; 2) Upload from ground monitoring systems: ground detection base stations have long detection distances, and poor real-time and dynamic sensing.

[0004] To overcome the above technical difficulties, multi-aircraft situational collaborative perception provides a new solution for situational awareness in future autonomous operation scenarios. The purpose is to share situational information of multiple aircraft in the region, improve the range and credibility of trusted perception, and improve the robustness of the air traffic system.

[0005] However, the current research on airborne collaborative perception technology is still in its infancy, facing challenges such as the heterogeneity and complexity of multi-source data fusion and limited communication resources. Therefore, in order to solve the shortcomings of current technology, it is urgent to provide a multi-aircraft situation intelligent collaborative perception method for autonomous operation. Summary of the invention

[0006] In view of the above problems, the present invention provides a multi-aircraft airborne meteorological collaborative perception method, which is aimed at the intelligent collaborative perception of multi-aircraft situation in autonomous operation. By combining aircraft radar echo data, meteorological information and flight intentions, a dynamic collaborative multi-aircraft perception network in the region is constructed to achieve efficient transmission of situation feature information, improve the trusted perception range and credibility, improve the robustness of the air traffic system, and ensure flight safety.

[0007] The present invention discloses a multi-aircraft airborne meteorological collaborative sensing method, comprising the following steps:

[0008] Step 1. Collect the spatial position, motion data, weather, weather situation observation pictures, static information and airport information of the aircraft through the weather radar, GPS, inertial navigation system and flight management system installed in the aircraft; use the spatial position, motion data, weather, weather situation observation pictures, static information and airport information of the aircraft as a training set;

[0009] Step 2. Based on the goal of the collaborative sensing network, determine the maximum image fusion performance function of the multi-aircraft image fusion model under the limitation of the total communication bandwidth;

[0010] Step 3. Based on the training set of step 1 and the maximized image fusion performance function of step 2, a multi-aircraft image fusion model is obtained by combining spatiotemporal confidence, aircraft flight intention, sparsely connected two-way communication network and situation information fusion function;

[0011] Among them, the spatiotemporal confidence is obtained through the spatiotemporal confidence detection module, which is used to evaluate the confidence information in the spatial and temporal dimensions of the meteorological situation observation picture;

[0012] Through the aircraft flight intention module, the aircraft position, movement data, weather and static information are input to obtain the aircraft flight intention of the aircraft predicted arrival area in the future time;

[0013] Sparsely connected two-way communication network for transmitting sparse and critical characteristic information between aircraft;

[0014] The situation information fusion function is obtained through the situation information fusion module, which is used to fuse the spatiotemporal confidence and the flight intention information of the aircraft based on the directed graph of the collaborative relationship between the aircraft in a sparsely connected two-way communication network;

[0015] Step 4. Obtain the multi-aircraft airborne meteorological collaborative perception result image based on the multi-aircraft image fusion model.

[0016] Optionally, the weather is precipitation, hail, thunderstorm, and turbulence.

[0017] Optionally, the motion data includes aircraft speed, heading angle, and aircraft flight plan.

[0018] Optionally, the static information includes the latitude and longitude positions of the waypoints and airport information.

[0019] Optionally, the expression of the total communication bandwidth B in step 1 is:

[0020]

[0021] In the formula, is the binary selection matrix in space of the information transmitted from the i-th aircraft to the j-th aircraft in the k-th round of communication at time t, It represents the total number of grids in which the i-th aircraft transmits information to the j-th aircraft in the k-th round of communication at time t; D represents the feature channel dimension; float represents the precision coefficient; K represents the total number of communication rounds.

[0022] Optionally, the expression of the target of the collaborative sensing network in step 1 is:

[0023]

[0024] In the formula, represents the information transmitted by the j-th aircraft to the i-th aircraft in the k-th round of communication at time t; K represents the total number of communication rounds; N represents the total number of aircraft.

[0025] Optionally, the expression of the objective function for maximizing image fusion performance is:

[0026]

[0027] Where θ represents the parameters to be trained in the image fusion model; e(·) is the image fusion evaluation index; Φ θ (·) is the multi-aircraft image fusion model; It represents the observation picture of the surrounding meteorological situation by the i-th aircraft at time t; Represents the true perception picture of the surrounding weather situation of the i-th aircraft at time t.

[0028] Optionally, the spatiotemporal confidence detection module in step 3 includes a feature encoding submodule, a spatial confidence map generator submodule and a spatiotemporal confidence detection module;

[0029] The feature encoding submodule is used to extract high-dimensional features from the aircraft's observation pictures of the surrounding meteorological situation and obtain feature maps in the aircraft's communication;

[0030] The feature map is obtained by convolution of the RGB image. Through multiple layers of convolution, the edge, texture, geometry and intensity features of the weather in the predicted image are extracted;

[0031] A spatial confidence map generator submodule, for generating a spatial confidence map based on the characteristic map of each round of communication of each aircraft;

[0032] The spatiotemporal confidence detection module uses an exponential decay function to simulate the time decay of temporal confidence.

[0033] Optionally, the flight intention module of the aircraft in step 3 obtains the flight intention of the aircraft based on the spatial likelihood factor and the temporal likelihood factor.

[0034] Optionally, the sparsely connected bidirectional communication network in step 3 is used to transmit sparse and critical feature information in the following steps:

[0035] Confirms the perception confidence of each aircraft and the perception requests received from other aircraft;

[0036] Sparse and key feature information is extracted based on the perception confidence of each aircraft and the perception requests received from other aircraft.

[0037] Compared with the prior art, the present invention has at least the following beneficial effects:

[0038] 1) Improve the ability to perceive severe weather:

[0039] The present invention uses meteorological collaborative perception technology and data sharing and information fusion among multiple aircraft to make up for the insufficient perception of a single aircraft due to radar blind spots, distance attenuation and other reasons. Especially under complex weather conditions such as severe thunderstorms and low visibility, the situational awareness capability of high-risk areas is significantly improved.

[0040] 2) Optimize communication bandwidth utilization:

[0041] The present invention constructs a sparse communication network and a multi-head attention mechanism to dynamically select the situation information of key areas for sharing, thereby reducing unnecessary data transmission. While ensuring the perception accuracy, it greatly reduces the occupancy rate of the communication bandwidth, improves the efficiency of information sharing and the overall robustness of the system.

[0042] 3) Enhance the system's dynamic adaptability:

[0043] The present invention designs a time confidence assessment module and a flight intention analysis module, which can dynamically evaluate data reliability based on historical data, and predict the areas that may be reached in the future in combination with the aircraft's flight plan, thereby achieving rapid response and efficient coordination to short-term weather changes.

[0044] 4) Improve the accuracy and fusion quality of perception data:

[0045] The present invention utilizes a multimodal data fusion algorithm and a collaborative perception model based on a multi-head attention mechanism to efficiently fuse the meteorological observation data of multiple aircraft to generate a more comprehensive situation map, significantly improving the accuracy, completeness and spatial resolution of meteorological information.

[0046] 5) Meeting the needs of complex airspace operations:

[0047] The feasibility of the method was verified through simulation. The results show that the method can effectively ensure flight safety under the conditions of complex airspace, high aircraft density and drastic weather dynamic changes, and meet the actual needs of collaborative perception in civil aviation operations.

[0048] 6) Support multi-round collaborative perception:

[0049] The collaborative perception framework proposed in the present invention can support multi-round collaboration under limited communication resources. Experiments have shown that multi-round collaboration has significant advantages in improving perception effects, especially under limited bandwidth conditions, multi-round communication can significantly improve system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The present invention is a flowchart of the method for collaboratively sensing airborne weather on multiple aircrafts.

[0051] Figure 2 This is a flow chart of step 3 of the multi-aircraft airborne weather collaborative perception method of the present invention.

[0052] Figure 3 This is a flow chart of step 331 of the multi-aircraft airborne weather collaborative perception method of the present invention.

[0053] Figure 4 This is a flow chart of step 332 of the multi-aircraft airborne weather collaborative perception method of the present invention.

[0054] Figure 5 The meteorological situation observation pictures are input into the multi-aircraft image fusion model of the present invention.

[0055] Figure 6 This is a picture of the image fusion result obtained by the multi-aircraft airborne meteorological collaborative perception method of the present invention. DETAILED DESCRIPTION

[0056] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. In addition, the present invention can also be implemented in other ways different from those described herein, and therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0057] A specific embodiment of the present invention, as Figure 1-Figure 6 , discloses a multi-aircraft airborne weather collaborative perception method, comprising the following steps:

[0058] Step 1. Collect the spatial position, motion data, weather, meteorological situation observation pictures, static information and airport information of the aircraft through the weather radar, GPS, inertial navigation system and flight management system installed on the aircraft; use the spatial position, motion data, weather, meteorological situation observation pictures, static information and airport information of the aircraft as a training set.

[0059] Specifically, motion data includes aircraft speed and aircraft flight plan; meteorological situation observation pictures are pictures about weather displayed in meteorological radar observation pictures, including red, yellow and green, which respectively represent the intensity of radar echo in the observed area (i.e., route), with red being the strongest, yellow being the second, and green being the weakest. Areas without color represent no echo. The stronger the echo intensity, the greater the intensity of precipitation, hail, thunderstorm, and turbulence in the area; meteorological situation (i.e., weather) includes meteorological phenomena such as precipitation, hail, thunderstorm or turbulence; static information includes the latitude and longitude positions of waypoints; airport information includes runway location, number, length, airport name, and latitude and longitude positions.

[0060] Exemplarily, the weather situation observation picture is an airborne weather radar echo data observation picture in the form of an RGB picture from a bird's-eye view (BEV) of the surrounding weather situation.

[0061] Step 2. Based on the goal of the collaborative sensing network, determine the maximized image fusion performance function of the multi-aircraft image fusion model under the limitation of the total communication bandwidth.

[0062] Furthermore, the total communication bandwidth B is expressed as:

[0063]

[0064] In the formula, is the binary selection matrix in space of the information transmitted from the i-th aircraft to the j-th aircraft in the k-th round of communication at time t, It represents the total number of grids in which the i-th aircraft transmits information to the j-th aircraft in the k-th round of communication at time t; D represents the feature channel dimension; float represents the precision coefficient; K represents the total number of communication rounds.

[0065] Furthermore, float=32 / 8.

[0066] Exemplarily, the binary selection matrix The element values ​​include 1 and 0. The position with a value of 1 represents that in the k-th round of communication at time t, the i-th aircraft needs to transmit the corresponding feature information of the position to the j-th aircraft, and the position with a value of 0 represents that no transmission is required.

[0067] Furthermore, the expression of the goal of the collaborative sensing network is:

[0068]

[0069] In the formula, represents the information transmitted by the j-th aircraft to the i-th aircraft in the k-th round of communication at time t; K represents the total number of communication rounds; N represents the total number of aircraft.

[0070] Furthermore, the expression of the objective function to maximize the image fusion performance is:

[0071]

[0072] Where θ represents the parameters to be trained in the image fusion model; e(·) is the image fusion evaluation index; Φ θ (·) is a multi-aircraft image fusion model based on convolutional neural network and multi-head attention mechanism; It represents the observation picture of the surrounding meteorological situation by the i-th aircraft at time t; Represents the true perception picture of the surrounding weather situation of the i-th aircraft at time t.

[0073] Understandably, the message The sparse matrix information obtained after processing the observed pictures; the true value perception picture of the meteorological situation is the RGB radar echo image, which is used as the true value of the image fusion feature map obtained by the multi-aircraft image fusion model during the training process. In this way, the image fusion feature map obtained by the multi-aircraft image fusion model can continuously approach the true value during the continuous training process until convergence is achieved. Maximizing the image fusion performance means that the established image fusion model can occupy as little bandwidth as possible and achieve the best image fusion performance. Image fusion performance refers to the degree of fusion of various parameters and indicators of the image finally output by the image fusion model, including detection range, detection accuracy, distance, communication bandwidth, and the presence or absence of radar shadow areas. For example, by improving the aircraft's perception of the airspace meteorological situation through collaborative perception, the aircraft can obtain clearer information on dangerous weather such as thunderstorms, precipitation, hail, etc. on the route ahead, and obtain the best image fusion results.

[0074] Step 3. Based on the training set of step 1 and the maximized image fusion performance function of step 2, a multi-aircraft image fusion model is obtained by combining spatiotemporal confidence, aircraft flight intention, sparsely connected two-way communication network and situation information fusion function.

[0075] Among them, the training parameters θ of the multi-aircraft image fusion model are composed of the training parameters in each submodule, including the image encoder Φ in the spatiotemporal confidence detection module enc (.), spatial confidence map generation function Φ generator (.) and the selection function Φ select (.), and the situation fusion module Φ FFN (.).

[0076] Step 31. Construct a spatiotemporal confidence detection module to obtain spatiotemporal confidence, which is used to comprehensively evaluate the confidence information in the spatial and temporal dimensions of the observation image. The input is the observation image of the aircraft on the surrounding meteorological situation at the current time and the preset past time period.

[0077] Step 311. Construct a feature encoding submodule for extracting high-dimensional features from the aircraft's observation pictures of the surrounding meteorological situation and obtaining feature maps in the aircraft's communication;

[0078] The present invention extracts high-dimensional features from the initial aircraft's observation pictures of the surrounding meteorological situation, which can support the subsequent calculation of confidence.

[0079] When k = 0, the observation picture of the weather situation by the i-th aircraft at time t is encoded and mapped into the characteristic map of the initial round of communication of the aircraft at time t. It is expressed as:

[0080]

[0081] in, represents the characteristic graph of the i-th aircraft in the initial round of communication at time t; Φ enc (.) represents the image encoder, which consists of two convolutional layers and converts the RGB image into a vector of shape 16*H*W; represents a real number; H, W, and D represent the length, width, and number of channels of the feature map, respectively.

[0082] When k>0, the fusion feature map of the i-th aircraft in the previous round of communication at time t output by the multi-aircraft image fusion model As the feature map of the i-th aircraft in the current round of communication at time t, exemplarily, in the case of multiple rounds of fusion, the fused feature map output by the first round of fusion module is the feature map of the second round of communication.

[0083] Preferably, the characteristic graph of communication is a 16-channel characteristic graph.

[0084] It can be understood that the feature map is obtained by convolution of the RGB image. Through multiple layers of convolution, complex features such as edges, textures, geometric shapes, thunderstorm intensity, precipitation intensity, etc. in the predicted image are extracted.

[0085] Step 312. Construct a spatial confidence map generator submodule to generate a spatial confidence map based on the characteristic map of each aircraft, reflecting its perception ability of the surrounding airspace weather.

[0086] Furthermore, the spatial confidence map generator submodule is a convolutional layer with 16 input channels and 1 output channel, a convolution kernel size of 1, and the output value is mapped to the range of 0-1 through the sigmoid function.

[0087] Furthermore, the spatial confidence of the i-th aircraft in the k-th round of communication at time t is The expression is:

[0088]

[0089] Among them, Φ generator (.) represents the spatial confidence map generation function, which is used to convert the characteristic map of the i-th aircraft in the k-th round of communication at time t into Convert to a spatial confidence map with a range of 0-1 It consists of one convolutional layer; Represents the characteristic graph of the i-th aircraft in the k-th round of communication at time t.

[0090] Furthermore, the spatial confidence in each round of communication of each aircraft at each moment is facilitated to obtain a spatial confidence map.

[0091] Step 313. Construct a spatiotemporal confidence detection module, use an exponential decay function to simulate the time decay of the time confidence, and obtain the time confidence of the i-th aircraft in the k-th round of communication at time t before the preset past time period Δt The expression is:

[0092]

[0093] Wherein, e represents a natural constant; λ represents an attenuation coefficient; and Δt represents a preset past time period (eg, past 30 minutes).

[0094] Step 32. Construct an aircraft flight intention module to obtain the aircraft flight intention, input the aircraft position, motion data, weather and static information, and based on the aircraft's flight dynamics model, status information and airspace environment data, generate the aircraft flight intention that represents the probability distribution of the area that the aircraft may reach in the future. The higher the probability value, the greater the possibility of reaching the area.

[0095] Step 321. Obtain the spatial likelihood factor κ of the i-th aircraft i,1 , the expression is:

[0096]

[0097] in, represents normal distribution; ψ i,r (t) represents the angle of the direction of the i-th aircraft toward the target waypoint at time t; ψ i,ac (t) represents the heading angle of the ith aircraft at time t; the heading angle ψ i,ac The angle ψ with the direction towards the target waypoint i,rThe difference has a mean of 0 and a standard deviation of σ h =5° normal distribution.

[0098] Furthermore, the spatial position and motion data of the aircraft are collected through the GPS / other navigation systems (such as inertial navigation system INS, flight management system FMS) of the aircraft, and the angle and heading angle of the direction of the i-th aircraft toward the target waypoint at time t are obtained through the spatial position and motion data of the aircraft.

[0099] Furthermore, the standard deviation The normal distribution 3σ principle is adopted, that is: 3σ h The range is ±15°, which means that the aircraft heading angle has a 99.74% probability distribution within ψ r Within the range of ±15°.

[0100] It is understandable that the spatial likelihood factor κ i,1 Represents spatial information and is used to estimate the state of the aircraft.

[0101] Step 322. Obtain the time likelihood factor κ of the i-th aircraft i,2 , the expression is:

[0102]

[0103] in, Represents normal distribution; TTG i (WP r ) represents the arrival time WP of the i-th aircraft at the target waypoint r , is subject to the mean value t p , with standard deviation σ fp The normal distribution of .

[0104] It is understandable that the temporal likelihood factor κ i,2 Represents time information, used to estimate the arrival time of the target waypoint.

[0105] Step 323. Obtain the flight intention of the i-th aircraft at time t according to the likelihood function joint probability distribution The expression is:

[0106]

[0107] It is understandable that the input of the flight intention module includes position, speed, flight plan, weather and static information. Combined with the aircraft's flight dynamics information, state information and airspace environment information, the flight intention of the area that the aircraft may reach in the future is obtained. The larger the flight intention value, the more likely the aircraft is to reach the area, and in subsequent cooperative communications, the aircraft will pay more attention to the area.

[0108] Step 33. Build a sparsely connected bidirectional communication network to transmit sparse and critical feature information, thereby reducing the air-to-air communication bandwidth without affecting the perception accuracy.

[0109] Step 331. Confirm the perception confidence of each aircraft and the perception requests received from other aircraft;

[0110] Furthermore, based on the output of the spatiotemporal confidence generation module and the aircraft flight intention module, the spatial confidence map Temporal confidence map and aircraft flight intention Get the perception request of aircraft i in the kth round of communication at time t The expression is:

[0111]

[0112] In the formula, is the i-th aircraft in the preset time period t 0 ~t is the perceived confidence in the kth round of communication, is the flight intention of the i-th aircraft at time t, is the softmax function; α represents an adjustable parameter.

[0113] It can be understood that the perception request is used to reflect the need for coordination of the aircraft's position on that day. The higher the value of the perception request, the more the area needs enhanced perception and the more coordination is needed to supplement the perception information.

[0114] Furthermore, the expression of perception confidence is:

[0115]

[0116] in, Indicates that the i-th aircraft is in the preset time period t 0 The spatial confidence map in the kth round of communication at ~t, Indicates that the i-th aircraft is in the preset time period t 0 Time confidence graph in the kth round of communication at ~t:

[0117] Specifically, By the preset time period t 0 The corresponding time points within t Splicing composition; Similarly, it refers to a time period t 0 Time confidence corresponding to multiple time points within ~t Splicing composition; t 0~t means that the radar echo RGB information within this time period is used as the input of the network.

[0118] Among them, t 0 ~t is not continuous, there will be time intervals, and the time interval of the data is 6 minutes (related to the acquisition resolution of the meteorological radar).

[0119] Step 332: Extract key feature information based on the perception confidence of each aircraft and the perception requests received from other aircraft.

[0120] Step 3321. Obtain a binary selection matrix based on the perception confidence and the perception request The expression is:

[0121]

[0122] In the formula, e represents the element-by-element multiplication of the matrix, is the sensing request of aircraft i in the k-1th round of communication at time t, B is the communication bandwidth; Φ select (.) is the selection function, which is used to select the most critical area for transmission based on the input perception confidence map and perception request under a given communication bandwidth.

[0123] Specifically, the value of the element in the binary selection matrix represents whether the area is selected, 1 represents that the information of the area needs to be transmitted, and 0 represents that it does not need to be transmitted; the most critical areas are dangerous areas such as thunderstorms, precipitation, and radar shadow areas.

[0124] Step 3322. Extract the information transmitted from the jth aircraft to the ith aircraft in the kth communication at time t according to the binary selection matrix (i.e. key feature), the expression is:

[0125]

[0126] In the formula, represents the characteristic graph of the jth aircraft in the k-round communication at time t, Represents the binary selection matrix for the i-th aircraft to transmit information to the j-th aircraft in the k-th round of communication at time t.

[0127] Step 34. Construct a directed graph of collaborative relationships between aircraft. The nodes in the directed graph represent aircraft, and the edges represent information exchange between aircraft.

[0128] Specifically, the directed graph of the cooperative relationship between aircraft is the adjacency matrix A in the kth round of communication at time t: t,k , the adjacency matrix A t,kIt is composed of adjacent aircraft, where the i-th aircraft and the j-th aircraft are a pair of adjacent aircraft.

[0129] The adjacent aircraft consisting of the i-th aircraft and the j-th aircraft in the k-th round of communication at time t The expression is:

[0130]

[0131] Step 35. Based on the Transformer structure of the multi-head attention mechanism and the multi-aircraft coordination relationship diagram, a situation information fusion module is constructed to obtain the situation information fusion function (integrating the information of the spatiotemporal confidence detection module and the aircraft's flight intention module), which is expressed as:

[0132]

[0133] in, represents the situation information fusion function of the information transmitted by aircraft j under the attention weight of aircraft i in the k-th round of coordination at time t; d Z express Dimension.

[0134] Step 36. Combine the spatiotemporal confidence, the flight intention of the aircraft, the sparsely connected two-way communication network and the situation information fusion function to obtain the fusion feature map of the i-th aircraft in the k-th round of communication at time t output by the multi-aircraft image fusion model Adaptively fuse information from different aircraft to obtain the fusion result for the next round of communication, so as to improve the breadth and accuracy of aircraft's perception of airspace meteorological situation.

[0135] Furthermore, the fusion feature map of the i-th aircraft in the k-th round of communication at time t output by the multi-aircraft image fusion model is The expression is:

[0136]

[0137] In the formula, Φ FFN It is a feedforward network, which performs nonlinear transformation on the weighted feature information and outputs the reconstructed features. i Represents the adjacency matrix A t,k The neighbor aircraft of the i-th aircraft in .

[0138] Step 37. Determine whether the current communication round k is greater than or equal to K. If so, obtain a multi-aircraft image fusion model. If less than K, set k=k+1 and return to step 31.

[0139] Step 4. Obtain the multi-aircraft airborne meteorological collaborative perception result image based on the multi-aircraft image fusion model.

[0140] Specifically, a decoder is constructed. The decoder consists of two 3x3 convolutional layers (filled with 1) and a ReLU activation layer. The fusion feature map obtained by the multi-aircraft image fusion model is Convert to RGB image As the final output of multi-aircraft airborne meteorological collaborative perception, the expression is:

[0141]

[0142] In the formula, Φ dec is the decoder, which consists of two convolutional layers and fuses the feature maps 16-channel features; RGB image For 3 channels.

[0143] See also Figure 5 and 6 , Figure 5 It is a weather situation observation picture input into the multi-aircraft image fusion model. It is a picture scanned by the aircraft's onboard weather radar. The pixel color represents the radar echo intensity in the area. The area without color represents no echo. The greater the echo intensity, the greater the intensity of precipitation, hail, and thunderstorm in the area. There is a fan-shaped gap in the upper left corner of the radar echo, which represents the radar shadow area. The aircraft cannot obtain weather information in this area, which is very dangerous. Figure 6 This is the image fusion result picture obtained. It can be seen that the detection range is larger, the accuracy is higher, and the information of the radar shadow area is supplemented, which greatly improves flight safety.

[0144] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for collaboratively sensing airborne weather conditions on multiple aircraft, characterized in that: The following steps are involved: Step 1. Collect the spatial position, motion data, weather, weather situation observation pictures, static information and airport information of the aircraft through the weather radar, GPS, inertial navigation system and flight management system installed in the aircraft; use the spatial position, motion data, weather, weather situation observation pictures, static information and airport information of the aircraft as a training set; Step 2. Based on the goal of the collaborative sensing network, determine the maximum image fusion performance function of the multi-aircraft image fusion model under the limitation of the total communication bandwidth; Step 3. Based on the training set of step 1 and the maximized image fusion performance function of step 2, a multi-aircraft image fusion model is obtained by combining spatiotemporal confidence, aircraft flight intention, sparsely connected two-way communication network and situation information fusion function; Among them, the spatiotemporal confidence is obtained through the spatiotemporal confidence detection module, which is used to evaluate the confidence information in the spatial and temporal dimensions of the meteorological situation observation picture; Through the aircraft flight intention module, the aircraft position, movement data, weather and static information are input to obtain the aircraft flight intention of the aircraft predicted arrival area in the future time; Sparsely connected two-way communication network for transmitting sparse and critical characteristic information between aircraft; The situation information fusion function is obtained through the situation information fusion module, which is used to fuse the spatiotemporal confidence and the flight intention information of the aircraft based on the directed graph of the collaborative relationship between the aircraft in a sparsely connected two-way communication network; Step 4. Obtain the multi-aircraft airborne meteorological collaborative perception result image based on the multi-aircraft image fusion model.

2. The method for collaboratively sensing weather conditions on multiple aircraft according to claim 1, characterized in that: The weather is precipitation, hail, thunderstorms and turbulence.

3. The method for collaboratively sensing weather conditions on multiple aircraft according to claim 1, characterized in that: Movement data includes aircraft speed, heading angle, and aircraft flight plan.

4. The method for collaboratively sensing weather conditions on multiple aircraft according to claim 1, characterized in that: Static information includes the latitude and longitude positions of waypoints and airport information.

5. The method for collaboratively sensing weather conditions on multiple aircraft according to claim 1, characterized in that: The expression of the total communication bandwidth B in step 1 is: In the formula, is the binary selection matrix in space of the information transmitted from the i-th aircraft to the j-th aircraft in the k-th round of communication at time t, It represents the total number of grids in which the i-th aircraft transmits information to the j-th aircraft in the k-th round of communication at time t; D represents the feature channel dimension; float represents the precision coefficient; K represents the total number of communication rounds.

6. The method for cooperatively sensing weather conditions on multiple aircraft according to claim 5, characterized in that: The expression of the target of the collaborative sensing network in step 1 is: In the formula, represents the information transmitted from the jth aircraft to the ith aircraft in the kth round of communication at time t; K represents the total number of communication rounds; N represents the total number of aircraft.

7. The method for cooperatively sensing weather conditions on multiple aircraft according to claim 6, characterized in that: The expression of the objective function to maximize the image fusion performance is: Where θ represents the parameters to be trained in the image fusion model; e(·) is the image fusion evaluation index; Φ θ (·) is the multi-aircraft image fusion model; It represents the observation picture of the surrounding meteorological situation by the i-th aircraft at time t; Represents the true perception picture of the surrounding weather situation of the i-th aircraft at time t.

8. The method for collaboratively sensing weather conditions on multiple aircraft according to claim 1, characterized in that: The spatiotemporal confidence detection module in step 3 includes a feature encoding submodule, a spatial confidence map generator submodule and a spatiotemporal confidence detection module; The feature encoding submodule is used to extract high-dimensional features from the aircraft's observation pictures of the surrounding meteorological situation and obtain feature maps in the aircraft's communication; The feature map is obtained by convolution of the RGB image. Through multiple layers of convolution, the edge, texture, geometry and intensity features of the weather in the predicted image are extracted; A spatial confidence map generator submodule, for generating a spatial confidence map based on the characteristic map of each round of communication of each aircraft; The spatiotemporal confidence detection module uses an exponential decay function to simulate the time decay of temporal confidence.

9. The method for collaboratively sensing weather conditions on multiple aircraft according to claim 1, characterized in that: The aircraft's flight intention module in step 3 obtains the aircraft's flight intention based on the spatial likelihood factor and the temporal likelihood factor.

10. The method for collaboratively sensing weather conditions on multiple aircraft according to claim 1, characterized in that: The sparsely connected bidirectional communication network in step 3 is used to transmit sparse and critical feature information in the following steps: Confirms the perception confidence of each aircraft and the perception requests received from other aircraft; Sparse and key feature information is extracted based on the perception confidence of each aircraft and the perception requests received from other aircraft.

Images