Multi-aircraft airborne weather collaborative awareness method

Abstract

The present application relates to a kind of multi-aircraft airborne weather collaborative sensing method, belong to the technical field of air traffic management, solve the problem of low precision, small range of existing technology in airborne weather situation awareness.The multi-aircraft airborne weather collaborative sensing method of the present application, for the multi-machine situation intelligent collaborative sensing of autonomous operation, by combining aircraft radar echo data, weather information and flight intention, constructs the multi-aircraft sensing network of dynamic collaborative in region, realizes the efficient transmission of situation characteristic information, improves the range and reliability of trusted sensing, improves the robustness of air traffic system, guarantees flight safety.

Description

Technical Field

[0001] The present invention relates to the technical field of aviation traffic management, and specifically to a method for collaborative airborne weather perception by multiple aircraft. Background Art

[0002] With the continued development of the civil aviation industry, flight and passenger traffic volumes have increased annually. This rapid growth in air traffic has increased the complexity of air traffic operations, making the current centralized separation control model incapable of meeting the requirements for safe operations in highly complex airspace environments. Many existing trunk airway networks are exceeding capacity limits, leading to frequent flight delays during peak periods and placing immense pressure on air traffic management. The traditional centralized, ground-based control-centered approach is increasingly unable to meet the operational demands of this highly complex airspace environment.

[0003] To address this challenge, the Civil Aviation Administration of China (CAAC) has proposed the concept of "Autonomous Air Traffic Operations" to enhance navigation flexibility and safety. Airborne situational awareness is the primary prerequisite for autonomous air traffic operations. Reliable and accurate airspace situational information is the foundation for subsequent tasks such as trajectory decisions and conflict resolution. Currently, civil aviation airborne situational awareness primarily utilizes two approaches: 1) Traditional airborne situational awareness, which relies on individual aircraft sensors and instruments, resulting in limited sensing range and capabilities; and 2) Uploaded data from ground-based surveillance systems, which suffer from long detection ranges on ground-based detection base stations and limited real-time and dynamic sensing.

[0004] To overcome the above technical difficulties, multi-aircraft situational collaborative perception provides a new solution for situational awareness in future autonomous operation scenarios. The purpose is to share situational information of multiple aircraft in the region, improve the range and credibility of trusted perception, and improve the robustness of the air traffic system.

[0005] However, current research on airborne collaborative sensing technology is still in its infancy, facing challenges such as the heterogeneity and complexity of multi-source data fusion and limited communication resources. Therefore, to address the shortcomings of current technology, it is urgent to provide an intelligent collaborative sensing method for multi-aircraft situational awareness for autonomous operation. Summary of the Invention

[0006] In view of the above problems, the present invention provides a method for collaborative airborne meteorological perception by multiple aircraft. The method is aimed at intelligent collaborative perception of autonomous multi-aircraft situations. By combining aircraft radar echo data, meteorological information and flight intentions, a dynamic collaborative multi-aircraft perception network within the region is constructed to achieve efficient transmission of situation feature information, improve the trusted perception range and credibility, enhance the robustness of the air traffic system, and ensure flight safety.

[0007] The present invention discloses a multi-aircraft airborne weather collaborative perception method, comprising the following steps:

[0008] Step 1. Collect the aircraft's spatial position, motion data, weather, weather situation observation images, static information, and airport information using the aircraft's weather radar, GPS, inertial navigation system, and flight management system; and use the aircraft's spatial position, motion data, weather, weather situation observation images, static information, and airport information as a training set.

[0009] Step 2. Based on the goal of the collaborative sensing network and under the limitation of the total communication bandwidth, determine the maximum image fusion performance function of the multi-aircraft image fusion model;

[0010] Step 3. Based on the training set from step 1 and the maximized image fusion performance function from step 2, a multi-aircraft image fusion model is obtained by combining spatiotemporal confidence, aircraft flight intentions, a sparsely connected bidirectional communication network, and situational information fusion capabilities.

[0011] Among them, the spatiotemporal confidence is obtained through the spatiotemporal confidence detection module, which is used to evaluate the confidence information in the spatial and temporal dimensions of the meteorological situation observation picture;

[0012] The aircraft's flight intention module inputs aircraft position, motion data, weather and static information to obtain the aircraft's flight intention for the area it is expected to arrive at in the future.

[0013] Sparsely connected two-way communication network for transmitting sparse and critical characteristic information between aircraft;

[0014] The situation information fusion module obtains the situation information fusion function, which is used to fuse the spatiotemporal confidence and the flight intention information of aircraft based on the directed graph of the collaborative relationship between aircraft in a sparsely connected two-way communication network;

[0015] Step 4. Obtain the multi-aircraft airborne weather collaborative perception result image based on the multi-aircraft image fusion model.

[0016] Optionally, the weather is precipitation, hail, thunderstorm, and turbulence.

[0017] Optionally, the motion data includes aircraft speed, heading angle, and aircraft flight plan.

[0018] Optionally, the static information includes the latitude and longitude positions of the waypoints and airport information.

[0019] Optionally, the expression of the total communication bandwidth B in step 1 is:

[0020]

[0021] Where, is the binary selection matrix in space of the information transmitted from the i-th aircraft to the j-th aircraft in the k-th round of communication at time t, It represents the total number of grids in which the i-th aircraft transmits information to the j-th aircraft in the k-th round of communication at time t; D represents the dimension of the feature channel; float represents the precision coefficient; and K represents the total number of communication rounds.

[0022] Optionally, the objective of the collaborative sensing network in step 1 is expressed as:

[0023]

[0024] Where, represents the information transmitted by the j-th aircraft to the i-th aircraft in the k-th round of communication at time t; K represents the total number of communication rounds; N represents the total number of aircraft.

[0025] Optionally, the objective function for maximizing image fusion performance is expressed as:

[0026]

[0027] Where θ represents the parameters to be trained in the image fusion model; e(·) is the image fusion evaluation index; Φ θ (·) is the multi-aircraft image fusion model; represents the observation picture of the surrounding weather situation by the i-th aircraft at time t; Represents the true perception image of the surrounding weather situation of the i-th aircraft at time t.

[0028] Optionally, the spatiotemporal confidence detection module in step 3 includes a feature encoding submodule, a spatial confidence map generator submodule and a spatiotemporal confidence detection module;

[0029] The feature encoding submodule is used to extract high-dimensional features from the aircraft's observation images of the surrounding meteorological situation and obtain feature maps in the aircraft's communication;

[0030] The feature map is obtained by convolution of the RGB image. Through multiple layers of convolution, the edge, texture, geometry and intensity features of the weather in the predicted image are extracted;

[0031] A spatial confidence map generator submodule is used to generate a spatial confidence map based on the characteristic map of each round of communication of each aircraft;

[0032] The spatiotemporal confidence detection module uses an exponential decay function to simulate the time decay of temporal confidence.

[0033] Optionally, the aircraft's flight intention module in step 3 obtains the aircraft's flight intention based on a spatial likelihood factor and a temporal likelihood factor.

[0034] Optionally, the sparsely connected bidirectional communication network in step 3 is used to transmit sparse and critical feature information in the following steps:

[0035] Confirm the perception confidence of each aircraft and the perception requests received from other aircraft;

[0036] Sparse and key feature information is extracted based on the perception confidence of each aircraft and the perception requests received from other aircraft.

[0037] Compared with the prior art, the present invention has at least the following beneficial effects:

[0038] 1) Improve the ability to perceive severe weather:

[0039] This invention uses meteorological collaborative perception technology and data sharing and information fusion among multiple aircraft to make up for the insufficient perception of a single aircraft due to radar blind spots, distance attenuation and other reasons. Especially in complex weather conditions such as severe thunderstorms and low visibility, it significantly improves the situational awareness capability of high-risk areas.

[0040] 2) Optimize communication bandwidth utilization:

[0041] By building a sparse communication network and a multi-head attention mechanism, this method dynamically selects key areas for situational information sharing, reducing unnecessary data transmission. While ensuring perception accuracy, it significantly reduces communication bandwidth usage, improves information sharing efficiency, and improves the overall robustness of the system.

[0042] 3) Enhance the system's dynamic adaptability:

[0043] The present invention has designed a time confidence assessment module and a flight intention analysis module, which can dynamically evaluate data reliability based on historical data, and predict the areas that may be reached in the future in combination with the aircraft's flight plan, thereby achieving rapid response and efficient coordination to short-term weather changes.

[0044] 4) Improve the accuracy and fusion quality of perception data:

[0045] The present invention utilizes a multimodal data fusion algorithm and a collaborative perception model based on a multi-head attention mechanism to efficiently fuse meteorological observation data from multiple aircraft to generate a more comprehensive situation map, significantly improving the accuracy, completeness and spatial resolution of meteorological information.

[0046] 5) Meeting the needs of complex airspace operations:

[0047] The feasibility of the proposed method was verified through simulation. The results show that the method can effectively ensure flight safety under conditions of complex airspace, high aircraft density, and drastic weather dynamics, and can meet the actual needs of collaborative sensing in civil aviation operations.

[0048] 6) Support multi-round collaborative perception:

[0049] The collaborative perception framework proposed in this invention can support multi-round collaboration under limited communication resources. Experiments have shown that multi-round collaboration has significant advantages in improving perception effects. Especially in the case of limited bandwidth, multi-round communication can significantly improve system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The figure is a flow chart of the multi-aircraft airborne weather collaborative perception method of the present invention.

[0051] Figure 2 This is a flowchart of step 3 of the multi-aircraft airborne weather collaborative perception method of the present invention.

[0052] Figure 3 This is a flow chart of step 331 of the multi-aircraft airborne weather collaborative sensing method of the present invention.

[0053] Figure 4 This is a flow chart of step 332 of the multi-aircraft airborne weather collaborative sensing method of the present invention.

[0054] Figure 5 The meteorological situation observation pictures are input into the multi-aircraft image fusion model of the present invention.

[0055] Figure 6 This is a picture of the image fusion result obtained by the multi-aircraft airborne meteorological collaborative perception method of the present invention. DETAILED DESCRIPTION

[0056] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. In addition, the present invention can also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited by the specific embodiments disclosed below.

[0057] A specific embodiment of the present invention, as Figures 1-6 , discloses a multi-aircraft airborne weather collaborative perception method, comprising the following steps:

[0058] Step 1. Collect the aircraft's spatial position, motion data, weather, weather situation observation pictures, static information, and airport information through the weather radar, GPS, inertial navigation system, and flight management system installed on the aircraft; and use the aircraft's spatial position, motion data, weather, weather situation observation pictures, static information, and airport information as a training set.

[0059] Specifically, motion data includes aircraft speed and flight plan. Meteorological situation observation images are weather-related images displayed within weather radar observation images. These images contain red, yellow, and green, representing the intensity of radar echoes within the observed area (i.e., the route). Red is the strongest, yellow is the second strongest, and green is the weakest. Areas without color represent no echoes. A stronger echo indicates a greater likelihood of precipitation, hail, thunderstorms, or turbulence in that area. Meteorological situation (i.e., weather) includes meteorological phenomena such as precipitation, hail, thunderstorms, or turbulence. Static information includes the longitude and latitude of waypoints. Airport information includes runway location, number, length, airport name, and longitude and latitude.

[0060] Exemplarily, the weather situation observation picture is an airborne weather radar echo data observation picture in the form of an RGB picture from a bird's-eye view (BEV) of the surrounding weather situation.

[0061] Step 2. Based on the goal of the cooperative sensing network, under the limitation of the total communication bandwidth, determine the maximized image fusion performance function of the multi-aircraft image fusion model.

[0062] Furthermore, the total communication bandwidth B is expressed as:

[0063]

[0064] Where, is the binary selection matrix in space of the information transmitted from the i-th aircraft to the j-th aircraft in the k-th round of communication at time t, It represents the total number of grids in which the i-th aircraft transmits information to the j-th aircraft in the k-th round of communication at time t; D represents the dimension of the feature channel; float represents the precision coefficient; and K represents the total number of communication rounds.

[0065] Furthermore, float=32 / 8.

[0066] For example, the binary selection matrix The element values ​​include 1 and 0. The position with a value of 1 represents that in the k-th round of communication at time t, the i-th aircraft needs to transmit the corresponding feature information of the position to the j-th aircraft, and the position with a value of 0 represents that no transmission is required.

[0067] Furthermore, the objective of the collaborative sensing network is expressed as:

[0068]

[0069] Where, represents the information transmitted by the j-th aircraft to the i-th aircraft in the k-th round of communication at time t; K represents the total number of communication rounds; N represents the total number of aircraft.

[0070] Furthermore, the expression of the objective function to maximize the image fusion performance is:

[0071]

[0072] Where θ represents the parameters to be trained in the image fusion model; e(·) is the image fusion evaluation index; Φ θ (·) is a multi-aircraft image fusion model based on convolutional neural networks and a multi-head attention mechanism; represents the observation picture of the surrounding weather situation by the i-th aircraft at time t; Represents the true perception image of the surrounding weather situation of the i-th aircraft at time t.

[0073] Understandably, the message The sparse matrix information obtained after processing the observed image; the true value perception image of the weather situation The RGB radar echo image serves as the true value for comparison of the image fusion feature map obtained by the multi-aircraft image fusion model during training, so that the image fusion feature map obtained by the multi-aircraft image fusion model can continuously approach the true value during continuous training until convergence is achieved. Maximizing image fusion performance means that the established image fusion model can occupy as little bandwidth as possible to achieve the best image fusion performance. Image fusion performance refers to the degree of fusion of various parameters and indicators of the image finally output by the image fusion model, including detection range, detection accuracy, distance, communication bandwidth, and the presence of radar shadow areas. For example, by improving the aircraft's perception of the airspace meteorological situation through collaborative perception, the aircraft can obtain clearer information on dangerous weather conditions such as thunderstorms, precipitation, hail, etc. on the route ahead, thereby obtaining the optimal image fusion result.

[0074] Step 3. Based on the training set of step 1 and the maximized image fusion performance function of step 2, a multi-aircraft image fusion model is obtained by combining spatiotemporal confidence, aircraft flight intention, sparsely connected two-way communication network and situation information fusion function.

[0075] Among them, the training parameters θ of the multi-aircraft image fusion model are composed of the training parameters in each submodule, including the image encoder Φ in the spatiotemporal confidence detection module. enc (.), spatial confidence map generation function Φ generator (.) and the selection function Φ select (.), and the situation fusion module's Φ FFN (.).

[0076] Step 31. Construct a spatiotemporal confidence detection module to obtain spatiotemporal confidence, which is used to comprehensively evaluate the confidence information in the spatial and temporal dimensions of the observation image. The input is the observation image of the aircraft on the surrounding meteorological situation at the current moment and the preset past time period.

[0077] Step 311: Construct a feature encoding submodule for extracting high-dimensional features from the aircraft's observation images of the surrounding meteorological situation and obtaining a feature map of the aircraft's communication.

[0078] The present invention extracts high-dimensional features from the initial aircraft's observation pictures of the surrounding meteorological situation, which can support the subsequent calculation of confidence.

[0079] When k = 0, the observation picture of the weather situation by the i-th aircraft at time t is encoded and mapped into the characteristic map of the initial round of communication of the aircraft at time t. Expressed as:

[0080]

[0081] in, represents the characteristic graph of the i-th aircraft in the initial round of communication at time t; Φ enc (.) represents the image encoder, which consists of two convolutional layers and converts the RGB image into a vector of shape 16*H*W; represents a real number; H, W, and D represent the length, width, and number of channels of the feature map, respectively.

[0082] When k>0, the fusion feature map of the i-th aircraft in the previous round of communication at time t is output by the multi-aircraft image fusion model. As the feature map of the i-th aircraft in the current round of communication at time t, exemplarily, in the case of multiple rounds of fusion, the fused feature map output by the first round fusion module is the feature map of the second round of communication.

[0083] Preferably, the characteristic graph of communication is a 16-channel characteristic graph.

[0084] It can be understood that the feature map is obtained by convolution of the RGB image. Through multi-layer convolution, complex features such as edges, textures, geometric shapes, thunderstorm intensity, precipitation intensity, etc. in the predicted image are extracted.

[0085] Step 312: Construct a spatial confidence map generator submodule to generate a spatial confidence map based on the characteristic map of each aircraft, reflecting its perception ability of the surrounding airspace weather.

[0086] Furthermore, the spatial confidence map generator submodule is a convolutional layer with 16 input channels and 1 output channel, a convolution kernel size of 1, and the output value is mapped to the 0-1 range through the sigmoid function.

[0087] Furthermore, the spatial confidence of the i-th aircraft in the k-th round of communication at time t is The expression is:

[0088]

[0089] Among them, Φ generator (.) represents the spatial confidence map generation function, which is used to convert the characteristic map of the i-th aircraft in the k-th round of communication at time t into Converted to a spatial confidence map with a value range of 0-1 It consists of one convolutional layer; It represents the characteristic graph of the i-th aircraft in the k-th round of communication at time t.

[0090] Furthermore, the spatial confidence of each aircraft in each round of communication at each moment is facilitated to obtain a spatial confidence map.

[0091] Step 313. Construct a spatiotemporal confidence detection module, use an exponential decay function to simulate the time decay of the time confidence, and obtain the time confidence of the i-th aircraft in the k-th round of communication at time t before the preset past time period Δt. The expression is:

[0092]

[0093] Wherein, e represents a natural constant; λ represents an attenuation coefficient; and Δt represents a preset past time period (e.g., the past 30 minutes).

[0094] Step 32. Construct an aircraft flight intention module to obtain the aircraft flight intention. Input the aircraft position, motion data, weather and static information. Based on the aircraft's flight dynamics model, status information and airspace environment data, generate the aircraft flight intention that represents the probability distribution of the area that the aircraft may reach in a certain period of time in the future. The higher the probability value, the greater the possibility of reaching the area.

[0095] Step 321. Obtain the spatial likelihood factor κ of the i-th aircraft i,1 , the expression is:

[0096]

[0097] in, represents normal distribution; ψ i,r (t) represents the angle of the direction of the i-th aircraft toward the target waypoint at time t; ψ i,ac (t) represents the heading angle of the i-th aircraft at time t; the heading angle ψ i,ac Angle ψ with the direction towards the target waypoint i,rThe difference obeys a normal distribution with mean 0 and standard deviation h = 5°.

[0098] Further, the spatial position and motion data of the aircraft are collected by the GPS / other navigation system (such as inertial navigation system INS, flight management system FMS) of the aircraft, and the angle of the direction of the i-th aircraft towards the target waypoint and the heading angle at time t are obtained from the spatial position and motion data of the aircraft.

[0099] Further, the standard deviation The normal distribution 3σ principle is adopted, that is: 3σ h The range is ± 15°, which means that the heading angle of the aircraft has a 99.74% probability of being distributed in the range of ψ r ± 15°.

[0100] It can be understood that the spatial likelihood factor κ i,1 represents the spatial information and is used for state estimation of the aircraft.

[0101] Step 322. Obtain the time likelihood factor κ i,2 of the i-th aircraft, expressed as:

[0102]

[0103] wherein, represents a normal distribution; TTG i (WP r ) represents the arrival time WP r of the i-th aircraft to the target waypoint. p is a normal distribution with mean t fp and standard deviation σ i,2 .

[0104] It can be understood that the time likelihood factor κ i,2 represents the time information and is used for estimation of the arrival time of the target waypoint.

[0105] Step 323. Obtain the flight intent of the i-th aircraft at time t according to the joint probability distribution of the likelihood function expressed as:

[0106]

[0107] It can be understood that the input of the flight intent module includes position, speed, flight plan, weather and static information. Combined with the flight dynamics information, state information and airspace environment information of the aircraft, the flight intent of the region that the aircraft is likely to reach in a future period of time is obtained. The greater the flight intent value is, the more likely the aircraft is to reach the region, and in subsequent cooperative communication, the aircraft will pay more attention to the region.

[0108] Step 33. Build a sparsely connected bidirectional communication network to transmit sparse and critical feature information, thereby reducing the air-to-air communication bandwidth without affecting the perception accuracy.

[0109] Step 331: Confirm the perception confidence of each aircraft and the perception requests received from other aircraft.

[0110] Furthermore, based on the output of the spatiotemporal confidence generation module and the aircraft's flight intention module, the spatial confidence map Temporal confidence map and aircraft flight intentions Get the perception request of aircraft i in the kth round of communication at time t The expression is:

[0111]

[0112] Where, is the perception confidence of the i-th aircraft in the k-th round of communication in the preset time period t0~t, is the flight intention of the i-th aircraft at time t, is the softmax function; α represents an adjustable parameter.

[0113] It can be understood that the perception request is used to reflect the need for coordination of the aircraft's position on that day. The higher the value of the perception request, the more the area needs enhanced perception and the more coordination is needed to supplement perception information.

[0114] Furthermore, the expression of perception confidence is:

[0115]

[0116] in, represents the spatial confidence map of the i-th aircraft in the k-th round of communication during the preset time period t0~t, Represents the time confidence graph of the i-th aircraft in the k-th round of communication in the preset time period t0~t:

[0117] Specifically, The time points corresponding to the preset time period t0~t Splicing composition; Similarly, it refers to the time confidence corresponding to multiple time points in a time period t0~t Splicing composition; t0~t means that the radar echo RGB information within this time period is used as the input of the network.

[0118] Among them, t0~t are not continuous, there will be time intervals, and the time interval of the data is 6 minutes (related to the acquisition resolution of the weather radar).

[0119] Step 332: Extract key feature information based on the perception confidence of each aircraft and the perception requests received from other aircraft.

[0120] Step 3321. Obtain a binary selection matrix based on the perception confidence and the perception request The expression is:

[0121]

[0122] Where e represents the element-by-element multiplication of the matrix, is the sensing request of aircraft i in the k-1th round of communication at time t, B is the communication bandwidth; Φ select (.) is the selection function, which is used to select the most critical area for transmission based on the input perception confidence map and perception request under a given communication bandwidth.

[0123] Specifically, the value of the element in the binary selection matrix represents whether the area is selected, 1 means that the information of the area needs to be transmitted, and 0 means that it does not need to be transmitted; the most critical areas are dangerous areas such as thunderstorms, precipitation, and radar shadow areas.

[0124] Step 3322. Extract the information transmitted from the jth aircraft to the ith aircraft in the kth communication at time t according to the binary selection matrix. (i.e. key feature), the expression is:

[0125]

[0126] Where, represents the characteristic graph of the j-th aircraft in the k-round communication at time t, Represents the binary selection matrix for transmitting information from the i-th aircraft to the j-th aircraft in the k-th communication round at time t.

[0127] Step 34. Construct a directed graph of collaborative relationships between aircraft. The nodes in the directed graph represent aircraft, and the edges represent information exchange between aircraft.

[0128] Specifically, the directed graph of the cooperative relationship between aircraft is the adjacency matrix A in the kth round of communication at time t: t,k , the adjacency matrix A t,k It consists of adjacent aircraft, where the i-th aircraft and the j-th aircraft are a pair of adjacent aircraft.

[0129] The adjacent aircraft consisting of the i-th aircraft and the j-th aircraft in the k-th round of communication at time t The expression is:

[0130]

[0131] Step 35. Based on the Transformer structure of the multi-head attention mechanism and the multi-aircraft collaborative relationship diagram, a situation information fusion module is constructed to obtain the situation information fusion function (integrating the information of the spatiotemporal confidence detection module and the aircraft's flight intention module). The expression is:

[0132]

[0133] in, represents the situation information fusion function under the attention weight of the information transmitted by aircraft j to aircraft i in the k-th round of collaboration at time t; d Z express dimension.

[0134] Step 36. Combine the spatiotemporal confidence, the aircraft's flight intention, the sparsely connected two-way communication network, and the situation information fusion function to obtain the fusion feature map of the i-th aircraft in the k-th round of communication at time t output by the multi-aircraft image fusion model. Adaptively fuse information from different aircraft to obtain the fusion results for the next round of communication, so as to improve the breadth and accuracy of aircraft's perception of the airspace meteorological situation.

[0135] Furthermore, the fusion feature map of the i-th aircraft in the k-th round of communication at time t is output by the multi-aircraft image fusion model The expression is:

[0136]

[0137] Where, Φ FFN It is a feedforward network, which performs nonlinear transformation on the weighted feature information and outputs the reconstructed features. i Represents the adjacency matrix A t,k The neighbor aircraft of the i-th aircraft in .

[0138] Step 37. Determine whether the current communication round k is greater than or equal to K. If so, obtain a multi-aircraft image fusion model. If less than K, set k=k+1 and return to step 31.

[0139] Step 4. Obtain the multi-aircraft airborne weather collaborative perception result image based on the multi-aircraft image fusion model.

[0140] Specifically, a decoder is constructed, which consists of two 3x3 convolutional layers (filled with 1) and a ReLU activation layer. The fusion feature map obtained by the multi-aircraft image fusion model is transformed into Convert to RGB image The final output of multi-aircraft airborne weather collaborative perception is expressed as:

[0141]

[0142] Where, Φ dec It is a decoder consisting of two convolutional layers, fusing feature maps 16-channel features; RGB image For 3 channels.

[0143] See also Figure 5 and 6 , Figure 5 This is a weather observation image fed into the multi-aircraft image fusion model. It is a scanned image from the aircraft's onboard weather radar. Pixel color represents the radar echo intensity in that area; areas without color represent no echo. Stronger echoes indicate greater precipitation, hail, or thunderstorm intensity. The fan-shaped notch in the upper left corner of the radar echo represents a radar shadow area, where aircraft cannot obtain weather information, creating a dangerous situation. Figure 6 This is the image fusion result picture obtained. It can be seen that the detection range is larger, the accuracy is higher, and the information of the radar shadow area is supplemented, which greatly improves flight safety.

[0144] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A method for collaborative airborne weather perception by multiple aircraft, characterized in that: The following steps are involved: Step 1. Collect the aircraft's spatial position, motion data, weather, weather situation observation images, static information, and airport information using the aircraft's weather radar, GPS, inertial navigation system, and flight management system; and use the aircraft's spatial position, motion data, weather, weather situation observation images, static information, and airport information as a training set. Step 2. Based on the goal of the collaborative sensing network and under the limitation of the total communication bandwidth, determine the maximum image fusion performance function of the multi-aircraft image fusion model; Step 3. Based on the training set from step 1 and the maximized image fusion performance function from step 2, a multi-aircraft image fusion model is obtained by combining spatiotemporal confidence, aircraft flight intentions, a sparsely connected bidirectional communication network, and situational information fusion capabilities. Among them, the spatiotemporal confidence is obtained through the spatiotemporal confidence detection module, which is used to evaluate the confidence information in the spatial and temporal dimensions of the meteorological situation observation picture; The aircraft's flight intention module inputs aircraft position, motion data, weather and static information to obtain the aircraft's flight intention for the area it is expected to arrive at in the future. Sparsely connected two-way communication network for transmitting sparse and critical characteristic information between aircraft; The situation information fusion module obtains the situation information fusion function, which is used to fuse the spatiotemporal confidence and the flight intention information of aircraft based on the directed graph of the collaborative relationship between aircraft in a sparsely connected two-way communication network; Step 4. Obtain a multi-aircraft airborne weather collaborative perception result image based on the multi-aircraft image fusion model; Among them, the expression of the total communication bandwidth B is: Where, is the binary selection matrix in space of the information transmitted from the i-th aircraft to the j-th aircraft in the k-th round of communication at time t, represents the total number of grids in which the i-th aircraft transmits information to the j-th aircraft in the k-th round of communication at time t; D represents the dimension of the feature channel; float represents the precision coefficient; K represents the total number of communication rounds; The expression of the goal of the collaborative sensing network is: Where, represents the information transmitted by the jth aircraft to the ith aircraft in the kth round of communication at time t; K represents the total number of communication rounds; N represents the total number of aircraft; The expression of the objective function to maximize the image fusion performance is: Where θ represents the parameters to be trained in the image fusion model; e(·) is the image fusion evaluation index; Φ θ (·) is the multi-aircraft image fusion model; represents the observation picture of the surrounding weather situation by the i-th aircraft at time t; Represents the true perception image of the surrounding weather situation of the i-th aircraft at time t.

2. The multi-aircraft airborne weather collaborative sensing method according to claim 1, characterized in that: Weather conditions include precipitation, hail, thunderstorms and turbulence.

3. The multi-aircraft airborne weather collaborative sensing method according to claim 1, characterized in that: Motion data includes aircraft speed, heading angle, and aircraft flight plan.

4. The multi-aircraft airborne weather collaborative sensing method according to claim 1, characterized in that: Static information includes the latitude and longitude positions of waypoints and airport information.

5. The multi-aircraft airborne weather collaborative sensing method according to claim 1, characterized in that: The spatiotemporal confidence detection module in step 3 includes a feature encoding submodule, a spatial confidence map generator submodule, and a spatiotemporal confidence detection module; The feature encoding submodule is used to extract high-dimensional features from the aircraft's observation images of the surrounding meteorological situation and obtain feature maps in the aircraft's communication; The feature map is obtained by convolution of the RGB image. Through multiple layers of convolution, the edge, texture, geometry and intensity features of the weather in the predicted image are extracted; A spatial confidence map generator submodule is used to generate a spatial confidence map based on the characteristic map of each round of communication of each aircraft; The spatiotemporal confidence detection module uses an exponential decay function to simulate the time decay of temporal confidence.

6. The method for collaborative weather perception onboard multiple aircraft according to claim 1, characterized in that: The aircraft's flight intention module in step 3 obtains the aircraft's flight intention based on the spatial likelihood factor and the temporal likelihood factor.

7. The multi-aircraft airborne weather collaborative sensing method according to claim 1, characterized in that: The sparsely connected bidirectional communication network in step 3 is used to transmit sparse and critical feature information in the following steps: Confirm the perception confidence of each aircraft and the perception requests received from other aircraft; Sparse and key feature information is extracted based on the perception confidence of each aircraft and the perception requests received from other aircraft.

Images