Airspace safety guarantee capability assessment method based on reinforcement learning

Abstract

The invention discloses an airspace safety guarantee capability assessment method based on reinforcement learning, and the method comprises the steps: carrying out the analysis of the influence factors of the airspace safety guarantee capability based on an accident tree, and determining the influence factors of the airspace safety guarantee capability; based on the airspace operation unsafe event danger coefficient, carrying out airspace safety guarantee capability clustering analysis, and classifying the events into clustering centers meeting requirements; constructing a GA-BP neural network model based on the airspace security assurance capability influence factors and the airspace security assurance capability clustering result, and training and optimizing the GA-BP neural network model based on a particle swarm algorithm; and inputting the airspace current situation into the trained GA-BP neural network model to obtain an airspace safety guarantee capability evaluation result. According to the method, the airspace safety guarantee capability can be evaluated and the development trend of the airspace can be mastered while the use efficiency of the airspace is improved at present, weak links of airspace management are deeply excavated according to the evaluation result, improvement measures are provided in a targeted manner, and the operation efficiency of the airspace is comprehensively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of airspace management, and more specifically to an airspace safety assurance capability assessment method based on reinforcement learning. Background Art

[0002] Airspace operation efficiency and safety assurance capability are the two core indicators for evaluating airspace management capabilities. With the acceleration of my country's economic, social and national defense development, military aviation, public transport aviation and low-altitude economy have all developed rapidly, and the demand for air has continued to grow. It is urgent to improve the efficiency of airspace use to alleviate the emerging contradictions in air demand. While improving the efficiency of airspace use, it is necessary to ensure airspace safety assurance capabilities in order to ensure safe and efficient operation of air traffic.

[0003] Therefore, how to provide an airspace security assurance capability assessment method based on reinforcement learning is an urgent problem that technicians in this field need to solve. Summary of the invention

[0004] In view of this, the present invention provides an airspace security assurance capability assessment method based on reinforcement learning. While currently focusing on improving the efficiency of airspace utilization, the airspace security assurance capability can be evaluated to grasp its development trend. Based on the assessment results, the weak links in airspace management can be deeply explored, and targeted improvement measures can be proposed to comprehensively improve the airspace operation efficiency.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] A method for evaluating airspace safety assurance capability based on reinforcement learning, including:

[0007] Analyze the factors affecting airspace safety assurance capability based on the accident tree and determine the factors affecting airspace safety assurance capability;

[0008] Conduct cluster analysis of airspace safety assurance capabilities based on the risk factor of airspace operation unsafe events, and classify the events into cluster centers that meet the requirements;

[0009] A GA-BP neural network model was constructed based on the influencing factors of airspace safety assurance capability and the clustering results of airspace safety assurance capability, and the GA-BP neural network model was trained and optimized based on the particle swarm algorithm.

[0010] The current status of the airspace is input into the trained GA-BP neural network model to obtain the airspace safety assurance capability assessment results.

[0011] Preferably, a GA-BP neural network model is constructed based on the influencing factors of airspace safety assurance capability and the clustering results of airspace safety assurance capability, specifically including:

[0012] Determine the topological structure of the GA-BP neural network model, and set the number of nodes in the input layer, hidden layer, and output layer to be L, J, and K respectively. The input layer is determined based on the number of factors affecting the safety assurance capability, and the output layer is determined based on the clustering results of the airspace safety assurance capability.

[0013] Initialize the weights of the GA-BP neural network, the weights and thresholds from the input layer to the hidden layer are w 入隐 and θ 入隐 , the weights and thresholds from the hidden layer to the output layer are w 隐出 and θ 隐出 .

[0014] Preferably, the GA-BP neural network model is trained and optimized based on a particle swarm algorithm, specifically including:

[0015] Step 1: Initialize the particle solution set:

[0016] Set particle solution set o = (w 入隐 ,w 隐出 ,θ 入隐 ,θ 隐出 ), where w 入隐 is a J×L matrix, representing the weights from the input layer to the hidden layer of the GA-BP neural network model, θ 入隐 is a 1×L matrix, representing the threshold from the input layer to the hidden layer of the GA-BP neural network model, w 隐出 is a J×K matrix, representing the weights from the hidden layer to the output layer of the GA-BP neural network model, θ 隐出 is a 1×K matrix, representing the threshold from the hidden layer to the output layer of the GA-BP neural network model; L, J, and K are the number of nodes in the input layer, hidden layer, and output layer of the BP neural network respectively;

[0017] Initialize the speed and position of each particle to obtain a set of weights and thresholds of the GA-BP neural network model corresponding to each particle solution set;

[0018] Step 2 calculates the initial fitness of each particle based on the particle solution set:

[0019]

[0020] Among them, pg p is the airspace safety assurance capability evaluation value of the GA-BP neural network model corresponding to the sample data in the pth training set, sj p is the actual airspace safety assurance capability value corresponding to the pth sample data, p = 1, 2, ... P, P is the total number of sample data in the GA-BP neural network model training set;

[0021] Step 3: Optimize particle speed and position based on initial fitness:

[0022] The particle's local optimal position update formula is:

[0023]

[0024] Among them, f(x i (t+1)) is the fitness value of the particle corresponding to the t+1th time, f(p i (t)) is the fitness value corresponding to the best position experienced by the particle in the previous t iterations, among which the fitness value corresponding to the best position experienced by the particle in the previous t iterations is the smallest, p i (t+1) is the local optimal position of the particle in the previous t+1 iterations, x i (t+1) is the position of the particle at the t+1th iteration, p i (t) is the local optimal position of the particle in the previous t iterations, i = 1, 2, ..., s, s represents the size of the particle swarm;

[0025] The global optimal position in the population is denoted as p best (t), p best (t)∈{p1(t),…,p s (t)}, and f(p best (t))=min{f(p19t)),…,f(p s (t))}, then the global optimal position update formula is as follows:

[0026]

[0027] When f(p best (t))-f(p best (t+1))≤ε1, then output the current global optimal position p best (t), where ε1 is the maximum allowable error, otherwise it returns to the initialization particle solution set and continues the optimization iteration;

[0028] Step 4: Output the global optimal position p based on the particle swarm algorithm best (t) After that, the sample data of the test set is input into the GA-BP neural network model to evaluate the airspace safety assurance capability pg:

[0029] The airspace safety assurance capability pg calculated by evaluation is compared with the actual value sj of the airspace safety assurance capability to determine whether |sj-pg|<ε, where ε is the preset error value; if so, the global optimal position p best (t) The corresponding weights and thresholds meet the requirements, that is, the weights and thresholds corresponding to the GA-BP neural network model;

[0030] Otherwise, at the global optimal position pbest (t), iterative calculation is performed to update the weights and thresholds corresponding to the GA-BP neural network model:

[0031]

[0032] Among them, w 入隐 (n), w 隐出 (n) are the weights from the input layer to the hidden layer and the weights from the hidden layer to the output layer of the GA-BP neural network obtained in the nth iteration, respectively. 入隐 (n), θ 隐出 (n) are the thresholds from the input layer to the hidden layer and from the hidden layer to the output layer of the GA-BP neural network obtained in the nth iteration, respectively. 入隐 (n+1), w 隐出 (n+1) are the weights from the input layer to the hidden layer and the weights from the hidden layer to the output layer of the GA-BP neural network obtained in the n+1th iteration, θ 入隐 (n+1),θ 隐出 (n+1) are the thresholds from the input layer to the hidden layer and from the hidden layer to the output layer of the GA-BP neural network obtained in the n+1th iteration, respectively. The initial value of n is 0, and w 入隐 (0), w 隐出 (0) are the global optimal positions p best (t corresponds to the weights from the input layer to the hidden layer and from the hidden layer to the output layer of the GA-BP neural network, θ 入隐 (0),θ 隐出 (0) are the global optimal positions p best (t) The threshold from the input layer to the hidden layer and the threshold from the hidden layer to the output layer of the corresponding GA-BP neural network, where η is the step size of the GA-BP neural network model, E 总 is the total error generated by all sample data, pg p The airspace security assurance capability evaluation value of the GA-BP neural network model obtained by inputting the p-th sample data into the n-th iteration weight and threshold assignment, sj p is the actual value of the airspace security assurance capability corresponding to the pth sample data, the value of p is 1, 2, ..., P, P is the total number of sample data in the GA-BP neural network model training set, and the weights and thresholds from the input layer to the hidden layer when the neural network is initialized are w 入隐 and θ 入隐 , the weights and thresholds from the hidden layer to the output layer are w 隐出 and θ 隐出 .

[0033] Preferably, the current status of the airspace is input into the trained GA-BP neural network model to obtain the airspace safety assurance capability assessment results, which specifically include:

[0034] The current status of the airspace is used as the input of the trained GA-BP neural network model. The current status of the airspace is described by the safety assurance capability influencing factor L, which is represented as an L-dimensional array Each dimension of data in the array represents an influencing factor;

[0035] Calculate w 入隐 and w 隐出 :

[0036]

[0037] Represents the weight matrix w 入隐 The matrix element at row J and column L, Represents the weight matrix w 隐出 The matrix element at row J and column K;

[0038] Calculate the output Z of the hidden layer:

[0039]

[0040] Among them, σ 入隐 represents the activation function of the neural GA-BP from the network input layer to the hidden layer, z J represents the J-th output of the hidden layer;

[0041] Calculate the output Y of the neural network:

[0042]

[0043] Among them, σ 隐出 represents the activation function from the hidden layer to the output layer of the GA-BP neural network, y K represents the Kth output of the output layer;

[0044] Calculate the airspace safety assurance capability: The airspace safety assurance capability evaluation value is calculated using the array Y = [y1, y2, y3…y K ] means that the array [y1, y2, y3…y K ] is 1, which means the airspace safety assurance capability is at level K.

[0045] Preferably, the factors affecting airspace safety assurance capability are analyzed based on the accident tree to determine the factors affecting airspace operation unsafe events, including:

[0046] Based on the analysis of airspace safety assurance capabilities based on the accident tree, the top event was determined to be an unsafe airspace operation event;

[0047] Determine the A-level intermediate event and the B-level intermediate event based on the top event;

[0048] Basic events are determined based on Level A intermediate events and Level B intermediate events. Basic events are the influencing factors of unsafe airspace operation events.

[0049] Preferably, the Class A intermediate events include ACAS warnings, air traffic control equipment failures, runway incursions / occupancy, less than flight intervals, and near control intervals, and the Class B intermediate events include human factors, machine equipment factors, environmental factors, and management factors;

[0050] Basic events include pilots, flight commanders, traffic controllers, navigators, aircraft onboard equipment failures, cockpit warning system failures, communication equipment failures, navigation equipment failures, surveillance equipment failures, control equipment failures, meteorological equipment failures, weather environment, geographical environment, airspace structure, airspace demarcation, flight traffic characteristics, airport clearance conditions, runway order, navigation aids and facilities, visual navigation runway markings, unidentified flying objects, electromagnetic environment, aircraft management, airspace management and organizational management.

[0051] Preferably, cluster analysis of airspace operation unsafe events is performed based on the risk factor of airspace operation unsafe events, and the events are classified into cluster centers that meet the requirements, specifically including:

[0052] Determine the number of clusters of airspace safety assurance capability events and obtain the airspace safety assurance capability level;

[0053] Allocate airspace safety assurance capability value ranges based on airspace safety assurance capability levels;

[0054] Initialize K cluster centers according to the range of airspace safety assurance capability assignment;

[0055] Based on the risk factor of airspace operation unsafe events, the distance from each airspace operation unsafe event to the cluster center is calculated.

[0056] Preferably, K-means clustering uses the sum of squares of distances between samples and cluster centers as a measure of similarity, and the similarity function is expressed as:

[0057]

[0058] Where E is the sum of square errors of all airspace operation unsafe events, C e is the e-th cluster, d is the risk factor of unsafe events in airspace operations, μ e is the cluster center of the e-th cluster, and K is the airspace safety assurance capability level determined according to the risk factor of airspace unsafe events.

[0059] It can be seen from the above technical solution that, compared with the prior art, the present invention discloses an airspace security assurance capability assessment method based on reinforcement learning, which can accurately assess the airspace security assurance capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0061] Figure 1 A flow chart of an airspace security assurance capability assessment method based on reinforcement learning provided by the present invention.

[0062] Figure 2 This is a processing flow chart of the GA-BP neural network model provided by the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] The embodiment of the present invention discloses a method for evaluating airspace security capabilities based on reinforcement learning, such as Figure 1 As shown, including:

[0065] Step 1: Analyze the main factors affecting airspace safety assurance capabilities based on the accident tree and determine the main causes of unsafe airspace operation events, including:

[0066] (1) Based on the accident tree analysis, the main factors affecting airspace safety assurance capabilities were identified and the top event was determined to be an unsafe airspace operation event.

[0067] (2) Based on the top events, Class A intermediate events and Class B intermediate events are determined. Class A intermediate events are divided into ACAS alerts, air traffic control equipment failures, less than flight interval, runway incursion / occupancy, and adjacent control intervals. Class B intermediate events are divided into human factors, machine equipment factors, environmental factors, and management factors.

[0068] (3) Based on the A-level intermediate events and the B-level intermediate events, L basic events are determined, such as pilots, flight commanders, air traffic controllers, navigators, aircraft onboard equipment failures, cockpit warning system failures, communication equipment failures, navigation equipment failures, monitoring equipment failures, control equipment failures, meteorological equipment failures, weather environment, geographical environment, airspace structure, airspace demarcation, flight flow characteristics, airport clearance conditions, runway order, navigation aids and facilities, visual navigation runway markings, unidentified flying objects, electromagnetic environment, aircraft management, airspace management and organizational management. Basic events are the main causes of airspace operation unsafe events.

[0069] Step 2: Perform cluster analysis of KM’s airspace operation unsafe events based on the airspace operation unsafe event risk factor, including:

[0070] (1) The number of clusters of airspace safety assurance capability events is determined as K, where K is the airspace safety assurance capability level determined according to the risk factor of airspace unsafe events, and K is an integer.

[0071] (2) Based on the standardized airspace operation unsafe events and K airspace safety assurance capability levels, the airspace safety assurance capability is assigned a value range. For example, the value range of the first-level airspace safety assurance capability is [0.6, 1.5], the value range of the second-level airspace safety assurance capability is [1.6, 2.5], the value range of the third-level airspace safety assurance capability is [2.6, 3.5], the value range of the fourth-level airspace safety assurance capability is [3.6, 4.5], and the value range of the K-level airspace safety assurance capability is [K-0.4, K+0.5].

[0072] (3) Initialize K cluster centers according to the value range of airspace safety assurance capability. The root cause of airspace operation unsafe events is assigned a value range according to the category. According to the basic principle of K-means clustering, determine that K cluster centers are μ1 = 1, μ2 = 2, μ3 = 3, …, μ k =K.

[0073] (4) Calculate the distance from each airspace unsafe operation event to the cluster center, and cluster each airspace unsafe operation event with the closest center. K-means clustering uses the sum of the squares of the distances between the sample and the cluster center as the metric of similarity, and the similarity function is expressed as:

[0074]

[0075] Where E is the sum of square errors of all airspace operation unsafe events, C e is the e-th cluster, d is the risk factor of unsafe events in airspace operations, μ e is the cluster center of the e-th cluster, and K is the airspace safety assurance capability level determined according to the risk factor of airspace unsafe events.

[0076] Step 3: Construct a GA-BP neural network model based on the influencing factors of airspace safety assurance capability and the clustering results of airspace safety assurance capability, and train and optimize the GA-BP neural network model based on the particle swarm algorithm, including:

[0077] (1) GA-BP neural network model construction

[0078] Determine the topological structure of the GA-BP neural network model, and set the number of nodes in the input layer, hidden layer, and output layer to be L, J, and K respectively. Initialize the weights of the neural network, and the weights and thresholds from the input layer to the hidden layer to be w 入隐 and θ 入隐 , the weight and threshold from the hidden layer to the output layer are w 隐出 and θ 隐出 ,The number of nodes L in the input layer is determined based on the number of ,factors affecting the airspace safety assurance capability, and the number of nodes K in the output layer is determined ,based on the clustering results of the airspace safety assurance capability.

[0079] (2) GA-BP neural network model training and optimization based on particle swarm algorithm

[0080] Initialize the particle solution set. Set the particle solution set o = (w 入隐 ,w 隐出 ,θ 入隐 ,θ 隐出 ), where w 入隐 is a J×L matrix, representing the weights from the input layer to the hidden layer of the GA-BP neural network model, θ 入隐 is a 1×L matrix, representing the threshold from the input layer to the hidden layer of the GA-BP neural network model, w 隐出 is a J×K matrix, representing the weights from the hidden layer to the output layer of the GA-BP neural network model, θ 隐出 is a 1×K matrix, representing the threshold from the hidden layer to the output layer of the GA-BP neural network model; set the particle swarm size to s, and the maximum number of evolutionary iterations to t max , the speed and position of each particle o are initialized in each iteration to obtain a set of weights and thresholds of the GA-BP neural network model corresponding to each particle solution set.

[0081] Calculate the initial fitness value. Assign a set of weights and thresholds corresponding to the solution set of each particle to the GA-BP neural network model, and calculate the inverse of the training error of the GA-BP neural network model to obtain the fitness value f(x i (t)).

[0082]

[0083] Among them, pg p is the airspace safety assurance capability evaluation value of the GA-BP neural network model corresponding to the sample data in the pth training set, sj p is the actual airspace safety assurance capability value corresponding to the pth sample data, and P is the total number of sample data in the GA-BP neural network model training set;

[0084] Optimize particle speed and position. For each particle, compare its fitness value with the best position p experienced in the previous t iterations. i The fitness value of (t) is compared, and the local optimal position of the particle is updated according to the following formula:

[0085]

[0086] Among them, f(x i (t+1)) is the fitness value of the particle corresponding to the t+1th time, f(p i (t)) is the fitness value corresponding to the best position experienced by the particle in the previous t iterations, among which the fitness value corresponding to the best position experienced by the particle in the previous t iterations is the smallest, p i (t+1) is the local optimal position of the particle in the previous t+1 iterations, x i (t+1 is the position of the particle at the t+1th iteration, i=1,2,...,s, s represents the size of the particle swarm;

[0087] The global optimal position in the population is denoted as p best (t), p best (t)∈{p1(t),…,p s (t)}, and f(p best (t))=min{f(p1(t)),…,f(p s (t))}, then the global optimal position update formula is as follows:

[0088]

[0089] When f(p best (t))-f(p best (t+1))≤ε1, then output the current global optimal position p best (t), where ε1 is the maximum allowable error, otherwise it returns to the initialization particle solution set and continues the optimization iteration.

[0090] Output the global optimal position p based on particle swarm algorithm best(t) After that, the sample data of the test set is input into the GA-BP neural network model to evaluate the airspace safety assurance capability pg; the evaluated airspace safety assurance capability pg is compared with the actual value of the airspace safety assurance capability sj to determine whether |sj-pg|<ε, ε is the preset error value; if so, the global optimal position p best (t) The corresponding weights and thresholds meet the requirements, that is, the weights and thresholds corresponding to the GA-BP neural network model;

[0091] Otherwise, at the global optimal position p best (t), iterative calculation is performed to update the weights and thresholds corresponding to the GA-BP neural network model;

[0092]

[0093] Among them, w 入隐 (n), w 隐出 (n) are the weights from the input layer to the hidden layer and from the hidden layer to the output layer of the GA-BP neural network obtained in the nth iteration during the optimization process of the GA-BP neural network model, θ 入隐 (n), θ 隐出 (n) are the thresholds from the input layer to the hidden layer and from the hidden layer to the output layer of the neural network obtained in the nth iteration during the optimization process of the GA-BP neural network model, respectively. 入隐 (n+1), w 隐出 (n+1) are the weights from the input layer to the hidden layer and the weights from the hidden layer to the output layer of the neural network obtained in the n+1th iteration during the optimization process of the GA-BP neural network model, θ 入隐 (n+1),θ 隐出 (n+1) are the thresholds from the input layer to the hidden layer and from the hidden layer to the output layer of the neural network obtained in the n+1th iteration during the optimization process of the GA-BP neural network model. The initial value of n is 0, and w 入隐 (0), w 隐出 (0 are the global optimal positions p best (t) corresponds to the weights from the input layer to the hidden layer and from the hidden layer to the output layer of the neural network, θ 入隐 (0),θ 隐出 (0) are the global optimal positions p best (t) corresponds to the threshold from the input layer to the hidden layer and the threshold from the hidden layer to the output layer of the neural network, where η is the step size of the neural network model, E 总 is the total error generated by all sample data, pg p The airspace security assurance capability evaluation value of the GA-BP neural network model obtained by inputting the p-th sample data into the n-th iteration weight and threshold assignment, sj pis the actual value of the airspace safety assurance capability corresponding to the pth sample data, the value of p is 1, 2, ..., P, and P is the total number of sample data in the GA-BP neural network model training set.

[0094] Step 4: Input the current status of the airspace into the trained GA-BP neural network model to obtain the airspace safety assurance capability assessment results, such as Figure 2 As shown, specifically including:

[0095] The current status of airspace is used as the input of the GA-BP neural network model. The current status of airspace can be described by L factors that affect the airspace safety assurance capability, expressed as an L-dimensional array Each dimension of data in the array represents an influencing factor, mainly including pilots, flight commanders, traffic controllers, navigators, aircraft onboard equipment failures, cockpit warning system failures, communication equipment failures, navigation equipment failures, monitoring equipment failures, control equipment failures, meteorological equipment failures, weather environment, geographical environment, airspace structure, airspace demarcation, flight traffic, airport clearance conditions, runway order, navigation aids and facilities, visual navigation runway markings, unidentified flying objects, electromagnetic environment, aircraft management, airspace management and organizational management, etc.

[0096] Secondly, after the GA-BP neural network training is completed, the weight w from the input layer to the hidden layer of the GA-BP neural network model is obtained. 入隐 and threshold θ 入隐 , the weight w from the hidden layer to the output layer 隐出 and threshold θ 隐出 , and perform operations according to the structure of the neural network. Among them,

[0097]

[0098] Represents the weight matrix w 入隐 The matrix element at row J and column L, Represents the weight matrix w 隐出 The matrix element at row J and column K.

[0099] Again, calculate the output Z of the hidden layer. The hidden layer output Z can be expressed as:

[0100]

[0101] Among them, σ 入隐 represents the activation function from the input layer to the hidden layer of the neural network, z J represents the J-th output of the hidden layer.

[0102] Then, calculate the output Y of the neural network. The output Y can be expressed as:

[0103]

[0104]

[0105] Among them, σ 隐出 Represents the activation function from the hidden layer to the output layer of the neural network, y K represents the Kth output of the output layer.

[0106] Finally, the airspace safety assurance capability is output. The airspace safety assurance capability evaluation value can be obtained by using the array Y = [y1, y2, y3…y K ] means that the array [y1, y2, y3…y K ] is 1, which means the airspace safety assurance capability is at level K.

[0107] The present invention first sorts out the main influencing factors of airspace safety assurance capability from the aspects of people, machines, environment and management, then clusters the unsafe events of airspace operation in previous years according to the causes, uses reinforcement learning technology to analyze the weight of each influencing factor on the airspace safety assurance capability, and finally inputs the status data of the existing influencing factors of the airspace to evaluate the airspace safety assurance capability.

[0108] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0109] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for evaluating airspace security capabilities based on reinforcement learning, characterized in that: include: Analyze the factors affecting airspace safety assurance capability based on the accident tree and determine the factors affecting airspace safety assurance capability; Conduct cluster analysis of airspace safety assurance capabilities based on the risk factor of airspace operation unsafe events, and classify the events into cluster centers that meet the requirements; A GA-BP neural network model was constructed based on the influencing factors of airspace safety assurance capability and the clustering results of airspace safety assurance capability, and the GA-BP neural network model was trained and optimized based on the particle swarm algorithm. The current status of the airspace is input into the trained GA-BP neural network model to obtain the airspace safety assurance capability assessment results.

2. The method for evaluating airspace security capabilities based on reinforcement learning according to claim 1, characterized in that: Based on the influencing factors of airspace safety assurance capability and the clustering results of airspace safety assurance capability, a GA-BP neural network model is constructed, which includes: Determine the topological structure of the GA-BP neural network model, and set the number of nodes in the input layer, hidden layer, and output layer to be L, J, and K respectively. The input layer is determined based on the number of factors affecting the safety assurance capability, and the output layer is determined based on the clustering results of the airspace safety assurance capability. Initialize the weights of the GA-BP neural network, the weights and thresholds from the input layer to the hidden layer are w 入隐 and θ 入隐 , the weights and thresholds from the hidden layer to the output layer are w 隐出 and θ 隐出 .

3. The method for evaluating airspace security capabilities based on reinforcement learning according to claim 1, characterized in that: The GA-BP neural network model is trained and optimized based on the particle swarm algorithm, including: Step 1: Initialize the particle solution set: Set particle solution set o = (w 入隐 ,w 隐出 ,θ 入隐 ,θ 隐出 ), where w 入隐 is a J×L matrix, representing the weights from the input layer to the hidden layer of the GA-BP neural network model, θ 入隐 is a 1×L matrix, representing the threshold from the input layer to the hidden layer of the GA-BP neural network model, w 隐出 is a J×K matrix, representing the weights from the hidden layer to the output layer of the GA-BP neural network model, θ 隐出 is a 1×K matrix, representing the threshold from the hidden layer to the output layer of the GA-BP neural network model; L, J, and K are the number of nodes in the input layer, hidden layer, and output layer of the BP neural network respectively; Initialize the speed and position of each particle to obtain a set of weights and thresholds of the GA-BP neural network model corresponding to each particle solution set; Step 2 calculates the initial fitness of each particle based on the particle solution set: Among them, pg p is the airspace safety assurance capability evaluation value of the GA-BP neural network model corresponding to the sample data in the pth training set, sj p is the actual airspace safety assurance capability value corresponding to the pth sample data, p = 1, 2, ... P, P is the total number of sample data in the GA-BP neural network model training set; Step 3: Optimize particle speed and position based on initial fitness: The particle's local optimal position update formula is: Among them, f(x i (t+1)) is the fitness value of the particle corresponding to the t+1th time, f(p i (t)) is the fitness value corresponding to the best position experienced by the particle in the previous t iterations, among which the fitness value corresponding to the best position experienced by the particle in the previous t iterations is the smallest, p i (t+1) is the local optimal position of the particle in the previous t+1 iterations, x i (t+1) is the position of the particle at the t+1th iteration, p i (t) is the local optimal position of the particle in the previous t iterations, i = 1, 2, ..., s, s represents the size of the particle swarm; The global optimal position in the population is denoted as p best (t), p best (t)∈{p1(t),…,p s (t)}, and f(p best (t))=min{f(p1(t)),…,f(p s (t))}, then the global optimal position update formula is as follows: When f(p best (t))-f(p best (t+1))≤ε1, then output the current global optimal position p best (t), where ε1 is the maximum allowable error, otherwise it returns to the initialization particle solution set and continues the optimization iteration; Step 4: Output the global optimal position p based on the particle swarm algorithm best (t) After that, the sample data of the test set is input into the GA-BP neural network model to evaluate the airspace safety assurance capability pg: The airspace safety assurance capability pg calculated by evaluation is compared with the actual value sj of the airspace safety assurance capability to determine whether |sj-pg|<ε, where ε is the preset error value; if so, the global optimal position p best (t) The corresponding weights and thresholds meet the requirements, that is, the weights and thresholds corresponding to the GA-BP neural network model; Otherwise, at the global optimal position p best (t), iterative calculation is performed to update the weights and thresholds corresponding to the GA-BP neural network model: Among them, w 入隐 (n), w 隐出 (n) are the weights from the input layer to the hidden layer and the weights from the hidden layer to the output layer of the GA-BP neural network obtained in the nth iteration during the optimization process of the GA-BP neural network model, θ 入隐 (n), θ 隐出 (n) are the thresholds from the input layer to the hidden layer and from the hidden layer to the output layer of the GA-BP neural network obtained in the nth iteration during the optimization process of the GA-BP neural network model, respectively. 入隐 (n+1), w 隐出 (n+1) are the weights from the input layer to the hidden layer and from the hidden layer to the output layer of the GA-BP neural network obtained in the n+1th iteration during the optimization process of the GA-BP neural network model, θ 入隐 (n+1),θ 隐出 (n+1) are the thresholds from the input layer to the hidden layer and from the hidden layer to the output layer of the GA-BP neural network obtained in the n+1th iteration during the optimization of the GA-BP neural network model. The initial value of n is 0, and w 入隐 (0), w 隐出 (0) are the global optimal positions p best (t) The corresponding weights from the input layer to the hidden layer and from the hidden layer to the output layer of the GA-BP neural network, θ 入隐 (0),θ 隐出 (0) are the global optimal positions p best (t) The threshold from the input layer to the hidden layer and the threshold from the hidden layer to the output layer of the corresponding GA-BP neural network, where η is the step size of the GA-BP neural network model, E 总 is the total error generated by all sample data, pg p The airspace security assurance capability evaluation value of the GA-BP neural network model with weights and thresholds assigned after the p-th sample data is input into the n-th iteration, sj p is the actual value of the airspace security assurance capability corresponding to the pth sample data. When the neural network is initialized, the weights and thresholds from the input layer to the hidden layer are w 入隐 and θ 入隐 , the weights and thresholds from the hidden layer to the output layer are w 隐出 and θ 隐出 .

4. The method for evaluating airspace security capabilities based on reinforcement learning according to claim 3 is characterized in that: The current status of the airspace is input into the trained GA-BP neural network model to obtain the airspace safety assurance capability assessment results, including: The current status of the airspace is used as the input of the trained GA-BP neural network model. The current status of the airspace is described by the safety assurance capability influencing factor L, which is represented as an L-dimensional array Each dimension of data in the array represents an influencing factor; Calculate w 入隐 and w 隐出 : Represents the weight matrix w 入隐 The matrix element at row J and column L, Represents the weight matrix w 隐出 The matrix element at row J and column K; Calculate the output Z of the hidden layer: Among them, σ 入隐 represents the activation function of the neural GA-BP from the network input layer to the hidden layer, z J represents the Jth output of the hidden layer; Calculate the output Y of the neural network: Y=σ 隐出 (Zw 隐出 ) Among them, σ 隐出 represents the activation function from the hidden layer to the output layer of the GA-BP neural network, y K represents the Kth output of the output layer; Calculate the airspace safety assurance capability: The airspace safety assurance capability evaluation value is calculated using the array Y = [y1, y2, y3…y K ] means that the array [y1, y2, y3…y K ] is 1, which means the airspace safety assurance capability is at level K.

5. The method for evaluating airspace security capabilities based on reinforcement learning according to claim 1 is characterized in that: Based on the accident tree, the factors affecting airspace safety assurance capability are analyzed to determine the factors affecting airspace operation unsafe events, including: Based on the analysis of airspace safety assurance capabilities based on the accident tree, the top event was determined to be an unsafe airspace operation event; Determine the A-level intermediate event and the B-level intermediate event based on the top event; Basic events are determined based on Level A intermediate events and Level B intermediate events. Basic events are the influencing factors of unsafe airspace operation events.

6. The method for evaluating airspace security capabilities based on reinforcement learning according to claim 5 is characterized in that: Class A intermediate events include ACAS alerts, air traffic control equipment failures, runway incursions / occupancy, less than flight separation, and approaching control separation. Class B intermediate events include human factors, machine equipment factors, environmental factors, and management factors. Basic events include pilots, flight commanders, traffic controllers, navigators, aircraft onboard equipment failures, cockpit warning system failures, communication equipment failures, navigation equipment failures, surveillance equipment failures, control equipment failures, meteorological equipment failures, weather environment, geographical environment, airspace structure, airspace demarcation, flight traffic characteristics, airport clearance conditions, runway order, navigation aids and facilities, visual navigation runway markings, unidentified flying objects, electromagnetic environment, aircraft management, airspace management and organizational management.

7. The method for evaluating airspace security capabilities based on reinforcement learning according to claim 1, characterized in that: Based on the risk factor of airspace operation unsafe events, cluster analysis of airspace operation unsafe events is carried out, and the events are classified into cluster centers that meet the requirements, including: Determine the number of clusters of airspace safety assurance capability events and obtain the airspace safety assurance capability level; Allocate airspace safety assurance capability value ranges based on airspace safety assurance capability levels; Initialize K cluster centers according to the range of airspace safety assurance capability assignment; Based on the risk factor of airspace operation unsafe events, the distance from each airspace operation unsafe event to the cluster center is calculated.

8. The method for evaluating airspace security capabilities based on reinforcement learning according to claim 7 is characterized in that: K-means clustering uses the sum of squared distances between samples and cluster centers as a measure of similarity. The similarity function is expressed as: Where E is the sum of square errors of all airspace operation unsafe events, C e is the e-th cluster, d is the risk factor of unsafe events in airspace operations, μ e is the cluster center of the e-th cluster, and K is the airspace safety assurance capability level determined according to the risk factor of airspace unsafe events.

Images