Air traffic control system network attack path optimization method and system

Abstract

The invention relates to the technical field of network security, in particular to an air traffic control system network attack path optimization method and system, and the method comprises the steps: determining a network attack target host node, and obtaining a directed path and path cost between host nodes according to node network and vulnerability information in an air traffic control system network threat knowledge graph. A bidirectional ant colony algorithm is adopted, the algorithm comprises a forward ant colony and a reverse ant colony, the forward ant colony starts from a starting host node, the reverse ant colony starts from a target host node at the same time, the pheromone concentration of each path is updated through a PPO algorithm according to behaviors and rewards of the forward ant colony and the reverse ant colony, and the next action of the ant colony is updated accordingly; and the optimal attack path from the start to the target host node is obtained.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and particularly to a method and system for optimizing network attack paths in an air traffic control system. Background Art

[0002] Penetration Testing is a method for evaluating the security of computer network systems by simulating hacker attacks. Attack path planning is crucial in this process because it determines the efficiency and success rate of the test. In a complex network environment, attack path planning can help penetration testers systematically identify and analyze potential attack paths, thereby more effectively discovering and exploiting security vulnerabilities in the system.

[0003] In the field of attack path planning for penetration testing, the main issues studied are how to convert network information into knowledge information and how to combine the target network with path planning algorithms. For the various optional vulnerability exploitation actions and the complex state of target network hosts and their services in the air traffic control system, the traditional ant colony algorithm has applicability problems in network attack path planning. The traditional ant colony algorithm gradually optimizes the path through multiple iterations. In network attack path planning, time is a key factor, and the convergence speed of the traditional ant colony algorithm is relatively slow, resulting in low attack efficiency. Moreover, the traditional ant colony algorithm is prone to falling into local optimal solutions during the search process, especially in a complex network environment, where the diversity of path selection is insufficient, making it difficult for the algorithm to find the global optimal path. In addition, the search process of the traditional ant colony algorithm is unidirectional, resulting in a slow search speed and low algorithm efficiency. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for optimizing network attack paths in an air traffic control system to solve the technical problems of low efficiency and slow search speed of the traditional ant colony algorithm in attack path planning.

[0005] The object of the present invention is achieved by the following technical solutions: In a first aspect, the present invention provides a method for optimizing network attack paths in an air traffic control system, including: Determine the target host nodes of network attacks, and obtain the directed paths between host nodes and the corresponding attack costs according to the node network information and vulnerability information in the network threat knowledge graph of the air traffic control system; Taking the minimum attack cost as the path planning goal, use the bidirectional ant colony algorithm to plan the attack paths of the network threat knowledge graph of the air traffic control system. The ant colonies in the bidirectional ant colony algorithm include a forward ant colony and a reverse ant colony; the forward ant colony starts from the starting host node, and the reverse ant colony starts from the target host node simultaneously; According to the behaviors and rewards of the forward ant colony and the reverse ant colony, use the PPO algorithm to update the pheromone concentration on each path, and update the next action of the ant colony according to the pheromone concentration until the optimal attack path from the starting host node to the target host node is obtained.

[0006] As a further improvement of the present invention, a bidirectional ant colony algorithm is used for the attack path planning of the air traffic control system network threat knowledge graph, specifically including: The path search strategy of the bidirectional ant colony algorithm is: The forward ant colony uses the ant colony algorithm to determine the path, determines the next moving direction according to the pheromone concentration on the path, and uses the roulette method to select the host of the next transfer path to obtain multiple candidate paths; The reverse ant colony selects the candidate path with the highest transfer probability as the local best path and uses the local best path as the next path.

[0007] As a further improvement of the present invention, during the movement of the forward ant colony and the reverse ant colony, pheromones are respectively released and the pheromone concentration on the path is updated, and the next moving direction is determined according to the pheromone concentration on the path; When the forward ants and the reverse ants meet, evaluate the quality of the path according to the pheromone concentration and update the pheromone; through the sharing and update of the pheromone, further find the path with the highest pheromone concentration as the optimal path from the starting host node to the target host node.

[0008] As a further improvement of the present invention, the bidirectional ant colony algorithm also performs path crossover optimization operations on the two paths, specifically including: When there are no identical host nodes in the two paths, select the path with the minimum total cost as the path after crossover; When there are multiple common path hosts in the two paths, select the path host in the middle position as the crossover point; cut the path according to the crossover point and recombine to generate a new path; calculate the cost corresponding to the new path and select the path with the minimum total cost.

[0009] As a further improvement of the present invention, according to the behaviors and rewards of the forward ant colony and the reverse ant colony, use the PPO algorithm to update the pheromone concentration on each path, specifically including: update the pheromone in the bidirectional ant colony algorithm according to the behaviors and rewards of the ants; the pheromone update method is:

[0010]

[0011]

[0012] In the formula, t represents the number of iterations, Denotes the pheromone concentration between path point i and path point j at time t, is the pheromone evaporation coefficient, E is the path adjustment factor, means that path (i, j) is part of the path of the nth ant; Q is the pheromone constant, L n is the path cost of the nth ant, L best is the total cost of the optimal path; Denotes the pheromone concentration between path point i and path point j at time t + 1; Denotes the pheromone contribution of the nth ant to the path between path point i and path point j at time step t; is the best path; Denotes the sum of the pheromone increments of all ants n to path (i, j) at time step t; On the best path, the pheromone increment between path point i and path point j is is the pheromone contribution of the optimal path to path (i, j).

[0013] As a further improvement of the present invention, the PPO algorithm takes maximizing the cumulative reward as the objective function to optimize the pheromone update strategy of the bidirectional ant colony algorithm; the update rule of the PPO algorithm is:

[0014] In the formula, is the objective function of the PPO algorithm; is the expected value for all time steps t; are the parameters of the policy network; is the advantage function, representing the advantage value after performing the action; is the empirical advantage value; is the constraint parameter in the PPO algorithm, used to control the amplitude of policy update; The function is used to correct the update amplitude; Denotes truncating the advantage function by

[0015] As a further improvement of the present invention, the behaviors and reward situations of the ants in the forward ant colony and the reverse ant colony are obtained through the state space, action space, and reward function; The state space is:

[0016] The dynamic space is:

[0017] The reward function is:

[0018] In the formula, s is the state space, x and y are respectively the coordinates on the two-dimensional plane of the current position of the ant, f is the feature vector describing the surrounding environment information, p is the pheromone concentration vector describing the pheromone distribution; represents the change amount of the i th pheromone, i = 1, 2,..., n; R(s,a) is the reward function, indicating the reward for executing the action s in the state a ; J(s) is the objective function, used to evaluate the quality of the current state space s ; is the weight parameter for pheromone update, is the sum of the influences of all actions on the state change, and n is the number of actions.

[0019] In a second aspect, the present invention provides an optimized system for the air traffic control system network attack path, used to implement the above-mentioned optimized method for the air traffic control system network attack path, including: A target host node determination module, used to determine the target host node of the network attack, and obtain the directed path between host nodes and the corresponding attack cost according to the node network information and vulnerability information in the air traffic control system network threat knowledge graph; A path planning module, taking the minimum attack cost as the path planning goal, and using a bidirectional ant colony algorithm to plan the attack path of the air traffic control system network threat knowledge graph. The ant colonies in the bidirectional ant colony algorithm include a forward ant colony and a reverse ant colony; the forward ant colony starts from the starting host node, and the reverse ant colony starts from the target host node simultaneously; An optimal path search module, according to the behaviors and reward situations of the forward ant colony and the reverse ant colony, updates the pheromone concentration on each path using the PPO algorithm, and updates the next action of the ant colony according to the pheromone concentration until the optimal attack path from the starting host node to the target host node is obtained.

[0020] In a third aspect, the present invention provides a computer-readable storage medium storing one or more programs, the one or more programs including instructions which, when executed by a computing device, cause the computing device to execute the above-mentioned method for optimizing the network attack path of an air traffic control system.

[0021] In a fourth aspect, the present invention provides a computing device, comprising: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include steps for executing the method for optimizing the network attack path of the air traffic control system described above.

[0022] The beneficial effects of the present invention are as follows: The present invention provides a method for optimizing the network attack path of an air traffic control system. In the present invention, the forward ant colony starts from the starting host node, and the reverse ant colony starts from the target host node simultaneously. The ant colonies in both directions search in parallel. The two-way search divides the search space into two parts and advances from the starting point and the end point in both directions, effectively avoiding the redundant calculations that may occur during single-direction search, thereby greatly shortening the search time and improving the overall search efficiency. During the search process, the forward ant colony and the reverse ant colony do not operate independently, but maintain a close pheromone communication and update mechanism. By sharing the path information discovered during the search process, the two ant colonies can learn from each other and cooperate to optimize, further accelerating the discovery and optimization process of potential attack paths. In order to more intelligently guide the search behavior of the ant colonies, the present invention introduces the PPO algorithm. This algorithm dynamically adjusts the pheromone update strategy according to the behavior performance of the ants during the search process and the rewards obtained. By precisely regulating the pheromone concentration, the PPO algorithm can more accurately reflect the quality of the paths and balance the exploration and exploitation relationship of the pheromone. This not only avoids the ants converging to the local optimal solution prematurely but also can make full use of the known best path information to guide the ant colonies to move towards more promising search directions.

[0023] In addition, the present invention also provides a quantitative basis for evaluating the risks and feasibility of different attack paths by accurately calculating the attack cost of each host node. This measure enables penetration testers to more comprehensively understand the actual situation of the attack paths, thereby formulating more scientific and effective attack strategies.

[0024] Furthermore, the two-way search strategy of the present invention divides the search space into two parts. The two-way retrieval can reduce the size of the search space. Compared with the one-way search, the two-way search can find high-quality paths faster and improve the search efficiency. The forward ant colony uses the roulette wheel method for path selection, which has a certain degree of randomness and can explore more potential paths, thereby avoiding falling into local optima. The reverse ant colony selects the path with the highest transition probability as the local best path, which can ensure that the reverse ant colony can quickly find high-quality paths. The two-way retrieval can find high-quality paths while ensuring the search efficiency.

[0025] Furthermore, the present invention enables the ant colony to quickly find high-quality paths through the sharing and updating of pheromones. When the forward ant colony and the reverse ant colony meet, the evaluation and pheromone update of the meeting path can further accelerate the convergence process and shorten the search time.

[0026] Furthermore, the present invention also generates better paths through the crossover operation on two paths. The crossover operation also introduces a mechanism for path recombination, which can explore more potential path combinations and improve the global search ability. The present invention also effectively reduces the attack cost through the crossover optimization operation by comparing and selecting the path with the minimum total attack cost.

[0027] Furthermore, by using the PPO algorithm to update the pheromone concentration, the two-way ant colony algorithm can more effectively guide the path search of ants and improve the efficiency and quality of path search. The PPO algorithm can more accurately evaluate the impact of ant behavior on rewards through policy gradients, thereby more effectively updating the pheromone concentration. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0029] Figure 1 It is a schematic flow chart of the method for optimizing the network attack path of the air traffic control system of the present invention.

[0030] Figure 2 It is a schematic diagram in the graph structure centered on ADS-B assets of the present invention.

[0031] Figure 3 It is of the present invention Figure 2 Partial schematic diagram in

[0032] Figure 4 is a comparison chart of the optimal path cost between different algorithms and this algorithm of the present invention varying with the number of iterations; (a) is the algorithm comparison result of the ACO-PSO algorithm and this method; (b) is the algorithm comparison result of the DVRPTW-ACO algorithm and this method; (c) is the algorithm comparison result of the HG-ACO algorithm and this method; (d) is the algorithm comparison result of the A*-ACO algorithm and this method; (e) is the algorithm comparison result of the IACO algorithm and this method.

[0033] Figure 5 It is a comparison result chart of the optimal path planning of different algorithms and this method algorithm of the present invention.

[0034] Figure 6 It is a schematic structural diagram of the electronic device of the present invention. Detailed implementation manners

[0035] In order to make the purpose and technical solutions of the present invention clearer and easier to understand. The following further describes the present invention in detail with reference to the drawings and embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0036] Term explanation: PPO (Proximal Policy Optimization): Proximal Policy Optimization algorithm.

[0037] BACO (Bi-directional Ant Search Strategy): Bi-directional Ant Search Strategy.

[0038] The technical solutions of the present invention will be clearly and completely described below with reference to the drawings and specific embodiments. Among them, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0039] Embodiment 1 As Figure 1 - Figure 2 shown, this embodiment provides a method for optimizing the network attack path of an air traffic control system. By introducing two ant groups, a forward one and a reverse one, starting from the starting point and the ending point respectively, through the communication and update of pheromones, an optimal path connecting the starting point and the ending point is finally found. The advantage of this algorithm is that it can reduce the search space, speed up the path planning speed, realize the automatic exploration of the target network by ants and the utilization of host vulnerabilities in the air traffic control system, and find the optimal attack path. The following are the specific implementation manners.

[0040] First, determine the target host node of the network attack, and obtain the directed paths between host nodes and the corresponding attack costs according to the node network information and vulnerability information in the air traffic control system network threat knowledge graph.

[0041] In this embodiment, the problem of penetration testing attack path planning is abstracted as an ant pathfinding problem, and the node network information and vulnerability information in the air traffic control system network threat knowledge graph are formally described using the Planning Domain Definition Language (PDDL).

[0042] In this embodiment, the network connection relationship of the host assets in the air traffic control system is transformed into a directed acyclic graph according to the entity relationship of the network threat knowledge graph. The optimal attack path between host assets in the network threat knowledge graph is selected based on the BACO algorithm.

[0043] Furthermore, the host assets mainly include host identifiers, running service lists, and interface information. The host identifier is uniquely determined by the asset name, corresponding to the Name attribute of the host asset ontology in the air traffic control system; the running service list represents the services running on the host asset, corresponding to the Services attribute of the host asset ontology in the air traffic control system; the interface represents the interface list of the host asset, corresponding to the Interfaces attribute of the host asset ontology in the air traffic control system.

[0044] Secondly, with the minimum attack cost as the path planning goal, a bidirectional ant colony algorithm is used for the attack path planning of the air traffic control system network threat knowledge graph. The ant colony in the bidirectional ant colony algorithm includes a forward ant colony and a reverse ant colony; the forward ant colony starts from the starting host node, and the reverse ant colony starts from the target host node simultaneously.

[0045] In this embodiment, the minimum attack cost is used as the planning goal of the attack path. The attack cost is mainly determined by the vulnerability information of the host. In this embodiment, the vulnerability is abstractly represented by an action, believing that the host can be captured after performing a vulnerability exploitation action on a certain host or a specific service, ignoring the privilege escalation in the actual process. And define e ij =<acts, cost> , indicating i The host can attack acts the host through vulnerability exploitation j , cost is the attack cost. The attack cost cost is jointly determined by the high-risk degree of the vulnerability s and the time-consuming of vulnerability exploitation t .

[0046] Define the selection of the minimum cost as the attack cost of this host. With the maximum value and the minimum value as the boundaries, the time required for the attack t is standardized, and the calculation of the vulnerability attack cost cost i is as follows:

[0047] In the formula, is the cost of the i-th sample, s i represents a vulnerability i 's CVSS rating, t i represents a vulnerability i 's time required for an attack, S and T respectively represent the set of ratings and the set of attack times of all vulnerabilities, k represents an adjustment factor, with a value range of (0, 1). When k is 0.5, cost i takes values between (0, 10), is the minimum value of the set of attack times, is the maximum value of the set of attack times, is the maximum value in the set of ratings of all vulnerabilities, is the minimum value in the set of ratings of all vulnerabilities.

[0048] In this embodiment, a two-way ant colony algorithm is adopted to plan the attack path. The two-way ant colony includes a forward ant colony and a backward ant colony. The forward ant colony starts from the starting host node, determines the path using the conventional ant colony algorithm, determines the next moving direction according to the pheromone concentration on the path, and selects the host node of the next transfer path using the roulette method. For the forward ant colony, when using the roulette method to select the host of the next transfer path, the forward ant colony can explore unknown areas, gradually expand the search space outward to enhance the global search ability, and thus obtain multiple candidate paths.

[0049] The backward ant colony starts simultaneously from the target host node, selects the candidate path with the highest transfer probability as the local optimal path, and uses the local optimal path as the host of the next transfer path. Prioritize the local optimal path to improve the quality of the search path. The backward search searches from the target node to the starting node, reduces the number of search nodes, avoids efficiency degradation, and ensures that the search path is different from the forward search, thereby reducing the repetition rate. This two-ant colony search method combines the advantages of forward and backward searches, improving the search efficiency and path quality.

[0050] In addition, the forward ants and backward ants in this embodiment share path information by exchanging pheromones. Each ant selects the next moving direction according to the pheromone concentration. During the movement, the ants release pheromones and update the pheromone concentration on the path. The path with a higher pheromone concentration is more likely to be selected, which helps the ant colony gradually focus on better paths during the search process. When two ants meet, they evaluate the quality of the path based on the current pheromone concentration and update the pheromone. Through the transmission and update of pheromones, the ant colony gradually focuses on the path with a higher pheromone concentration after multiple iterations, and then finds the optimal path. The two-way ant colony algorithm effectively optimizes the path planning process and improves the search efficiency and path quality by introducing the collaborative work of two ant colonies.

[0051] The process specifically includes: (1) Initializing the pheromone concentration on all possible paths; (2) The forward ants and backward ants start from the starting node and the target node respectively, and select paths according to the pheromone concentration and heuristic information; (3) The two-way ants release pheromones respectively during the movement and update the pheromone concentration on the path; (4) When the forward ants and backward ants meet, evaluate the pheromone concentration of the meeting path and update the pheromone; (5) Repeat steps (1) to (4) until the termination condition is met (such as reaching the maximum number of iterations or finding a satisfactory solution), and select the path with the highest pheromone concentration as the optimal path.

[0052] The two-way ant colony algorithm also performs path crossover optimization operations on the two paths during the attack path planning process, specifically including: When there are no identical host nodes in the two paths, select the path with the minimum total attack cost as the path after crossover; When there are multiple common path hosts in the two paths, select the path host in the middle position as the crossover point; Cut the paths according to the crossover point and recombine them to generate new paths; Calculate the attack cost corresponding to the new paths and select the path with the minimum total attack cost.

[0053] Specifically, the principle of path crossover optimization is as follows: Let the first path L 1 and the second path L 2 both have the same host nodes, that is, the host nodes f p = b q :

[0054]

[0055] The convergence of the crossover point performs a cutting process on the original first path and the second path:

[0056]

[0057] If the following conditions are met:

[0058] Then it is divided into new paths , otherwise ; Similarly, if:

[0059] Then the new path , otherwise . The updated path .

[0060] Among them, is the first path, is the 0th host node on the first path, is the 1st host node on the first path, is the pth host node on the first path, is the sth host node on the first path. is the second path, is the 0th host node on the second path, is the 1st host node on the second path, is the qth host node on the second path, is the tth host node on the second path. is the path total cost from the node corresponding to the first path to the node , is the path total cost from the node corresponding to the second path to the node . is the path total cost from the node corresponding to the updated first path to the node , is the path total cost from the node corresponding to the updated second path to the node .

[0061] If there are no identical host nodes in the two paths, directly select the path with the minimum total cost as the crossed path. If there are multiple common path hosts in the two paths, in order to enhance the effect of cross optimization, select the path host in the middle position as the crossing point. First, obtain the list of common host nodes of the two paths , among which , among which, as iWith the increase of , the length of the host list increases. The closer the small descriptions near the starting host are, the larger the value of i is.

[0062] Finally, according to the behaviors and rewards of the forward ant colony and the reverse ant colony, the PPO algorithm is used to update the pheromone concentration on each path. According to the pheromone concentration, the next action of the ant colony is updated until the optimal attack path from the starting host node to the target host node is obtained.

[0063] When the ants complete the search and update the pheromone concentration on the path, in order to avoid the reverse ant colony converging to the local optimal solution prematurely, resulting in a decline in the algorithm's exploration ability, this embodiment uses the PPO algorithm to update the pheromone concentration on each path, balance the pheromone updates of the forward ant colony and the reverse ant colony, and improve the search performance of the algorithm and the optimization of the final result. The specific process of using the PPO algorithm to update the pheromone concentration on each path includes: First, the state space, action space, and reward function are used to describe the position and state of the ants in the ant colony in the search space.

[0064] Among them, the position and environmental state of the ants in the search space are represented as the state space of the PPO algorithm. This state space includes the current position of the ants, the information of the surrounding environment, and the distribution of pheromones, etc. The state of the ants in the search space is represented as a state vector:

[0065] Among them, x and y are the coordinates of the current position of the ants, f is the feature vector describing the information of the surrounding environment, p is the pheromone concentration vector describing the distribution of pheromones.

[0066] In the bidirectional ant colony algorithm, pheromones are used to guide the movement of the ants in the ant colony in the search space. The update of pheromones is regarded as a strategy, and the level of pheromones is used as the action space in the PPO algorithm. The action space can be represented as an action vector, which includes the change amount of pheromones. Therefore, the action space is represented as:

[0067] In the formula, represents the change amount of the i th pheromone.

[0068] The reward function is calculated according to the objective function and the update of pheromones. By designing the reward function, it can prompt the ants to move towards the optimization goal and update the pheromones. The reward function is:

[0069] In the formula,R(s,a) is the reward function, representing the reward for executing the action s in the state a ; J(s) is the objective function, used to evaluate the quality of the current state s ; is the weight parameter for pheromone update.

[0070] The PPO algorithm is used to update the movement strategy of the ants. By maximizing the reward function, it guides the update of the strategy, enabling the ants to move more effectively in the search space. The goal of the PPO algorithm is to optimize the strategy by maximizing the cumulative reward, while ensuring a moderate update amplitude through constraints. The update rule of the PPO algorithm is expressed in the following form:

[0071] where is the loss function of the PPO algorithm; is the expected value for all time steps t; are the parameters of the policy network; is the advantage function, representing the advantage value after executing the action; is the empirical advantage value; is the constraint parameter in the PPO algorithm, used to control the update amplitude of the strategy; The function is used to correct the update amplitude; represents truncating the advantage function ;

[0072] According to the behavior and reward of the ants, the pheromone in the bidirectional ant colony algorithm is updated to guide the next action of the ants.

[0073] The pheromone update method is as follows:

[0074]

[0075]

[0076] where t represents the number of iterations, represents the pheromone concentration between path point i and path point j at time t, is the pheromone evaporation coefficient, E is the path adjustment factor, indicates that the path (i, j) is part of the path of the nth ant, Q is the pheromone constant, L n is the path cost traveled by the nth ant, Lbest is the total cost of the optimal path; represents the pheromone concentration between path point i and path point j at time t + 1; represents the pheromone contribution of the nth ant to the path between path point i and path point j at time step t; is the best path; represents the sum of the pheromone increments of all ants n for the path (i, j) at time step t; is on the best path; the pheromone increment between path point i and path point j ; is the pheromone contribution of the optimal path to the path (i, j). By adopting the way of updating pheromone, it can avoid the ants accumulating part of the path pheromone too fast.

[0077] This embodiment also conducts experiments on this method to verify the performance and effect of the algorithm. In order to better verify the effect of this algorithm and exclude the influence of algorithm randomness, this embodiment also uses other algorithms to compare with this method to evaluate the performance and effect of this method. Five algorithms, namely Improved Ant Colony Optimization (IACO), Ant Colony Optimization Particle Swarm Optimization (ACO - PSO), Hybrid Genetic Ant Colony Optimization (HG - ACO), A* Search Algorithm Ant Colony Optimization (A* - ACO), and Ant Colony Optimization for the Dynamic with Time Windows (DVRPTW - ACO), are mainly used to compare with the method of this embodiment. Statistical analysis after 20 runs of the six algorithms is used for evaluation. The number of iterations is set to 200, and the total number of 30 ants is the same. The experimental scenario in this embodiment selects the network composed of ADS - B (Automatic Dependent Surveillance - Broadcast) as the central node, as shown in Figure 2 , Figure 3 . Among them, Figure 3 is Figure 2 's partial structure diagram, composed of Figure 3It can be seen that the map includes multiple nodes (i.e., entities in the host assets), and the lines connecting the nodes are used to represent the dimensional relationships between the two nodes (such as belong to, threat, exploit, etc.). In actual situations, a node has different dimensional relationships with multiple nodes, and the Figure 2 and Figure 3 in this embodiment are only simple illustrations. Figure 2 、 Figure 3 are only simple illustrations.

[0078] Figures 4(a) to 4(e) and Table 1 respectively show the variation of the optimal path cost of the PPO-BACO, IACO, ACO-PSO, HG-ACO, A*-ACO, and TW-ACO algorithms with the number of iterations.

[0079] Table 1: Comparison results of path optimization for different algorithms

[0080] According to the data in Table 1, it can be seen that the six algorithms of IACO, ACO-PSO, HG-ACO, A*-ACO, DVRPTW-ACO, and PPO-BACO can all reach the optimal planned path in the end. Among them, compared with other algorithms, the PPO-BACO algorithm proposed in this embodiment shows advantages such as fast path planning speed, relatively small optimal path cost and average path cost. Among them, the number of iterations of the optimal path is the least compared with other algorithms, and thus shows less average running time and average path cost. The PPO algorithm optimizes the pheromone update scheme and guides the update of pheromones from the initial moment, thus significantly improving the path-finding quality of ants and enabling the algorithm to quickly approach the optimal solution within fewer iterations. As can be seen from Figures 4, Figure 5 it can be seen that the curve volatility of the IACO and HG-ACO algorithms is relatively high because after these two algorithms fall into the local optimal solution, ants need to continuously iterate and correct the path, resulting in an increase in the amount of calculation and a significant increase in time consumption, and they may fall into the local optimal solution in large-scale problems. In the A*-ACO and DVRPTW-ACO algorithms, the number of updates is less, but the path quality is increased through cross-optimization operations, and the search space is gradually optimized during the iteration process to improve the quality and efficiency of the final path. DTW-ACO is only applicable to the path planning of time series data. In contrast, the PPO-BACO algorithm combines the advantages of deep learning and heuristic optimization, plus a simple reverse ant path-finding strategy, and can quickly find the optimal path within fewer iterations.

[0081] In this embodiment, Figures 4, Figure 5In it, the abscissa Iteration represents the number of iterations of the optimal path, and the ordinate Optimal Path Cost represents the cost of the optimal path. Figure 4(a) shows the comparison results between the ACO-PSO algorithm and the PPO-BACO algorithm (i.e., ACO-PSO VS PPO-BACO Algorithm), Figure 4(b) shows the comparison results between the DVRPTW-ACO algorithm and the PPO-BACO algorithm (i.e., DVRPTW-ACO VS PPO-BACO Algorithm), Figure 4(c) shows the comparison results between the HG-ACO algorithm and the PPO-BACO algorithm (i.e., HG-ACO VS PPO-BACO Algorithm), Figure 4(d) shows the comparison results between the A*-ACO algorithm and the PPO-BACO algorithm (i.e., A*-ACO VS PPO-BACO Algorithm), and Figure 4(e) shows the comparison results between the IACO algorithm and the PPO-BACO algorithm (i.e., IACO VS PPO-BACO Algorithm).

[0082] In summary, in this method, a bidirectional ant colony algorithm is used for path planning. Two ant colonies in both directions are introduced, starting from the starting point and the ending point respectively. Through the communication and update of pheromones, an optimal path connecting the starting point and the ending point is finally found. It solves the problems existing in the classical ant colony algorithm, such as the lack of guidance for ants to find paths, resulting in low path planning quality and causing ants to get stuck in local areas. The advantages of this algorithm are that it can reduce the search space, speed up the path planning speed, realize the automatic exploration of the target network by ants and the utilization of vulnerabilities in the host of the air traffic control system, and find the optimal attack path. In addition, in this embodiment, by using a proxy objective function to update pheromones more stably, the limitations of the early policy gradient method are solved. By combining the PPO algorithm in the BACO algorithm, the reinforcement learning ability of the PPO strategy and the path finding optimization ability of the BACO algorithm are combined, reducing the search space, speeding up the path planning speed, realizing the automatic exploration of the target network by ants and the utilization of vulnerabilities in the host of the air traffic control system, and finding the optimal attack path. Attack path discovery can help better discover possible network attack behaviors, thus better ensuring the safety of air transportation.

[0083] Embodiment 2 This embodiment provides an air traffic control system network attack path optimization system for implementing the air traffic control system network attack path optimization method in Embodiment 1. The system includes: A target host node determination module that determines the target host node of the network attack, and obtains the directed path between host nodes and the corresponding attack cost according to the node network information and vulnerability information in the air traffic control system network threat knowledge graph; In this embodiment, the host assets mainly include host identifiers, running service lists, and interface information. The host identifier is uniquely determined by the asset name, corresponding to the Name attribute of the host asset ontology in the air traffic control system; the running service list represents the services running on the host asset, corresponding to the Services attribute of the host asset ontology in the air traffic control system; the interface represents the interface list of the host asset, corresponding to the Interfaces attribute of the host asset ontology in the air traffic control system.

[0084] The path planning module takes the minimum attack cost as the path planning goal and uses the bidirectional ant colony algorithm to plan the attack path of the air traffic control system network threat knowledge graph. The ant colony in the bidirectional ant colony algorithm includes a forward ant colony and a reverse ant colony; the forward ant colony starts from the starting host node, and the reverse ant colony starts from the target host node simultaneously; The best path search module updates the pheromone concentration on each path using the PPO algorithm according to the behaviors and rewards of the forward ant colony and the reverse ant colony, and updates the next actions of the ant colony according to the pheromone concentration until the optimal attack path from the starting host node to the target host node is obtained.

[0085] Among them, the path planning module takes the minimum attack cost as the planning goal of the attack path. The attack cost is mainly determined by the vulnerability information of the host. In this embodiment, the actions on the vulnerabilities are abstractly represented, and it is considered that the host can be captured after performing the vulnerability exploitation action on a certain host or a specific service, ignoring the privilege escalation in the actual process. And define e ij =< acts, cost> which means i The host can attack acts host j through vulnerability exploitation, cost and cost is the attack cost. The attack cost s is jointly determined by the high-risk degree of the vulnerability t and the time-consuming of vulnerability exploitation.

[0086] Define the selection of the minimum cost as the attack cost of this host. Taking the maximum value and the minimum value as the boundaries, standardize the time required for the attack t , and the vulnerability attack cost cost i is calculated as follows:

[0087] In the formula, is the cost of the i-th sample, s i represents the i CVSS rating of the vulnerability, t iIndicates vulnerability i The time required for an attack of S and T respectively represent the set of ratings of all vulnerabilities and the set of attack times, k represents the adjustment factor, with a value range of (0, 1). When k is 0.5, cost i takes values between (0, 10), is the minimum value of the set of attack times, is the maximum value of the set of attack times, is the maximum value in the set of ratings of all vulnerabilities, is the minimum value in the set of ratings of all vulnerabilities.

[0088] In this embodiment, the bidirectional ant colony includes a forward ant colony and a reverse ant colony. The forward ant colony starts from the starting host node, determines the path using the conventional ant colony algorithm, determines the next moving direction according to the pheromone concentration on the path, and selects the host node of the next transfer path using the roulette method. For the forward ant colony, when using the roulette method to select the host of the next transfer path, the forward ant colony can explore the unknown area, gradually expand the search space outward to enhance the global search ability, and thus obtain multiple candidate paths.

[0089] The reverse ant colony starts simultaneously from the target host node, selects the candidate path with the highest transfer probability as the local best path, and uses the local best path as the host of the next transfer path. Prioritize the selection of the local best path to improve the quality of the search path. The reverse search starts from the target node to the starting node, reducing the number of search nodes, avoiding a decrease in efficiency, and ensuring that the search path is different from the forward search, thereby reducing the repetition rate. This dual ant colony search method combines the advantages of forward and reverse searches, improving the search efficiency and path quality.

[0090] In addition, the forward ants and reverse ants in this embodiment share path information by exchanging pheromones. Each ant will select the next moving direction according to the pheromone concentration. During the movement, the ants release pheromones and update the pheromone concentration on the path. The path with a higher pheromone concentration is more likely to be selected, which helps the ant colony gradually concentrate on a better path during the search process. When two ants meet, they will evaluate the quality of the path according to the current pheromone concentration and update the pheromone. Through the transmission and update of pheromones, the ant colony gradually concentrates on the path with a higher pheromone concentration after multiple iterations, and thus finds the optimal path. The bidirectional ant colony algorithm effectively optimizes the path planning process by introducing the collaborative work of two ant colonies, improving the search efficiency and path quality.

[0091] The process specifically includes: (1) Initializing the pheromone concentration on all possible paths; (2) Forward ants and backward ants start from the starting node and the target node respectively, and select paths according to the pheromone concentration and heuristic information; (3) The two-way ants release pheromones respectively during the movement process, and update the pheromone concentration on the path; (4) When the forward ants and the backward ants meet, evaluate the pheromone concentration of the meeting path and update the pheromone; (5) Repeat steps (1) to (4) until the termination condition is met (such as reaching the maximum number of iterations or finding a satisfactory solution), and select the path with the highest pheromone concentration as the optimal path.

[0092] During the attack path planning process, the two-way ant colony algorithm also performs path crossover optimization operations on the two paths, specifically including: When there are no identical host nodes in the two paths, select the path with the minimum total attack cost as the path after crossover; When there are multiple common path hosts in the two paths, select the path host at the middle position as the crossover point; Cut the paths according to the crossover point and recombine them to generate new paths; Calculate the attack cost corresponding to the new paths, and select the path with the minimum total attack cost.

[0093] Specifically, the principle of path crossover optimization is as follows: Let the first path L 1 and the second path L 2 both have the same host nodes, that is, the host nodes f p = b q :

[0094]

[0095] The convergence of the crossover point performs a splitting process on the original first path and the second path:

[0096]

[0097] If it satisfies:

[0098] Then it is divided into new paths , otherwise ; Similarly, if:

[0099] Then the new path , otherwise . The updated path .

[0100] Among them, is the first path, is the 0th host node on the first path, is the 1st host node on the first path, is the pth host node on the first path, is the sth host node on the first path. is the second path, is the 0th host node on the second path, is the 1st host node on the second path, is the qth host node on the second path, is the tth host node on the second path. is the path cost from the node corresponding to the first path to the node The total path cost, is the path cost from the node corresponding to the second path to the node The total path cost. is the total path cost from the node corresponding to the updated first path to the node The total path cost, is the total path cost from the node corresponding to the updated second path to the node The total path cost.

[0101] If there are no identical host nodes in the two paths, directly select the path with the minimum total cost as the crossed path. If there are multiple common path hosts in the two paths, in order to enhance the effect of cross-optimization, select the path host at the middle position as the crossover point.

[0102] The optimal path search module is used to update the pheromone concentration on each path by adopting the PPO algorithm, balance the pheromone update of the forward ant colony and the backward ant colony, and improve the search performance of the algorithm and the optimization of the final result, specifically including: First, use the state space, action space, and reward function to describe the position and state of the ants in the ant colony in the search space.

[0103] Among them, represent the position and environmental state of the ants in the search space as the state space of the PPO algorithm. This state space includes the current position of the ants, information about the surrounding environment, and the distribution of pheromones, etc. Represent the state of the ants in the search space as a state vector:

[0104] Among them, x and y are the coordinates of the current position of the ants, fis a feature vector describing the surrounding environment information, p is a pheromone concentration vector describing the pheromone distribution.

[0105] In the bidirectional ant colony algorithm, pheromones are used to guide the movement of ants in the ant colony within the search space. Considering the update of pheromones as a strategy, and the level of pheromones as the action space in the PPO algorithm. The action space can be represented as an action vector, which includes the change amount of pheromones. Therefore, the action space is represented as:

[0106] where, represents the change amount of the i th pheromone.

[0107] The reward function is calculated based on the objective function and the update of pheromones. By designing the reward function, it can prompt the ants to move towards the optimization goal and update the pheromones. The reward function is:

[0108] where, R(s,a) is the reward function, indicating the reward for executing the action s in the state a ; J(s) is the objective function, used to evaluate the quality of the current state s ; is the weight parameter for pheromone update.

[0109] The PPO algorithm is used to update the movement strategy of ants. By maximizing the reward function to guide the update of the strategy, it enables the ants to move more effectively in the search space. The goal of the PPO algorithm is to optimize the strategy by maximizing the cumulative reward, while ensuring an appropriate update amplitude through constraints. The update rule of the PPO algorithm is expressed in the following form:

[0110] where, is the loss function of the PPO algorithm; is the expected value for all time steps t; are the parameters of the policy network; is the advantage function, indicating the advantage value after executing the action; is the empirical advantage value; is the constraint parameter in the PPO algorithm, used to control the update amplitude of the strategy; The function is used to correct the update amplitude; is for truncating the advantage function by the operation.

[0111] Update the pheromone in the bidirectional ant colony algorithm according to the behavior and reward of ants to guide the next action of ants.

[0112] The pheromone update method is as follows:

[0113]

[0114]

[0115] In the formula, t represents the number of iterations, represents the pheromone concentration between path points i and j at time t, is the pheromone evaporation coefficient, E is the path adjustment factor, means that the path (i, j) is part of the path of the nth ant; Q is the pheromone constant, L n is the path cost of the nth ant, L best is the total cost of the optimal path; represents the pheromone concentration between path points i and j at time t + 1, represents the pheromone contribution of the nth ant to the path between path points i and j at time step t, is the best path; represents the sum of the pheromone increments of all ants n to the path (i, j) at time step t; is the pheromone increment between path points i and j on the best path, is the pheromone contribution of the optimal path to the path (i, j). The way of updating pheromone can avoid the ants accumulating part of the path pheromone too fast.

[0116] Example 3 In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory). This storage medium is the memory device in the terminal device and is used to store programs and data. It should be noted that the computer-readable storage medium here not only includes the built-in storage medium of the terminal device, but also can include the extended storage medium supported by the terminal device. It can be any tangible medium that can contain or store programs, and these programs can be used by or in conjunction with an instruction execution system, apparatus, or device. The computer-readable storage medium provides a storage space for storing the operating system of the terminal. In addition, one or more instructions suitable for the processor to load and execute are stored in this storage space, and these instructions can be one or more computer programs (including program codes).

[0117] More specifically, examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories, optical fibers, portable compact disc read-only memories, optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0118] The computer-readable storage medium can also include data signals propagated in the baseband or as part of a carrier wave, in which the readable program code is carried. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than the readable storage medium, and this medium can send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted by any appropriate medium, including but not limited to wireless, wired, optical cable, etc., or any suitable combination of the above.

[0119] The program code for performing the operations of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages (such as Java, C++ etc.), and conventional procedural programming languages (such as the "C" language or similar programming languages). The program code can be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network or a wide area network, or can be connected to an external computing device (for example, through an Internet service provider using an Internet connection).

[0120] The processor can load and execute one or more instructions stored in a computer-readable storage medium to implement the corresponding steps of the method for optimizing the network attack path of the air traffic control system described in Embodiment 1 above.

[0121] Embodiment 4 As Figure 6 shown, another embodiment provided by the present invention provides a terminal device, specifically a computer device 60. Its main components include: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. The processor 61 is responsible for executing the computer program to implement the method for optimizing the network attack path of the air traffic control system. The memory 62 is used to store the computer program and other programs and data required for the operation of the device. The computer program 63 runs on the processor 61 to implement the method for optimizing the network attack path of the air traffic control system. To avoid repetition, details are not elaborated here.

[0122] The computer device 60 can be various forms of computing devices, including but not limited to: desktop computers, notebooks, palmtop computers, cloud servers, and other computing devices.

[0123] The processor 61 can be a central processing unit, or other general-purpose processors, central processors, graphics processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays, or other programmable logic devices, discrete gate or transistor logic devices, data processing logics based on quantum computing, discrete hardware components, etc. The general-purpose processor is an unprivileged processor or any conventional processor.

[0124] The memory 62 can be an internal storage unit of the computer device 60, such as the hard disk or memory of the computer device 60. The memory 62 can also be an external storage device of the computer device 60, such as a plug-in hard disk equipped on the computer device 60, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. The memory 62 is not only used to store the computer program, but also used to store other programs and data required for the operation of the device, as well as temporarily store the data that has been output or will be output.

[0125] Any reference to a memory, database, or other medium used in the embodiments provided by this application may include at least one of non-volatile and volatile memories. Non-volatile memory may include read-only memory, magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory, magnetoresistive random access memory, ferroelectric memory, phase change memory, graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory or dynamic random access memory, etc.

Claims

1. An optimization method for the network attack path of an air traffic control system, characterized in that Including: Determine the target host node of the network attack. According to the node network information and vulnerability information in the air traffic control system network threat knowledge graph, obtain the directed paths between host nodes and the corresponding attack costs; Taking the minimum attack cost as the path planning goal, use the bidirectional ant colony algorithm to plan the attack path of the air traffic control system network threat knowledge graph. The ant colony in the bidirectional ant colony algorithm includes a forward ant colony and a reverse ant colony; the forward ant colony starts from the starting host node, and the reverse ant colony starts from the target host node simultaneously; According to the behaviors and reward situations of the forward ant colony and the reverse ant colony, use the PPO algorithm to update the pheromone concentrations on each path, and update the next actions of the ant colony according to the pheromone concentrations until the optimal attack path from the starting host node to the target host node is obtained.

2. The method for optimizing the network attack path of the air traffic control system according to claim 1, wherein Using the bidirectional ant colony algorithm to plan the attack path of the air traffic control system network threat knowledge graph specifically includes: The path search strategy of the bidirectional ant colony algorithm is: The forward ant colony uses the ant colony algorithm to determine the path. According to the pheromone concentration on the path, determine the next moving direction, and use the roulette method to select the path host for the next transfer to obtain multiple candidate paths; The reverse ant colony selects the candidate path with the highest transfer probability as the local best path, and uses the local best path as the path host for the next transfer.

3. The method for optimizing the network attack path of the air traffic control system according to claim 2, wherein During the movement of the forward ant colony and the reverse ant colony, they respectively release pheromones and update the pheromone concentrations on the paths, and determine the next moving direction according to the pheromone concentrations on the paths; When the forward ants and the reverse ants meet, evaluate the quality of the path according to the pheromone concentration and update the pheromone; through the sharing and update of the pheromone, further find the path with the highest pheromone concentration as the optimal path from the starting host node to the target host node.

4. The method for optimizing the network attack path of the air traffic control system according to claim 1, characterized in that The bidirectional ant colony algorithm also performs path crossover optimization operations on the two paths, specifically including: When there are no identical host nodes in the two paths, select the path with the minimum total attack cost as the path after crossover; When there are multiple common path hosts in the two paths, select the path host in the middle position as the crossover point; cut the path according to the crossover point and recombine to generate a new path; calculate the attack cost corresponding to the new path, and select the path with the minimum total attack cost.

5. The method for optimizing the network attack path of the air traffic control system according to claim 4, wherein According to the behaviors and reward situations of the forward ant colony and the reverse ant colony, use the PPO algorithm to update the pheromone concentrations on each path, specifically including: Update the pheromone in the bidirectional ant colony algorithm according to the behaviors and reward situations of the ants. The pheromone update method is: wherein, t represents the number of iterations, represents the path point at time t, i and the path point j the pheromone concentration between them, is the pheromone evaporation coefficient, E is the path adjustment factor, means that the path (i, j) is part of the path of the nth ant; Q is the pheromone constant, L n is the path cost of the nth ant, L best is the total cost of the optimal path; represents the pheromone concentration between the path point i and the path point j at time t + 1; represents the pheromone contribution of the nth ant to the path point i and the path point j at time step t; is the best path; represents the sum of the pheromone increments of all ants n to the path (i, j) at time step t; is the pheromone increment between the path point i and the path point j on the best path, is the pheromone contribution of the optimal path to the path (i, j).

6. The method for optimizing the network attack path of the air traffic control system according to claim 5, wherein The PPO algorithm takes maximizing the cumulative reward as the objective function to optimize the pheromone update strategy of the bidirectional ant colony algorithm. The update rule of the PPO algorithm is: In the formula, is the objective function of the PPO algorithm; is the expected value for all time steps t; are the parameters of the policy network; is the advantage function, representing the advantage value after executing an action; is the empirical advantage value; is the constraint parameter in the PPO algorithm, used to control the magnitude of policy update; The function is used to correct the update magnitude; represents truncating the advantage function by the operation.

7. The method for optimizing the network attack path of an air traffic control system according to claim 6, wherein The behaviors and reward situations of the ants in the forward ant colony and the reverse ant colony are obtained through the state space, action space, and reward function; The state space is: The dynamic space is: The reward function is: where s is the state space, x and y are the coordinates on the two-dimensional plane of the current position of the ant, f is the feature vector describing the surrounding environment information, p is the pheromone concentration vector describing the pheromone distribution; represents the change amount of the i -th pheromone, i = 1, 2, … n; R(s,a) is the reward function, indicating the reward for executing the action s in the state a ; J(s) is the objective function, used to evaluate the quality of the current state space s ; is the weight parameter for pheromone update, is the sum of the influences of all actions on the state change, and n is the number of actions.

8. An air traffic control system network attack path optimization system for implementing the air traffic control system network attack path optimization method according to any one of claims 1 to 7, characterized in that, Including: A target host node determination module, used to determine the target host node of the network attack. According to the node network information and vulnerability information in the air traffic control system network threat knowledge graph, obtain the directed paths between host nodes and the corresponding attack costs; A path planning module, which takes the minimum attack cost as the path planning goal and uses the bidirectional ant colony algorithm to plan the attack path of the air traffic control system network threat knowledge graph. The ant colony in the bidirectional ant colony algorithm includes a forward ant colony and a reverse ant colony; the forward ant colony starts from the starting host node, and the reverse ant colony starts from the target host node at the same time; An optimal path search module, according to the behaviors and reward situations of the forward ant colony and the reverse ant colony, uses the PPO algorithm to update the pheromone concentration on each path, and updates the next actions of the ant colony according to the pheromone concentration until the optimal attack path from the starting host node to the target host node is obtained.

9. A computer-readable storage medium storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform the air traffic control system network attack path optimization method according to any one of claims 1 to 7.

10. A computing device, characterized in that, Comprising: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include those for performing the air traffic control system network attack path optimization method according to any one of claims 1 to 7.

Images